Registries: Use Cases for API Management and Microservices
By Phil Wilkins
Microservices and new 3rd-generation API Management capabilities are very natural technical partners. A (micro)service provides the execution logic for a single capability and 3rd generation API Management provides the means to control the exposure of each service to the outside world, and potentially between the (micro)services as well. Luis Weir's article, 3rd-Generation API Management, explains the evolution of the API Management capability, and what the 3rd generation offers that makes API Management mesh well with (micro)services. For this article, though, we are going to explore the role of registries in a (micro)services environment and ultimately their relationship to API Management. We aren't going to dive deeply into any specific solution, although we will reference some of the better known options, and provide links to where you can dig in further.
You'll note I have referred to microservices as (micro)services. This is very deliberate, and possibly contentious.Microservices are typically associated with certain technologies such as Docker and lightweight app containers such as Tomcat, but from my view point if you're going to be a purist about microservices, then (as with SOA) you need to think about the design paradigm and the principles—NOT the technologies. It is with this view point we often have to work as, unless you're fortunate to be working in a services organization that is very enlightened or just plain fortunate, the decisions and constraints you have to work with will mean you need to make some pragmatic decisions. An organization that has made heavy investment in WebLogic licensing isn't likely to be comfortable in giving it up—but this doesn't mean you can't adopt a microservices approach, although certainly you need to mitigate risks that using WebLogic can create when adopting the architecture. Secondly, the concept of registries isn't unique to microservices; in fact, some of these solutions have origins in solutions such as Big Data/Hadoop offerings, for example.
To underline my point, Gartner has started to talk about mini- or micro-apps as a flavor of microservices. The essence of this is the application of microservice principles but in a more pragmatic manner—not every organization needs hyperscaling and super elasticity like the poster children of microservices (e.g., Netflix and Uber). But we all do need the means to easily manage the addressing of services.
To understand the role of a registry, we need to step back and understand some of the ideas underpinning microservices. So let's just recap:
- Scaling and elasticity. This is to say that, as microservices are small contained pieces of functionality, it should be easy to launch additional instances of a service to meet demands in the environment for just that service. The scaling can include reducing the number of instances.
- A solution is made up of many separately deployed microservices. Therefore, no one piece should assume the existence of the others beyond a service contract. Because pieces are discretely deployed, you can't know where the other services will be physically. This separation allows us to change, add, and deploy services independently rather than deploy the entire solution in a monolith.
From these points alone we can see a need to know, firstly, where instance(s) of the services are—implicitly or explicitly, we need to know how many instances there are so we can understand the capacity available. In addition to that, a load balancer is needed. It is important to remember that, because microservices are independently deployed, they may not be running on the same port on every server the load balancer can see. While you could deploy that way, you would effectively be treating all your services as a monolith. We will come back to the question of load balancer and routing.
The role of the registry, as the name suggests, is to keep a register of what services exist and where. As each new microservice starts up, one of its initial actions is to declare its presence to the registry. After this, each service needs to work with the registry to ensure that, when it is shutdown or unexpectedly dies, the registry finds out.
This leads us to two questions:
- Does a registry not represent a single point of failure?
- How do I know where the registry is?
The last thing we want is a single point of failure with the registry as that would undermine the goals we have been seeking to achieve. Fortunately, all the significant registries available incorporate some means to collaborate and share information. There are several common strategies adopted by different registry implementations, which include:
- RAFT: https://raft.github.io/
- PAXOS: http://the-paper-trail.org/blog/consensus-protocols-paxos/
- SWIM: https://prakhar.me/articles/swim/
The second point can be addressed with the microservice having a list of predefined addresses (ideally, DNS-based or Virtual IP); the microservice works through that list, trying to contact a registry until one responds. This model of finding nodes to coordinate with is fairly common--for example, ActiveMQ message brokers when working in a cluster operate this way. The last option, in terms of initiating a communication, is to use a network broadcast. This is less attractive, as many services starting can create a lot of network traffic, but some CORBA brokers used to adopt this approach.
We now know how to register ourselves with a registry and keep it aware of our existence. But this is only half the problem solved. We actually need to understand how to exploit this information so that all the workload is spread amongst all the instances and the network infrastructure routes the load to where the microservice actually resides. There are two core models for using a registry:
- Client-side registry, where the microservices have the knowledge of registries, locate an instance, and then call them to get the address of a service they need.
The client may then cache the reference for a while. This, of course, means load balancing between instances can be problematic, and just bringing on new service instances won't immediately spread the load. The upside of this is that in global distribution scenarios you're likely to find your way to the nearest instant instantiation of microservices without any DNS manipulation. This model is supported by Netflix with its Eureka framework.
- Then there is the server-side registries approach. These registries typically hide behind a load balancer and other network infrastructure. The registry then has a relationship with the load balancer...
- The registry tells the load balancer where the microservice instances are and configures the balancer to spread the load the best way.
- The load balancer is dumb and calls to a registry asking for the address to use for a specific service.
This server-side approach in many respects reflects the truism of microservices: do one thing and do it well.Load balancers do load balancing, registries just manage a register, and microservices care nothing of how their request is serviced—be it another microservice or a monolith. This comes at the cost of reduced performance, and the realization is somewhat dependent upon specific products (e.g., NetScaler or F5 BigIP).
If you are building from a green field and won't need to deal with legacy monoliths, or wish to keep the management of routing within your application domain rather than network infrastructure, then the client-side model is ideal. If you're going to blend microservices with other elements that could be monoliths, or wish to exploit network infrastructure elements such as DNS to help optimal routing, then you may wish to look at a server-side model as it means DNS and network routing can hide the differences. But the alternate option is to establish a proxy component that registers the monolith's end points and fakes the monolith's heartbeat. This is shown in the following diagram.
With the client-side model, the need to have balancing incorporated with the gateway has meant solutions such as SkyDNS and Eureka have effectively become API Load Balancers (to differentiate this from the next evolution, we'll call this model 1st Generation).
So far, we have only really considered what would be a tightly controlled or closed ecosystem of microservices--to put it another way, an environment where we don't need to focus on access controls, throttling, security, monetization and so on. As soon as these become issues, then a firewall or better still an API Gateway needs to be introduced.
If you adopt a client-side registry model with an API Gateway, then you need to very carefully work out how the gateway is positioned, otherwise it becomes easy for services to ignore or by pass the gateway. This means that the benefits the gateway provides in terms of execution insight and control can be lost, as demonstrated in the following diagram.
With a server-side registry it becomes simpler, as the gateway need only sit beside the load balancer, as the following diagram illustrates.
This has led us to what we could describe as the 2nd-Generation API Load Balancer. This approach takes the idea of merging not just the load balancing and registry into a single component, but also the gateway characteristics. This is not dissimilar to the evolution of the network infrastructure solutions, which may have been either firewalls or load balancers but are now a composite solution.
Gone is the overhead of performance cost of communicating between the registry, load balancer and gateway, and the environment becomes very simple in terms of microservices.
Actually, what I have just described as a 2nd-Generation API Load Balancer can actually be part of a 3rd-Generation API Management solution as described in the article by Luis Weir that I mentioned at the start of this article.
A couple of arguments could be used to challenge this approach:
- A microservice purist may argue that a hybrid solution means you're less likely to get the best registry solution or the best load balancer (i.e., go against the point that a service should do only one thing, and do it well), as a hybrid is unlikely to have the sophistication of a good load balancer (think F5 BigIP or NetScaler) and the simplicity and arguably reliability of a registry.
- It may also be argued that the registry is a lot heavier (therefore less throughput) than a pure play load balancer.
Unless the solution becomes core IP to an organization or someone comes to the market with giant-killer product/open source framework, then the optimizations that NetScaler and the F5 kit have are unlikely to be beaten. However, we can win by eliminating a chunk of overhead in several areas. Consider these savings:
- The microservices don't have the overhead of communicating with a registry.
- You reduce the overhead of heartbeats as any service-to-service call will go through the API load balancer (API Manager), which can reset the clock on the heartbeat every time it hears from a service.
- The coordination between load balancer and registry will become simply an in-memory activity, and the registry will become inherently resilient as the load balancing becomes resilient.
- High-performance mechanisms of the load balancer can be exploited by the registry logic. Further, the services require even less framework (particularly on the client side), as I need only signal the API-load balancer on start-up and shutdown.Calling another API is just like calling a web address.
When it comes to throughput, there is the potential cost, but if the gateway functionality is policy-driven, and the cost is only as high as the policy complexity, and if we can separate policies for an external call to an API vs an internal one, then the load can be tailored to be more security sensitive on one side and not the other.
So, if the 2nd-Generation API Load Balancer is a good idea, why doesn't it exist yet?Our answer comes down to the fact that the gateways are still maturing – some gateway solutions are moving towards lightweight ESBs (and reaching a point where I'd agree with the arguments outlined previously) and this is reflected in ThoughtWorks Tech Radar position. But we're seeing initiative moving us in the direction of this kind of need – consider the mechanisms of Feign to simplify the making of APIs visible to a registry. As we move towards the 4th-Generation of APIs (aka everything as an API),where APIs become pervasive, the need for this 2nd generation will grow. To look at it another way, 3rd-Generation API Management has load balancing incorporated into it.
Within the Oracle PaaS ecosystem the API Platform Cloud Service is still young, so features that reflect key demands are being delivered. But it has the means to be extended via an SDK to deliver the capabilities, and the gateway engine part of the API Platform is both highly scalable and has some of its foundations originating from the telco market so is inherently high-performance anyway. Oracle's Application Container Cloud currently has within its platform a 1st-Generation API Load Balancer; I wouldn't be surprised if it adopts a 2nd-Generation model as a key differentiator in the future. The other possibility here is that ACCS could switch to using API Platform CS with the balancing capability if a customer were to adopt both products.
- 3rd Generation API Management, by Luis Weir
- Registry Discovery, by Arun Gupta
- Service-Oriented Architecture (SOA) Definition, by Douglas K. Barry
- OpenFeign (GitHub)
- Oracle API Platform Cloud Service
- Eureka at a glance (GitHub)
- Pattern: Client-side service discovery, by Chris Richardson
- Overambitious API gateways (Thoughtworks Tech Radar)
About the Author
Oracle ACE Associate Phil Wilkins is a Senior Consultant at Capgemini, where he iPaaS, middleware and Oracle technologies. He is co-author of Implementing Oracle Integration Cloud Service (2017, Packt) and frequently contributes content to OTN, UKOUG Oracle Scene, and other publications.