With the adoption of micro-services, we need to tackle hosts of issues associated with remote calls and networking - because what used to be an in-process function call now becomes a RPC call that need to be handled by a service which needs to be discovered. Service discovery has its own issues - among others, the most important being able to discover services that are active. Once services are discovered - we need to handle uniform spread of requests among discovered service instances. Traffic encryption becomes another issue that we need to handle once a call goes out over the wire from one micro-service to another.
Another very obvious requirement in a cluster of micro-services is the need to monitor and trace requests. Without this requirement being taken care, it is difficult how to figure out how services are performing a distributed setup of a cluster of micro-services.
While there still many more issues that we need to take care of in a micro-services' setup - what should be our approach on a high-level to tackle them?
Among others - one approach could be to develop client side libraries - which handle issues of service discovery, load balancing(retry, timeout, circuit breaking etc) and others. This was the approach Netflix Eureka/ribbon/zuul stack had proposed. Eureka client acted as a service proxy - eureka server acting as a service registry and ribbon providing client side load balancing and zuul handling request routing.
While the library approach works - there are few things to consider while we embark on the library journey:
- Business code now get entangled with infrastructure code.
- We are sticking are head out either as an one language shop or we are undertaking quite a complicated job of developing/maintaining client libraries in multiple programming languages.
Is there an easy way to tackle the issues associated with micro-services? Yes! That is what out-of-process proxies like envoy and linkerd provide. All ready existing proxies like Nginx and HA-PROXY etc are adding support for capabilities similar to those provided by envoy proxy. We will discuss linkerd in a later post - here we will talk about envoy proxy.
Envoy's website talks of envoy being a edge and service service proxy - this means we can use envoy for north-south as well as east-west traffic. Envoy is written in modern C++ and most of requests handling run concurrently in lock free code. This make envoy very fast.
Envoy has lot of goodies to offer - it's out of process architecture straight away boosts developer productivity. The network and myriad of associated issues go away instantly - letting the developer focus on his business problem. Envoy being out of process - provides a huge advantage - we can develop our services in any language of our choice and necessity.
As mentioned earlier - envoy provides many load balancing features - automatic request retries, circuit breaking, request timeouts, request shadowing, rate limit etc.
With envoy's traffic routing features, we can easily do rolling upgrade of services, blue/green and canary deployment etc.
Envoy provides wire level observability of request traffic and native support for distributed tracing.
Envoy supports HTTP/1.1, HTTP/2 & gRPC - it transparently switches between HTTP/1.1 and HTTP/2.
One very important aspect of envoy proxy is that - while envoy can be configured statically - yet it provides robust APIs for dynamic configuration. What is more - envoy can patched and upgraded without shutting it down via what is called "Hot-Restart". We would configure some of envoy's features in upcoming posts - but before that following concepts about envoy would help.
Envoy proxy concepts:
Downstream: A downstream host connects to Envoy, sends requests, and receives responses.
Upstream: An upstream host receives connections and requests from Envoy and returns responses.
Listeners: They are the addresses where envoy process listens for incoming connections. For example,
0.0.0.0:9090. There can multiple listeners in an envoy process.
Filter chains: Each listener in an envoy process can be configured with filter chains - where each chain consists of one or more filters. Filter chains are selected based on incoming request and some matching criteria.
Routes: Based on matching criteria - requests are delegated to be handled by back-end clusters.
Clusters: Named collection of back-ends called endpoints. Requests are load balanced among the cluster endpoints.
Endpoints: Endpoints are the delegates which handles requests. They are part of cluster definition.
In the next post - we would install envoy and look at traffic routing. Stay tuned.