Service Discovery & Routing

Service Discovery & Routing

1. Client-Side vs. Server-Side Discovery

In a microservices world, service instances scale up and down constantly, meaning their IP addresses change. Service Discovery is how services find each other.

Client-Side Discovery

The client is responsible for looking up available instances of a service from a Service Registry (e.g., Netflix Eureka, Consul) and deciding which one to call using a load-balancing algorithm.

  • How it works: Service A asks the Registry, “Where is Service B?” The Registry returns a list of IPs. Service A chooses one and calls it directly.

  • Pros: Fewer network hops; the client can make smart, application-specific load-balancing decisions.

  • Cons: You must implement discovery logic in every language/framework your services use; couples the client to the registry.

Server-Side Discovery

The client makes a request to a central Load Balancer (e.g., AWS ALB, NGINX). The load balancer queries the Service Registry and proxies the request to an available instance.

  • How it works: Service A calls http://service-b. The Load Balancer intercepts it, asks the Registry where Service B lives, and forwards the request.

  • Pros: Simple client code; completely abstracts discovery away from the application.

  • Cons: Introduces an extra network hop; the load balancer can become a bottleneck or a single point of failure if not scaled properly.

2. Service Mesh (Istio, Linkerd) vs. API Gateway

While both handle traffic, they focus on entirely different boundaries and problems.

Feature API Gateway (e.g., Kong, Apisix, AWS API Gateway) Service Mesh (e.g., Istio, Linkerd)
Traffic Direction North-South (External client to internal services). East-West (Service-to-service internal traffic).
Primary Goal Expose APIs safely to the outside world. Connect, secure, and monitor internal microservices.
Key Features Rate limiting, authentication (OAuth/JWT), API monetization, request transformation. Mutual TLS (mTLS) encryption, internal circuit breaking, traffic splitting (canary deployments), distributed tracing.

What a Service Mesh Solves

Before service meshes, if you wanted secure internal communication (mTLS) or resilient retries between Service A and Service B, you had to bake that logic directly into your application code using heavy libraries (like Netflix Hystrix).

If you had a polyglot architecture (Java, Go, Node.js), you had to rewrite that logic for every language. A Service Mesh solves this by offloading network concerns entirely from the application code to the infrastructure layer.

3. Sidecar Pattern

The Sidecar Pattern is the architectural foundation of a Service Mesh.

Just like a sidecar attached to a motorcycle, this pattern attaches a helper container to your main application container. They share the same lifecycle, network namespace (localhost), and storage.

How it applies to Service Mesh:

Instead of your application handling network security, retries, and metrics, a tiny network proxy (like Envoy in Istio) runs as a sidecar next to your application container.

  1. Inbound Traffic: Trapped by the sidecar proxy, decrypted (via mTLS), inspected, and handed off to the app via localhost.

  2. Outbound Traffic: The app sends a request to localhost, the sidecar intercepts it, handles service discovery, encrypts it, and sends it across the wire to the destination sidecar.

Why do it? Your application code stays completely pure. It doesn’t know it’s in a mesh, it doesn’t know about mTLS, and it doesn’t care. It just handles business logic.