Kubernetes for Microservices

Table of Contents

Kubernetes for Microservices — Architect-Level Deep Dive

1. Service Discovery

In a microservices architecture, service discovery answers “given a logical service name, how do I find a healthy instance to call right now” — and Kubernetes solves most of this natively, which is precisely why teams migrating from VM-based microservices (where they ran Eureka, Consul, or ZooKeeper for this exact purpose) often find they can retire that entire layer.

The native mechanism, end to end:

A Service object provides the stable logical name; CoreDNS resolves <service>.<namespace>.svc.cluster.local to the Service’s ClusterIP (covered in depth in the networking module).
Behind that VIP, EndpointSlices track the actual healthy pod IPs backing the Service, continuously updated as pods pass/fail readiness probes — this is the live, dynamic membership list that makes “discovery” actually work, not just naming.
Client-side resolution is what actually happens on each call: the calling pod resolves DNS once (or per the OS resolver’s TTL/caching behavior), gets the ClusterIP, and kube-proxy/eBPF dataplane handles the actual load-balancing across endpoints at the connection/packet level — the calling application code itself does zero discovery logic; it just makes an HTTP call to a hostname like any other.

Where this native model has genuine limits, and why service meshes/registries still come up in microservices conversations:

No client-side load-balancing intelligence — kube-proxy/iptables-based routing is largely random or round-robin at the connection level, with no latency-awareness, no outlier detection (continuing to send traffic to a slow-but-technically-healthy pod), no weighted traffic shifting. This gap is exactly what a service mesh’s sidecar proxy (Envoy, via Istio/Linkerd) adds — L7-aware load balancing with active health checking, retries, and traffic-shaping decisions made per-call, not just per-connection.
Multi-cluster service discovery isn’t native at all — Kubernetes Service discovery is fundamentally single-cluster. Cross-cluster discovery requires either a service mesh’s multi-cluster federation (Istio’s multi-cluster mesh, Linkerd multi-cluster), or external mechanisms (Route53-based discovery, AWS Cloud Map, or the Multi-Cluster Services API — ServiceExport/ServiceImport CRDs, increasingly relevant as architectures span multiple EKS clusters for blast-radius isolation or multi-region DR).
Discovery for things outside the cluster (a legacy on-prem service, an external SaaS dependency) — ExternalName Services provide a thin abstraction (a CNAME-style DNS alias to an external hostname), letting in-cluster code address external dependencies through the same Service-naming convention, which keeps the discovery model consistent even at the cluster boundary.

Architect’s framing worth stating plainly: for a single-cluster microservices estate, native K8s Service discovery genuinely is sufficient — reaching for a separate service registry (Eureka/Consul) on top of Kubernetes is usually redundant unless there’s a specific multi-cluster or hybrid (VM+K8s) discovery requirement that native Services and the Multi-Cluster Services API don’t cover.

2. API Gateway Patterns

The API Gateway sits at the edge of the microservices estate, providing a single entry point that handles cross-cutting concerns so individual services don’t each have to reimplement them.

Core responsibilities an architect should be able to list without hesitation:

Routing — path/host-based dispatch to the correct backend service (the same fundamental job as Ingress, but typically with richer L7 capability).
Authentication/authorization — JWT validation, OAuth2 token introspection, API key checking — centralized once at the edge rather than reimplemented in every microservice.
Rate limiting & quota enforcement — per-client/per-API-key throttling.
Request/response transformation — protocol translation (REST-to-gRPC), header injection, response shaping for different client types (mobile vs web).
Observability injection — this is the entry point where trace context (from the tracing module) typically gets initiated if the originating client doesn’t already propagate it.

Kubernetes-specific implementation spectrum, worth contrasting explicitly:

Ingress + Ingress Controller (NGINX, ALB Controller) — covers basic L7 routing and TLS termination, but the annotation-based extension model is the limiting factor for true API-Gateway-grade features (sophisticated rate limiting, request transformation, fine-grained auth policies) — you end up bolting on controller-specific annotations that aren’t portable and don’t compose well.
Gateway API (Gateway/HTTPRoute/GRPCRoute) — the modern, more expressive successor explicitly designed to close this gap, with a role-oriented split: infrastructure teams own GatewayClass/Gateway (the actual load balancer/proxy infrastructure), application teams own HTTPRoute (routing rules for their own service) — a much cleaner separation of concerns than Ingress’s single flat object, and the direction the ecosystem has been consolidating toward.
Dedicated API Gateway products (Kong, Apigee, AWS API Gateway itself fronting EKS via VPC Link, Ambassador/Emissary) — reached for when you genuinely need developer-portal features, API monetization/billing, sophisticated plugin ecosystems, or multi-cluster/multi-cloud gateway consistency that the K8s-native options don’t provide out of the box.
Service mesh ingress gateway (Istio Gateway, distinct from core Gateway API though increasingly implementing it) — when the mesh is already handling east-west (service-to-service) traffic, using its own gateway component for north-south (edge) traffic gives policy consistency (the same AuthorizationPolicy/mTLS model applies at the edge and internally) rather than having two disconnected policy systems.

Architect’s decision lens worth stating in a review: the question isn’t “which gateway is best” in the abstract — it’s “how much of this cross-cutting behavior genuinely needs centralizing at the edge, versus how much belongs in the mesh (service-to-service) layer.” A common, sound pattern in 2026: Gateway API + a gateway implementation (Envoy Gateway, Istio Gateway, or cloud ALB) for edge concerns, paired with a service mesh for internal east-west concerns — treating them as complementary layers rather than picking one to do everything.

3. Sidecar Pattern

The sidecar pattern attaches a helper container to the main application container within the same pod, sharing the pod’s network namespace (so localhost calls reach the sidecar) and optionally shared volumes — used to extract cross-cutting concerns out of application code without modifying that code.

The canonical microservices use case — service mesh data plane: in Istio/Linkerd, every pod gets an Envoy (or linkerd-proxy) sidecar auto-injected (via a mutating webhook, triggered by a namespace label like istio-injection: enabled). All inbound/outbound traffic for the application container is transparently redirected through this sidecar via iptables rules set up by an init container — the application code makes a normal HTTP call to localhost or the target service name, completely unaware a proxy is intercepting it.

What this buys you, concretely, without touching app code:

mTLS between every service-to-service call, automatically, with cert rotation handled by the mesh’s control plane (Istio’s istiod) — this is the single biggest “do this once, not in every microservice” win.
Fine-grained traffic management — retries, timeouts, circuit breaking (next section), traffic splitting for canary — all configured declaratively via CRDs (VirtualService, DestinationRule) rather than library code embedded in each service.
Uniform telemetry — the sidecar emits consistent L7 metrics (request rate, error rate, latency — exactly the RED metrics from the observability module) and trace spans for every service automatically, regardless of what language/framework that service is written in — this “polyglot consistency” is a major draw for organizations with mixed Java/Go/Python/Node microservices, where getting consistent observability via per-language libraries would mean maintaining N different instrumentation approaches.

The cost side an architect must weigh, not just the benefits:

Per-pod resource overhead — every pod now runs at least one extra container, multiplying CPU/memory baseline cluster-wide; at thousands of pods, this is a genuinely material capacity-planning line item, not a rounding error.
Added latency per hop — every call now traverses two proxies (caller’s outbound sidecar, callee’s inbound sidecar) — typically single-digit milliseconds with Envoy, but worth explicitly modeling for latency-sensitive paths.
Operational complexity — mesh control plane upgrades, sidecar version skew during rolling mesh upgrades, debugging “is this a sidecar issue or an app issue” — a genuinely new failure-mode category introduced by adopting the pattern.
Ambient mesh (Istio’s newer sidecar-less data plane model, using per-node ztunnel proxies instead of per-pod sidecars) is worth naming as the direction addressing exactly the resource-overhead concern — worth a mention if discussing mesh adoption strategy in 2026, as it’s increasingly the recommended starting point for new Istio adoptions specifically to avoid the per-pod sidecar tax.

Beyond service mesh — other legitimate sidecar uses worth naming: log-shipping sidecars (tailing app log files the app can’t redirect to stdout), config-reload sidecars (watching a mounted ConfigMap/Secret and signaling the main container to reload without a full restart), and the ambassador pattern (a sidecar proxying outbound calls to an external dependency, centralizing retry/auth logic for that one dependency specifically) — all instances of the same underlying principle: extract a concern into a co-located, independently-deployable helper rather than embedding it in the primary container’s codebase.

4. Circuit Breaker Pattern

The circuit breaker exists to stop a failing downstream dependency from cascading into a failure of the calling service — instead of every caller continuing to hammer a struggling dependency with requests (each one timing out slowly, exhausting the caller’s own thread/connection pool), the breaker “trips” and fails fast once failure thresholds are crossed.

Three-state model worth being able to describe precisely:

Closed — normal operation, requests pass through, failures are counted.
Open — failure threshold exceeded; requests are rejected immediately without even attempting the call, for a configured cool-down period — this is what protects both the caller (no wasted resources waiting on a doomed call) and the struggling callee (no continued load while it’s trying to recover).
Half-open — after the cool-down, a limited number of trial requests are let through; success transitions back to Closed, continued failure trips back to Open.

Where this actually gets implemented in a Kubernetes microservices stack — and the architectural choice worth being explicit about:

Library-level (in application code) — Resilience4j (Java), Polly (.NET), Hystrix (legacy, now in maintenance mode/largely superseded) — circuit-breaking logic lives inside each service’s codebase. Works, but means every service in every language needs its own library and its own consistent configuration — a genuine maintenance burden in a polyglot estate.
Mesh-level (sidecar-enforced) — Istio’s DestinationRule with outlierDetection and connectionPool settings implements circuit-breaking at the Envoy sidecar, completely outside application code:

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: payment-service
spec:
  host: payment-service
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 100
      http:
        http1MaxPendingRequests: 50
        maxRequestsPerConnection: 10
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 30s
      baseEjectionTime: 30s
      maxEjectionPercent: 50

This is the pattern most architects should advocate for in a genuinely polyglot microservices estate — circuit-breaking becomes a platform-level, uniformly-configured concern rather than N different library implementations with N different default thresholds that drift out of sync over time.

Worth naming the related-but-distinct concept of outlierDetection specifically as “passive health checking” — it’s evaluating actual request outcomes (5xx responses, connection failures) to eject a specific endpoint from the load-balancing pool temporarily, which is subtly different from (and complementary to) the active health-checking that readiness probes already provide — a pod can be probe-ready (process is up, accepting connections) while still returning elevated error rates under load, and outlier detection is what catches that gap that probes structurally can’t see.

5. Canary Deployments

Canary deployment shifts a small percentage of live traffic to a new version while the majority continues hitting the stable version, observing real production metrics on that small slice before deciding whether to progressively increase exposure or roll back.

Two distinct mechanisms for actually achieving traffic split, worth distinguishing clearly:

Replica-ratio canary (the “poor man’s canary”) — run two Deployments (app-stable with 9 replicas, app-canary with 1 replica) behind the same Service selector — Kubernetes’ inherently round-robin-ish Service load balancing then approximates a 90/10 split purely through replica count ratio. Crude (no precise percentage control, scales awkwardly — wanting exactly 5% canary traffic on a 10-pod deployment means non-integer replica counts), but achievable with zero additional tooling.
L7 weighted traffic splitting (the real mechanism) — a service mesh’s VirtualService (Istio) or an Ingress controller supporting traffic-split annotations defines an exact percentage split independent of replica count — canary can run as a single pod and still receive precisely 5% of traffic by weight, decoupling “how many pods are running” from “what fraction of traffic do they see”:

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: app
spec:
  hosts: [app]
  http:
    - route:
        - destination: {host: app, subset: stable}
          weight: 95
        - destination: {host: app, subset: canary}
          weight: 5

Progressive delivery tooling — this is where the analysis loop gets automated, and it’s the piece that elevates “canary” from a manual, eyeballed process into something genuinely production-grade:

Argo Rollouts and Flagger (referenced earlier in the rolling-updates module) automate the entire canary lifecycle: incrementally shift weight (5% → 20% → 50% → 100%) on a defined schedule, query Prometheus at each step for error-rate/latency regression against the stable baseline, and automatically roll back if metrics breach defined thresholds — without a human watching a dashboard and manually adjusting weights.
This closes precisely the gap named in the rolling-updates discussion: vanilla Deployment rollouts have no automated failure detection; canary-plus-progressive-delivery tooling is the answer specifically for microservices architectures where “did this version actually behave correctly under real production load” needs to be answered with real telemetry, not just “did the readiness probe pass.”

Worth flagging the dependency this creates explicitly in any review: a meaningful canary analysis loop is only as good as the metrics it’s querying — this is precisely why the observability module (Prometheus/RED metrics, proper instrumentation) is a prerequisite for genuinely automated canary deployments, not a parallel/unrelated concern. Teams that adopt Argo Rollouts without first having solid per-service error-rate/latency metrics end up with canary automation that has nothing meaningful to gate on.

6. Blue-Green Deployments

Blue-green takes the opposite philosophical approach from canary: instead of gradually shifting a fraction of traffic, you run two complete, fully-scaled environments (Blue = current production, Green = new version) side by side, validate Green fully (often via internal/synthetic traffic, not real users yet), then cut traffic over atomically — typically 100% at once, not incrementally.

Kubernetes implementation pattern, mechanically:

Two Deployments, app-blue and app-green, each fully scaled to production capacity, with distinct labels (version: blue / version: green).
A single Service’s selector determines which one is “live” — cutover is a single kubectl patch (or GitOps commit) changing the selector, e.g., from version: blue to version: green — traffic shifts essentially instantly because the Service’s Endpoints are recalculated immediately upon the selector change, no gradual ramp.
The old (Blue) environment stays running, unmodified, for a defined soak period after cutover — this is the actual point of the pattern: if something’s wrong with Green, rollback is just flipping the selector back, instantaneous, with zero redeployment/rebuild needed, since Blue never went away.

Genuine tradeoffs versus canary, worth being explicit about rather than presenting blue-green as strictly better or worse:

Resource cost: blue-green needs 2x full production capacity running simultaneously during the transition window (both environments fully scaled) — a real, non-trivial cost, especially for large/expensive deployments — versus canary’s incremental resource ramp (the canary replica count grows only as confidence grows).
Blast radius on a bad release: blue-green is all-or-nothing — if Green has a subtle bug that only manifests under specific traffic patterns, 100% of users hit it the instant cutover happens, whereas canary’s gradual exposure limits the bad-version blast radius to whatever percentage hadn’t yet been ramped up.
Database/state compatibility is the sharp edge that catches teams off guard with blue-green specifically: because both versions can theoretically need to coexist (during validation, and especially during any rollback window), schema changes must be backward-and-forward compatible for the full overlap period — you can’t blue-green a deployment that includes a breaking database migration without separately solving that compatibility problem (expand-contract migration pattern), and this exact issue applies to canary too but is more acute in blue-green because the full-traffic cutover means there’s no gradual-exposure cushion if a stale-schema assumption breaks.
Session/connection draining: existing in-flight requests/long-lived connections to Blue at the moment of cutover need graceful handling — this is where preStop hooks and terminationGracePeriodSeconds (allowing Blue to finish in-flight work before being scaled down) matter operationally, same mechanism as graceful pod termination generally, just with higher stakes given the instant, complete nature of the cutover.

Architect’s synthesis for when to reach for which, worth stating explicitly in a design review: canary is the better default for stateless, high-traffic services where gradual, metric-gated exposure genuinely reduces risk and resource doubling is wasteful; blue-green is the better fit for deployments where partial/mixed-version traffic is actively dangerous or meaningless (e.g., a release that’s genuinely all-or-nothing in its correctness — certain monolith-adjacent services, or releases where running two versions simultaneously under real mixed traffic for an extended canary window would itself create inconsistent user-facing behavior) and where the 2x resource cost during a short transition window is acceptable. Many mature platforms support both via the same tooling (Argo Rollouts implements blue-green as a first-class strategy alongside canary, using the same Rollout CRD), making this a per-service policy decision rather than a cluster-wide architectural commitment.