How an API Gateway Works — One Door, Many Rooms

The problem: every client talks to every service

You split your backend into microservices — users, orders, payments — and it feels great until you look at it from the client's side. Now the web app, the mobile app, and that IoT device in the field each need to know where every service lives, speak its dialect, and handle its quirks.

Worse, every service now has to solve the same problems on its own:

• Who is this caller? (authentication)
• Are they allowed to do this? (authorization)
• Are they hammering us? (rate limiting)
• CORS, TLS, logging, metrics… again and again, in every codebase.

And you can never refactor: the moment you split orders into two services, every client in the wild breaks.

Enter the API Gateway

An API gateway is a single entry point that sits between your clients and your services — a reverse proxy with opinions. Clients only ever see one host (api.yoursite.com); the gateway decides what happens to each request and which service ultimately answers it.

Here is the whole idea in motion. The amber packet is a request, the green one is a response — try the failure scenarios too:

The gateway rejects bad requests at the edge — invalid tokens and rate-limited clients never touch your services.

What happens to a request, step by step

1. TLS termination. The gateway accepts the HTTPS connection, so internal traffic can stay simple. Certificates live in exactly one place.
2. Authentication. It validates the JWT or API key once, at the edge. Services behind it can trust the identity the gateway forwards (e.g. as headers or a signed token) instead of re-implementing auth six times.
3. Rate limiting. A counter (usually a token bucket) per client or API key. Over quota? 429 Too Many Requests — the request dies here, cheaply, before it costs you a database query.
4. Routing. The gateway matches the path, method, or headers against its route table — /orders/** goes to the orders service — and picks a healthy upstream instance (that's load balancing, for free).
5. Transformation. It can rewrite paths, inject headers, translate protocols (HTTP outside, gRPC inside), or aggregate several service calls into one response so the mobile app makes one round trip instead of five.
6. Response. On the way back it can cache, compress, and strip internal headers before the client sees anything.

The cross-cutting freebies

Because every request flows through it, the gateway is the natural home for concerns you'd otherwise scatter everywhere:

• Observability — one place to log every request and measure every latency.
• Resilience — retries, timeouts and circuit breakers around flaky upstreams.
• Safe deployments — canary releases by routing 5% of traffic to v2.
• Caching — serve hot GETs without waking the service at all.

What it looks like in practice

Most gateways are just configuration. Here's a route in YARP (the .NET reverse proxy) — path prefix in, upstream cluster out:

{
  "Routes": {
    "orders": {
      "ClusterId": "orders-cluster",
      "Match": { "Path": "/orders/{**rest}" },
      "RateLimiterPolicy": "per-user",
      "AuthorizationPolicy": "authenticated"
    }
  },
  "Clusters": {
    "orders-cluster": {
      "Destinations": {
        "a": { "Address": "http://orders-svc:8080/" }
      }
    }
  }
}

Popular picks: NGINX / Kong / Envoy if you self-host, AWS API Gateway / Azure API Management as managed services, YARP or Ocelot in .NET land, and service meshes like Istio use the same pattern (Envoy) inside the cluster.

What NOT to put in a gateway

The gateway is infrastructure, not a service. Two classic mistakes:

Business logic in the gateway. The moment "route the request" becomes "compute the discount", you've built a monolith with extra steps. Keep the pipes dumb and the endpoints smart.

And remember the trade-offs: it's an extra network hop (a bit of latency) and a single point of failure — run it replicated behind a load balancer, never as one instance.

TL;DR

One front door. Authenticate, throttle and route at the edge; keep the services behind it simple, trusting, and free to change shape without anyone outside noticing.