StackPractices
intermediate By Mathias Paulenko

Gateway Routing Pattern

Route requests to multiple backend services through a single entry point that handles cross-cutting concerns.

Overview

The Gateway Routing Pattern places a single entry point in front of your backend services. Rather than letting every service face clients directly, the gateway takes each request and sends it to whichever upstream matches, by path, method, header, or any rule you configure. It also centralizes cross-cutting concerns such as TLS termination, authentication, rate limiting, and logging, so individual services don’t have to.

The pattern comes from the cloud design catalog published by Microsoft, where it sits next to Gateway Offloading and Gateway Aggregation. All three share the same idea: clients should talk to one stable address, and everything behind that address should be free to change. New services can appear, old ones can move or split, and clients never notice because the routing table absorbs the change.

This matters most in microservices architectures, where a single user action often fans out across a dozen services. It pays off during migrations too: peel endpoints off the monolith one route at a time while the public API doesn’t move.

When to Use

Use this pattern when:

  • You’ve got backend services that clients must reach through a single address
  • You need to enforce TLS, authentication, or rate limiting in one place
  • You want to route traffic by URL path, host, or API version without changing clients
  • You’re migrating from a monolith to microservices and need to hide internal changes
  • You need to compose responses from several services or translate between protocols

Skip it when a single backend serves all traffic (a reverse proxy is enough), or when services already live behind a full service mesh that handles internal routing; adding a second routing layer just doubles the hops. The gateway is also the wrong place for business rules; if routing decisions need domain knowledge, that logic belongs in the services or in a Backend for Frontend layer.

One more boundary worth drawing early: the gateway routes requests, it doesn’t orchestrate them. If a client needs data from five services joined into one response, that’s aggregation, a different pattern with different failure modes.

Solution

At its simplest, a gateway is a routing table plus a proxy. This Express setup forwards each path prefix to a different upstream service:

// Minimal gateway route configuration (Express + http-proxy-middleware)
import express from 'express';
import { createProxyMiddleware } from 'http-proxy-middleware';

const app = express();

app.use('/users', createProxyMiddleware({
  target: 'http://users-service:3001',
  changeOrigin: true,
}));

app.use('/orders', createProxyMiddleware({
  target: 'http://orders-service:3002',
  changeOrigin: true,
}));

app.use('/inventory', createProxyMiddleware({
  target: 'http://inventory-service:3003',
  changeOrigin: true,
}));

app.listen(3000, () => console.log('Gateway listening on port 3000'));

The same routing translates directly to NGINX, which is what most production gateways use:

# NGINX location-based routing
server {
  listen 443 ssl;
  server_name api.example.com;

  location /users {
    proxy_pass http://users-service;
  }

  location /orders {
    proxy_pass http://orders-service;
  }

  location /inventory {
    proxy_pass http://inventory-service;
  }
}
flowchart diagram: Clients

Clients only ever see the gateway address. Everything to the right of it can scale, move, or be replaced without a client change.

Explanation

When a request arrives, the gateway runs it through a matching pipeline: it extracts the request attributes (path, host, method, headers), finds the first rule that matches, applies the middleware chain for that route, and forwards the request to the selected upstream. The response travels back through the same gateway, which can rewrite headers, wrap the payload, or cache the result before returning it.

The matching rules are the heart of the pattern. Common strategies:

StrategyRoutes onTypical use
Path/users/*, /orders/*One service per resource domain
Hostapi.example.com, admin.example.comSeparate public and admin APIs
HeaderX-API-Version, AcceptVersioned APIs, content negotiation
MethodGET vs POSTRead replicas vs write primaries
WeightedPercentage splitCanary releases, A/B rollouts

Rules are evaluated in order, so put the most specific routes first. A common bug: declaring /api before /api/admin sends admin traffic to the general API service.

Four responsibilities usually ride along with routing:

  • Load balancing: distribute requests across healthy upstream instances, pulling dead ones out of rotation
  • Security: terminate TLS, validate tokens, and enforce rate limits before traffic reaches the services
  • Observability: attach a correlation ID at the edge so every upstream log and trace shares it; see Health Check Pattern for the health-check side
  • Resilience: timeouts, bounded retries, and circuit breakers to stop a slow service from dragging the whole gateway down

A few edge cases bite in production. Trailing slashes matter: /users and /users/ can match different rules depending on the proxy, so normalize them at the edge. Header case is inconsistent across clients: Node lowercases everything, some Java stacks don’t. Large uploads and SSE/WebSocket streams need the proxy configured to forward bodies and upgrade connections instead of buffering the whole request. And redirects issued by upstreams must have their Location headers rewritten, or clients get bounced to an internal hostname they can’t resolve.

The trade-off is that the gateway itself becomes critical infrastructure. Every request flows through it, so a gateway outage is a total outage. Run at least two instances behind a load balancer, keep the gateway stateless so any instance can serve any request, and watch its latency budget: each middleware in the chain adds milliseconds that every request pays.

Variants

VariantUse caseTrade-off
API GatewayExpose public APIs to external clientsCentralized but can become a bottleneck
Backend for FrontendTailor APIs for web, mobile, or partner clientsAdds a service per client type
Edge GatewayHandle TLS, DDoS, and caching at the network edgeSimplifies origins but adds vendor dependency
Internal GatewayRoute traffic inside a cluster with mTLSKeeps traffic private but overlaps with the mesh

Gateway routing also overlaps with two neighbors. A service mesh routes between services inside the cluster: it’s east-west traffic, while the gateway owns north-south traffic from outside. Many systems run both. A Backend for Frontend goes further than routing: it composes and reshapes responses for one client type, at the cost of one gateway per client. If your routing rules start accumulating response-composition logic, that’s the signal to split out a BFF rather than grow the gateway.

Picking an implementation comes down to who should own the routing layer. NGINX or HAProxy config files work when the route table changes rarely and ops teams already run them. A managed gateway (AWS ALB, Azure Application Gateway, Cloudflare) trades flexibility for someone else’s uptime and DDoS protection. App-level gateways like the Express example fit teams that want routing logic in the same language and deploy pipeline as their services. On Kubernetes, the same pattern shows up as Ingress controllers and the Gateway API, covered in depth in the Kubernetes Ingress guide.

What Works

  • Keep the gateway stateless so it can scale horizontally and survive instance loss
  • Store routing rules in configuration (YAML, a control plane, service discovery), not in code
  • Use health checks so traffic only flows to instances that respond
  • Offload TLS at the gateway to keep certificate handling out of the services
  • Limit gateway logic to cross-cutting concerns; the moment a route needs domain rules, move it
  • Attach a correlation ID at the edge and propagate it to every upstream call

Common Mistakes

  • Putting business logic in the gateway, where it’s hard to test and version
  • Routing every microservice through one under-scaled gateway; that’s the SPOF problem. Run two or more instances with health checks
  • Ignoring timeout and retry settings, which turns a slow upstream into cascading failures
  • Overloading the gateway with heavy transformations; complex reshaping belongs in a BFF
  • Forgetting to validate TLS certificates on upstream connections, which silently disables authentication between hops
  • Ordering rules wrong so a broad pattern like /api swallows traffic meant for a specific route

Advanced Patterns

The static examples above cover the common case. Once traffic grows, four extensions come up repeatedly: dynamic upstream resolution, per-service circuit breaking, transformation, and weighted splits.

Dynamic routing with service discovery

Static route tables go stale as soon as services autoscale or redeploy. Wiring the gateway to a service registry keeps upstreams current: when instances register or deregister, the router re-resolves them per request.

import express from 'express';
import { randomUUID } from 'node:crypto';
import { createProxyMiddleware } from 'http-proxy-middleware';
import { ServiceRegistry } from './service-registry';

class DynamicGateway {
  private registry: ServiceRegistry;
  private app: express.Application;

  constructor(registry: ServiceRegistry) {
    this.registry = registry;
    this.app = express();
  }

  async setupRoutes() {
    // Correlation ID middleware must run before any proxy route.
    this.app.use((req, _res, next) => {
      req.headers['x-request-id'] ??= randomUUID();
      next();
    });

    const services = await this.registry.getAllServices();

    services.forEach(service => {
      this.app.use(service.path, createProxyMiddleware({
        target: `http://${service.instances[0].host}:${service.instances[0].port}`,
        changeOrigin: true,
        router: () => {
          // Round-robin replacement: pick a healthy instance per request.
          const healthy = service.instances.filter(i => i.healthy);
          const pick = healthy[Math.floor(Math.random() * healthy.length)];
          return `http://${pick.host}:${pick.port}`;
        },
        onProxyReq: (proxyReq, req) => {
          proxyReq.setHeader('X-Request-ID', req.headers['x-request-id']);
        },
        onError: (_err, _req, res) => {
          (res as express.Response).status(502).json({ error: 'Bad Gateway' });
        },
      }));
    });
  }

  listen(port: number) {
    this.app.listen(port, () => console.log(`Gateway listening on ${port}`));
  }
}

Circuit breaker per upstream

Wrapping each upstream call in a circuit breaker stops a failing service from exhausting the gateway’s connection pool. The state lives per service, so one dead upstream doesn’t trip the others:

import CircuitBreaker from 'opossum';

const breakerOptions = {
  timeout: 3000,
  errorThresholdPercentage: 50,
  resetTimeout: 30000,
};

class CircuitBreakerGateway {
  private breakers = new Map<string, CircuitBreaker>();

  getBreaker(serviceName: string) {
    if (!this.breakers.has(serviceName)) {
      const breaker = new CircuitBreaker(
        async (url: string, init: RequestInit) => {
          const response = await fetch(url, init);
          if (!response.ok) throw new Error(`HTTP ${response.status}`);
          return response.json();
        },
        breakerOptions,
      );

      breaker.on('open', () => console.log(`Circuit opened for ${serviceName}`));
      breaker.on('halfOpen', () => console.log(`Circuit half-open for ${serviceName}`));
      breaker.on('close', () => console.log(`Circuit closed for ${serviceName}`));

      this.breakers.set(serviceName, breaker);
    }
    return this.breakers.get(serviceName)!;
  }

  async proxyRequest(serviceName: string, path: string, method: string, body?: string) {
    const breaker = this.getBreaker(serviceName);
    return breaker.fire(`http://${serviceName}:3001${path}`, {
      method,
      headers: { 'content-type': 'application/json' },
      body,
    });
  }
}

Request and response transformation

Header injection and response wrapping are safe gateway-level transforms. Protocol translation is heavier. The gRPC block below is a sketch, not runnable code, because it needs a generated client:

import express from 'express';

class TransformGateway {
  private app = express();

  setupTransforms() {
    // Inject headers before routing.
    this.app.use('/api/v1', (req, _res, next) => {
      req.headers['x-api-version'] = 'v1';
      req.headers['x-request-time'] = new Date().toISOString();
      next();
    });

    // Wrap v2 responses in a meta envelope.
    this.app.use('/api/v2', (_req, res, next) => {
      const originalJson = res.json.bind(res);
      res.json = (data: unknown) => originalJson({
        meta: { version: 'v2', timestamp: new Date().toISOString() },
        data,
      });
      next();
    });

    // Sketch: REST to gRPC translation. Wire a generated client here;
    // loadGrpcClient and the mappers come from your proto toolchain.
    this.app.post('/grpc-proxy', async (req, res) => {
      try {
        // const client = loadGrpcClient('users-service');
        // const grpcRes = await client.getUser(mapRestToGrpc(req.body));
        // res.json(mapGrpcToRest(grpcRes));
        res.status(501).json({ error: 'gRPC translation not configured' });
      } catch {
        res.status(500).json({ error: 'gRPC translation failed' });
      }
    });
  }
}

Weighted routing for canary releases

A percentage split sends a small share of traffic to a new version while the rest stays on the stable one. Roll the weight forward as metrics stay green:

# Weighted routing: 10% of traffic to the canary
routes:
  - path: /api/users
    upstreams:
      - service: users-service-v1
        weight: 90
      - service: users-service-v2   # canary
        weight: 10

Pair it with per-client rate limiting (counters in Redis for distributed gateways) and automatic rollback when error rates climb; the canary is only safe if something watches it.

See Also

Frequently Asked Questions

What is the difference between Gateway Routing and an API Gateway?

Routing is the piece that matches requests to upstreams. An API Gateway is the product built around it, so you usually get authentication, rate limiting, transformation, and a developer portal bundled in.

Should the gateway handle retries?

It can retry safe, idempotent requests (GET, PUT with idempotency keys), but be careful with POST and other state-changing operations; a retried charge or order can duplicate work. Always bound retries and add jitter.

Can I use this pattern with serverless functions?

They work fine. Each function registers as an upstream target and gets routed by path or method, the same as a container service. The catch is cold starts: a sleeping function can add seconds, so size your timeout budget for it.

How do I handle authentication at the gateway?

Validate JWTs or API keys at the edge, then hand identity to services as signed headers like X-User-Id and X-Scopes. Services check the signature, which is cheaper and safer than re-parsing tokens on every hop.

Should the gateway cache responses?

Caching GET responses for hot data pays off fast, but respect Cache-Control from upstreams and plan invalidation; stale data served quickly is still stale.

How do I monitor gateway performance?

Start with p50/p99 latency, error rate, and throughput per route, plus upstream health. Distributed tracing with the correlation ID is what ties a gateway span to the service spans behind it.