Gateway Routing Pattern
Route requests to multiple backend services through a single entry point that handles cross-cutting concerns.
Overview
The Gateway Routing Pattern places a single entry point in front of your backend services. Rather than letting every service face clients directly, the gateway takes each request and sends it to whichever upstream matches, by path, method, header, or any rule you configure. It also centralizes cross-cutting concerns such as TLS termination, authentication, rate limiting, and logging, so individual services don’t have to.
The pattern comes from the cloud design catalog published by Microsoft, where it sits next to Gateway Offloading and Gateway Aggregation. All three share the same idea: clients should talk to one stable address, and everything behind that address should be free to change. New services can appear, old ones can move or split, and clients never notice because the routing table absorbs the change.
This matters most in microservices architectures, where a single user action often fans out across a dozen services. It pays off during migrations too: peel endpoints off the monolith one route at a time while the public API doesn’t move.
When to Use
Use this pattern when:
- You’ve got backend services that clients must reach through a single address
- You need to enforce TLS, authentication, or rate limiting in one place
- You want to route traffic by URL path, host, or API version without changing clients
- You’re migrating from a monolith to microservices and need to hide internal changes
- You need to compose responses from several services or translate between protocols
Skip it when a single backend serves all traffic (a reverse proxy is enough), or when services already live behind a full service mesh that handles internal routing; adding a second routing layer just doubles the hops. The gateway is also the wrong place for business rules; if routing decisions need domain knowledge, that logic belongs in the services or in a Backend for Frontend layer.
One more boundary worth drawing early: the gateway routes requests, it doesn’t orchestrate them. If a client needs data from five services joined into one response, that’s aggregation, a different pattern with different failure modes.
Solution
At its simplest, a gateway is a routing table plus a proxy. This Express setup forwards each path prefix to a different upstream service:
// Minimal gateway route configuration (Express + http-proxy-middleware)
import express from 'express';
import { createProxyMiddleware } from 'http-proxy-middleware';
const app = express();
app.use('/users', createProxyMiddleware({
target: 'http://users-service:3001',
changeOrigin: true,
}));
app.use('/orders', createProxyMiddleware({
target: 'http://orders-service:3002',
changeOrigin: true,
}));
app.use('/inventory', createProxyMiddleware({
target: 'http://inventory-service:3003',
changeOrigin: true,
}));
app.listen(3000, () => console.log('Gateway listening on port 3000'));
The same routing translates directly to NGINX, which is what most production gateways use:
# NGINX location-based routing
server {
listen 443 ssl;
server_name api.example.com;
location /users {
proxy_pass http://users-service;
}
location /orders {
proxy_pass http://orders-service;
}
location /inventory {
proxy_pass http://inventory-service;
}
}
Clients only ever see the gateway address. Everything to the right of it can scale, move, or be replaced without a client change.
Explanation
When a request arrives, the gateway runs it through a matching pipeline: it extracts the request attributes (path, host, method, headers), finds the first rule that matches, applies the middleware chain for that route, and forwards the request to the selected upstream. The response travels back through the same gateway, which can rewrite headers, wrap the payload, or cache the result before returning it.
The matching rules are the heart of the pattern. Common strategies:
| Strategy | Routes on | Typical use |
|---|---|---|
| Path | /users/*, /orders/* | One service per resource domain |
| Host | api.example.com, admin.example.com | Separate public and admin APIs |
| Header | X-API-Version, Accept | Versioned APIs, content negotiation |
| Method | GET vs POST | Read replicas vs write primaries |
| Weighted | Percentage split | Canary releases, A/B rollouts |
Rules are evaluated in order, so put the most specific routes first. A common bug: declaring /api before /api/admin sends admin traffic to the general API service.
Four responsibilities usually ride along with routing:
- Load balancing: distribute requests across healthy upstream instances, pulling dead ones out of rotation
- Security: terminate TLS, validate tokens, and enforce rate limits before traffic reaches the services
- Observability: attach a correlation ID at the edge so every upstream log and trace shares it; see Health Check Pattern for the health-check side
- Resilience: timeouts, bounded retries, and circuit breakers to stop a slow service from dragging the whole gateway down
A few edge cases bite in production. Trailing slashes matter: /users and /users/ can match different rules depending on the proxy, so normalize them at the edge. Header case is inconsistent across clients: Node lowercases everything, some Java stacks don’t. Large uploads and SSE/WebSocket streams need the proxy configured to forward bodies and upgrade connections instead of buffering the whole request. And redirects issued by upstreams must have their Location headers rewritten, or clients get bounced to an internal hostname they can’t resolve.
The trade-off is that the gateway itself becomes critical infrastructure. Every request flows through it, so a gateway outage is a total outage. Run at least two instances behind a load balancer, keep the gateway stateless so any instance can serve any request, and watch its latency budget: each middleware in the chain adds milliseconds that every request pays.
Variants
| Variant | Use case | Trade-off |
|---|---|---|
| API Gateway | Expose public APIs to external clients | Centralized but can become a bottleneck |
| Backend for Frontend | Tailor APIs for web, mobile, or partner clients | Adds a service per client type |
| Edge Gateway | Handle TLS, DDoS, and caching at the network edge | Simplifies origins but adds vendor dependency |
| Internal Gateway | Route traffic inside a cluster with mTLS | Keeps traffic private but overlaps with the mesh |
Gateway routing also overlaps with two neighbors. A service mesh routes between services inside the cluster: it’s east-west traffic, while the gateway owns north-south traffic from outside. Many systems run both. A Backend for Frontend goes further than routing: it composes and reshapes responses for one client type, at the cost of one gateway per client. If your routing rules start accumulating response-composition logic, that’s the signal to split out a BFF rather than grow the gateway.
Picking an implementation comes down to who should own the routing layer. NGINX or HAProxy config files work when the route table changes rarely and ops teams already run them. A managed gateway (AWS ALB, Azure Application Gateway, Cloudflare) trades flexibility for someone else’s uptime and DDoS protection. App-level gateways like the Express example fit teams that want routing logic in the same language and deploy pipeline as their services. On Kubernetes, the same pattern shows up as Ingress controllers and the Gateway API, covered in depth in the Kubernetes Ingress guide.
What Works
- Keep the gateway stateless so it can scale horizontally and survive instance loss
- Store routing rules in configuration (YAML, a control plane, service discovery), not in code
- Use health checks so traffic only flows to instances that respond
- Offload TLS at the gateway to keep certificate handling out of the services
- Limit gateway logic to cross-cutting concerns; the moment a route needs domain rules, move it
- Attach a correlation ID at the edge and propagate it to every upstream call
Common Mistakes
- Putting business logic in the gateway, where it’s hard to test and version
- Routing every microservice through one under-scaled gateway; that’s the SPOF problem. Run two or more instances with health checks
- Ignoring timeout and retry settings, which turns a slow upstream into cascading failures
- Overloading the gateway with heavy transformations; complex reshaping belongs in a BFF
- Forgetting to validate TLS certificates on upstream connections, which silently disables authentication between hops
- Ordering rules wrong so a broad pattern like
/apiswallows traffic meant for a specific route
Advanced Patterns
The static examples above cover the common case. Once traffic grows, four extensions come up repeatedly: dynamic upstream resolution, per-service circuit breaking, transformation, and weighted splits.
Dynamic routing with service discovery
Static route tables go stale as soon as services autoscale or redeploy. Wiring the gateway to a service registry keeps upstreams current: when instances register or deregister, the router re-resolves them per request.
import express from 'express';
import { randomUUID } from 'node:crypto';
import { createProxyMiddleware } from 'http-proxy-middleware';
import { ServiceRegistry } from './service-registry';
class DynamicGateway {
private registry: ServiceRegistry;
private app: express.Application;
constructor(registry: ServiceRegistry) {
this.registry = registry;
this.app = express();
}
async setupRoutes() {
// Correlation ID middleware must run before any proxy route.
this.app.use((req, _res, next) => {
req.headers['x-request-id'] ??= randomUUID();
next();
});
const services = await this.registry.getAllServices();
services.forEach(service => {
this.app.use(service.path, createProxyMiddleware({
target: `http://${service.instances[0].host}:${service.instances[0].port}`,
changeOrigin: true,
router: () => {
// Round-robin replacement: pick a healthy instance per request.
const healthy = service.instances.filter(i => i.healthy);
const pick = healthy[Math.floor(Math.random() * healthy.length)];
return `http://${pick.host}:${pick.port}`;
},
onProxyReq: (proxyReq, req) => {
proxyReq.setHeader('X-Request-ID', req.headers['x-request-id']);
},
onError: (_err, _req, res) => {
(res as express.Response).status(502).json({ error: 'Bad Gateway' });
},
}));
});
}
listen(port: number) {
this.app.listen(port, () => console.log(`Gateway listening on ${port}`));
}
}
Circuit breaker per upstream
Wrapping each upstream call in a circuit breaker stops a failing service from exhausting the gateway’s connection pool. The state lives per service, so one dead upstream doesn’t trip the others:
import CircuitBreaker from 'opossum';
const breakerOptions = {
timeout: 3000,
errorThresholdPercentage: 50,
resetTimeout: 30000,
};
class CircuitBreakerGateway {
private breakers = new Map<string, CircuitBreaker>();
getBreaker(serviceName: string) {
if (!this.breakers.has(serviceName)) {
const breaker = new CircuitBreaker(
async (url: string, init: RequestInit) => {
const response = await fetch(url, init);
if (!response.ok) throw new Error(`HTTP ${response.status}`);
return response.json();
},
breakerOptions,
);
breaker.on('open', () => console.log(`Circuit opened for ${serviceName}`));
breaker.on('halfOpen', () => console.log(`Circuit half-open for ${serviceName}`));
breaker.on('close', () => console.log(`Circuit closed for ${serviceName}`));
this.breakers.set(serviceName, breaker);
}
return this.breakers.get(serviceName)!;
}
async proxyRequest(serviceName: string, path: string, method: string, body?: string) {
const breaker = this.getBreaker(serviceName);
return breaker.fire(`http://${serviceName}:3001${path}`, {
method,
headers: { 'content-type': 'application/json' },
body,
});
}
}
Request and response transformation
Header injection and response wrapping are safe gateway-level transforms. Protocol translation is heavier. The gRPC block below is a sketch, not runnable code, because it needs a generated client:
import express from 'express';
class TransformGateway {
private app = express();
setupTransforms() {
// Inject headers before routing.
this.app.use('/api/v1', (req, _res, next) => {
req.headers['x-api-version'] = 'v1';
req.headers['x-request-time'] = new Date().toISOString();
next();
});
// Wrap v2 responses in a meta envelope.
this.app.use('/api/v2', (_req, res, next) => {
const originalJson = res.json.bind(res);
res.json = (data: unknown) => originalJson({
meta: { version: 'v2', timestamp: new Date().toISOString() },
data,
});
next();
});
// Sketch: REST to gRPC translation. Wire a generated client here;
// loadGrpcClient and the mappers come from your proto toolchain.
this.app.post('/grpc-proxy', async (req, res) => {
try {
// const client = loadGrpcClient('users-service');
// const grpcRes = await client.getUser(mapRestToGrpc(req.body));
// res.json(mapGrpcToRest(grpcRes));
res.status(501).json({ error: 'gRPC translation not configured' });
} catch {
res.status(500).json({ error: 'gRPC translation failed' });
}
});
}
}
Weighted routing for canary releases
A percentage split sends a small share of traffic to a new version while the rest stays on the stable one. Roll the weight forward as metrics stay green:
# Weighted routing: 10% of traffic to the canary
routes:
- path: /api/users
upstreams:
- service: users-service-v1
weight: 90
- service: users-service-v2 # canary
weight: 10
Pair it with per-client rate limiting (counters in Redis for distributed gateways) and automatic rollback when error rates climb; the canary is only safe if something watches it.
See Also
- Companion code: the gateway-routing-pattern companion repo - runnable Express gateway, NGINX config, and a Docker Compose demo with echo upstreams
- API Gateway Design guide - resilience, security, and scaling patterns for production gateways
- Strangler Fig Pattern - gateway routing is how you peel routes off a monolith
- Anti-Corruption Layer Pattern - when the gateway also needs to translate between domain models
- Microsoft: Gateway Routing pattern - the catalog entry this pattern is based on
- NGINX proxy module docs -
proxy_passand upstream configuration - http-proxy-middleware - the Express middleware used in the examples
Frequently Asked Questions
What is the difference between Gateway Routing and an API Gateway?
Routing is the piece that matches requests to upstreams. An API Gateway is the product built around it, so you usually get authentication, rate limiting, transformation, and a developer portal bundled in.
Should the gateway handle retries?
It can retry safe, idempotent requests (GET, PUT with idempotency keys), but be careful with POST and other state-changing operations; a retried charge or order can duplicate work. Always bound retries and add jitter.
Can I use this pattern with serverless functions?
They work fine. Each function registers as an upstream target and gets routed by path or method, the same as a container service. The catch is cold starts: a sleeping function can add seconds, so size your timeout budget for it.
How do I handle authentication at the gateway?
Validate JWTs or API keys at the edge, then hand identity to services as signed headers like X-User-Id and X-Scopes. Services check the signature, which is cheaper and safer than re-parsing tokens on every hop.
Should the gateway cache responses?
Caching GET responses for hot data pays off fast, but respect Cache-Control from upstreams and plan invalidation; stale data served quickly is still stale.
How do I monitor gateway performance?
Start with p50/p99 latency, error rate, and throughput per route, plus upstream health. Distributed tracing with the correlation ID is what ties a gateway span to the service spans behind it.
Related Resources
API Gateway Design: Resilience, Routing, and Security
A practical guide to designing API gateways: routing, authentication, rate limiting, circuit breakers, caching, and deployment patterns for resilient APIs.
PatternBackend for Frontend (BFF) Pattern
Give each client type its own backend that aggregates downstream services and shapes data to that platform's needs.
PatternAnti-Corruption Layer: Isolate Legacy with Adapters
How to isolate legacy systems with translation adapters. Covers ACL facade, domain translation, bidirectional mapping, and gradual legacy replacement.
PatternStrangler Fig: Gradually Replace Legacy by Intercepting
How to gradually replace a legacy system by intercepting routes and routing traffic to new services. Covers strangler fig, incremental migration, and cutover.
GuideComplete Guide to Kubernetes Ingress
Configure and troubleshoot Kubernetes Ingress. Covers controllers, TLS with cert-manager, path routing, annotations, and the move to Gateway API.
PatternHealth Check Pattern: Expose Liveness and Readiness Probes
Implement liveness and readiness probes for container orchestrators. Covers Kubernetes probe config, dependency checks, and graceful degradation.