Ambassador: Offload Cross-Cutting Concerns to a Proxy

Overview

Every outbound call drags the same baggage behind it: retry loops, timeouts, TLS config, metrics. In a microservices deployment that logic gets copied into every client — and drifts out of sync within weeks.

The ambassador pattern puts that baggage in a proxy between your client and the outside world. The client calls the ambassador as if it were the real service; the ambassador adds connection pooling, retries, circuit breaking, and observability on the way through.

This page covers the infrastructure variant: the ambassador as a separate process or container — Envoy beside your app, Nginx in front of a database, or one shared proxy for a whole namespace. For the in-process variant (a wrapper class inside your client), see Ambassador Pattern: Services.

When to Use

• Several services in a namespace call the same external API and should share one connection pool instead of maintaining five.
• A legacy or third-party client you can’t rebuild needs retries, TLS, or metrics on its outbound calls.
• Outbound traffic needs central logging or rate limiting for audit, cost control, or a flaky provider.
• Database connections need pooling and the driver doesn’t do it — the PgBouncer scenario.

When NOT to Use

• The client library already handles resilience (resilience4j, Polly, grpc built-in retries). A proxy on top adds a hop for nothing.
• Latency is measured in microseconds. An extra network hop — even localhost — shows up in the tail.
• One client, one service, simple calls: a plain HTTP client with a timeout is enough.
• The “ambassador” is really business logic. Pricing rules, validation, and orchestration belong in the client or the service, not the proxy.

Solution

Every client in the diagram points at the ambassador’s address, not the real service. The ambassador owns the upstream connection pool, applies the resilience policy chain, and reports metrics — the clients stay dumb.

Ambassador as a reverse proxy (Envoy)

# envoy-ambassador.yaml — Envoy ambassador for outbound traffic
static_resources:
  listeners:
    - address:
        socket_address:
          address: 0.0.0.0
          port_value: 18080
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: outbound
                route_config:
                  virtual_hosts:
                    - name: external-api
                      domains: ["*"]
                      routes:
                        - match: { prefix: "/" }
                          route:
                            cluster: external-api
                            retry_policy:
                              retry_on: "5xx,connect-failure,refused-stream"
                              num_retries: 3
                              per_try_timeout: 5s
                            timeout: 30s
                http_filters:
                  - name: envoy.filters.http.router
  clusters:
    - name: external-api
      connect_timeout: 5s
      type: STRICT_DNS
      lb_policy: ROUND_ROBIN
      circuit_breakers:
        thresholds:
          - max_connections: 100
            max_pending_requests: 50
            max_requests: 100
            max_retries: 3
      load_assignment:
        cluster_name: external-api
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: api.external.com
                      port_value: 443
      transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
          sni: api.external.com

The clients hit localhost:18080. Envoy does TLS upstream, retries on 5xx and connect failures, and trips its circuit breaker at the configured thresholds — no client code involved.

Python ambassador proxy

# ambassador_proxy.py — Python ambassador for outbound API calls
import requests
import time
import logging
from collections import defaultdict

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

class AmbassadorProxy:
    """Proxy that adds retry, circuit breaking, and monitoring
    to outbound calls without modifying the client."""

    def __init__(self, target_url):
        self.target_url = target_url
        self.failure_count = 0
        self.failure_threshold = 5
        self.recovery_timeout = 30
        self.last_failure_time = None
        self.circuit_open = False
        self.request_stats = defaultdict(lambda: {"count": 0, "errors": 0, "latency_ms": []})

    def _check_circuit(self):
        if self.circuit_open:
            if time.time() - self.last_failure_time > self.recovery_timeout:
                logger.info("Circuit breaker recovery attempt")
                self.circuit_open = False
                self.failure_count = 0
            else:
                raise CircuitBreakerOpenError("Circuit breaker is open")

    def _record_success(self, endpoint, latency_ms):
        self.failure_count = 0
        self.circuit_open = False
        stats = self.request_stats[endpoint]
        stats["count"] += 1
        stats["latency_ms"].append(latency_ms)

    def _record_failure(self, endpoint, latency_ms):
        self.failure_count += 1
        self.last_failure_time = time.time()
        stats = self.request_stats[endpoint]
        stats["count"] += 1
        stats["errors"] += 1
        stats["latency_ms"].append(latency_ms)

        if self.failure_count >= self.failure_threshold:
            self.circuit_open = True
            logger.warning(f"Circuit breaker opened after {self.failure_count} failures")

    def request(self, method, path, max_retries=3, **kwargs):
        endpoint = f"{method} {path}"
        self._check_circuit()

        last_error = None
        for attempt in range(max_retries):
            start = time.time()
            try:
                url = f"{self.target_url}{path}"
                resp = requests.request(method, url, timeout=10, **kwargs)
                latency_ms = (time.time() - start) * 1000

                if resp.status_code < 500:
                    self._record_success(endpoint, latency_ms)
                    return resp
                else:
                    last_error = f"HTTP {resp.status_code}"
                    logger.warning(f"Attempt {attempt+1} failed: {last_error}")

            except requests.RequestException as e:
                latency_ms = (time.time() - start) * 1000
                last_error = str(e)
                logger.warning(f"Attempt {attempt+1} failed: {e}")

            if attempt < max_retries - 1:
                backoff = 2 ** attempt
                time.sleep(backoff)

        self._record_failure(endpoint, 0)
        raise AmbassadorError(f"All retries exhausted: {last_error}")

    def get_stats(self):
        return {
            endpoint: {
                "count": s["count"],
                "errors": s["errors"],
                "avg_latency_ms": sum(s["latency_ms"]) / len(s["latency_ms"]) if s["latency_ms"] else 0,
                "circuit_open": self.circuit_open
            }
            for endpoint, s in self.request_stats.items()
        }


class CircuitBreakerOpenError(Exception):
    pass

class AmbassadorError(Exception):
    pass

Client using the ambassador

# client_with_ambassador.py — client talks to ambassador, not external service
from ambassador_proxy import AmbassadorProxy

# Client points to ambassador, not the real API
ambassador = AmbassadorProxy("https://api.external.com")

# Client code doesn't know about retry, circuit breaking, or monitoring
def get_user_profile(user_id):
    resp = ambassador.request("GET", f"/users/{user_id}")
    return resp.json()

def create_order(order_data):
    resp = ambassador.request(
        "POST", "/orders",
        json=order_data,
        headers={"Content-Type": "application/json"}
    )
    return resp.json()

# Check ambassador stats
stats = ambassador.get_stats()
for endpoint, s in stats.items():
    print(f"{endpoint}: {s['count']} requests, {s['errors']} errors, avg {s['avg_latency_ms']:.0f}ms")

Nginx ambassador for database connection pooling

Raw TCP traffic — like PostgreSQL on port 5432 — can’t go through Nginx’s HTTP server block. It needs the stream module, which proxies L4 connections:

# nginx-db-ambassador.conf — Nginx stream ambassador for Postgres
stream {
    upstream database_pool {
        server db-primary:5432 max_conns=100;
        server db-replica:5432 max_conns=50 backup;
    }

    server {
        listen 5432;
        proxy_pass database_pool;
        proxy_connect_timeout 5s;
        proxy_timeout 30s;
    }
}

Clients connect to the ambassador on 5432; Nginx balances them across the pool and fails over to the replica when the primary stops answering.

Java ambassador with retry and metrics

// AmbassadorClient.java — Java ambassador for outbound calls
import java.net.http.*;
import java.net.URI;
import java.time.Duration;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;

public class AmbassadorClient {

    private final HttpClient client;
    private final String targetUrl;
    private final ConcurrentHashMap<String, RequestStats> stats = new ConcurrentHashMap<>();
    private final AtomicInteger failureCount = new AtomicInteger(0);
    private final int failureThreshold = 5;
    private volatile boolean circuitOpen = false;
    private volatile long lastFailureTime = 0;
    private final long recoveryTimeoutMs = 30000;

    public AmbassadorClient(String targetUrl) {
        this.targetUrl = targetUrl;
        this.client = HttpClient.newBuilder()
            .connectTimeout(Duration.ofSeconds(5))
            .build();
    }

    public HttpResponse<String> request(String method, String path,
                                        String body, int maxRetries) throws Exception {
        checkCircuit();

        Exception lastError = null;
        for (int attempt = 0; attempt < maxRetries; attempt++) {
            long start = System.currentTimeMillis();
            try {
                var builder = HttpRequest.newBuilder()
                    .uri(URI.create(targetUrl + path))
                    .timeout(Duration.ofSeconds(10));

                switch (method) {
                    case "GET" -> builder.GET();
                    case "POST" -> builder.POST(HttpRequest.BodyPublishers.ofString(body));
                    case "PUT" -> builder.PUT(HttpRequest.BodyPublishers.ofString(body));
                    case "DELETE" -> builder.DELETE();
                }

                var response = client.send(builder.build(),
                    HttpResponse.BodyHandlers.ofString());

                long latency = System.currentTimeMillis() - start;
                if (response.statusCode() < 500) {
                    recordSuccess(method + " " + path, latency);
                    return response;
                }
                recordFailure(method + " " + path, latency);
                lastError = new RuntimeException("HTTP " + response.statusCode());
            } catch (Exception e) {
                long latency = System.currentTimeMillis() - start;
                lastError = e;
                recordFailure(method + " " + path, latency);
            }

            if (attempt < maxRetries - 1) {
                Thread.sleep((long) Math.pow(2, attempt) * 1000);
            }
        }

        lastFailureTime = System.currentTimeMillis();
        if (failureCount.get() >= failureThreshold) {
            circuitOpen = true;
        }

        throw new RuntimeException("All retries exhausted", lastError);
    }

    private void checkCircuit() throws Exception {
        if (circuitOpen) {
            if (System.currentTimeMillis() - lastFailureTime > recoveryTimeoutMs) {
                circuitOpen = false;
                failureCount.set(0);
            } else {
                throw new RuntimeException("Circuit breaker is open");
            }
        }
    }

    private void recordSuccess(String endpoint, long latencyMs) {
        failureCount.set(0);
        circuitOpen = false;
        stats.computeIfAbsent(endpoint, k -> new RequestStats())
             .record(latencyMs, true);
    }

    private void recordFailure(String endpoint, long latencyMs) {
        failureCount.incrementAndGet();
        stats.computeIfAbsent(endpoint, k -> new RequestStats())
             .record(latencyMs, false);
    }

    static class RequestStats {
        final AtomicLong count = new AtomicLong(0);
        final AtomicLong errors = new AtomicLong(0);
        final AtomicLong totalLatency = new AtomicLong(0);

        void record(long latencyMs, boolean success) {
            count.incrementAndGet();
            totalLatency.addAndGet(latencyMs);
            if (!success) errors.incrementAndGet();
        }
    }
}

Note the ordering: a 5xx response is recorded as a failure before the retry loop continues. Record it as success only after checking the status — otherwise a flapping service keeps your failure counter at zero and the circuit never opens.

How It Works

Every request follows the same pipeline: the client opens a connection to the ambassador, the ambassador applies its policy chain — circuit check first, then retry loop, then per-attempt timeout — and forwards the call upstream. Responses come back the same way, with latency and status recorded before the client sees them.

The ordering matters. Checking the circuit before retrying makes failure cheap: when the upstream is down, the ambassador returns an error in microseconds instead of sleeping through three backoff rounds. And the shared pool is where the pattern pays for itself — ten clients that each held twenty connections to Postgres become one pool of fifty.

The trade-off is an extra network hop and a new thing to operate. Each call gains the proxy’s latency — a millisecond or two inside a cluster, more if the ambassador is remote — and you inherit its availability: if the ambassador dies, every client behind it goes dark. Run it as a Deployment with replicas, or as a per-node DaemonSet if you want the hop to stay local.

Ambassadors you already run: Envoy and Linkerd’s proxy are ambassadors for service-to-service traffic. PgBouncer and ProxySQL are ambassadors for database connections. HAProxy in front of a legacy API is one too. The pattern is old — the name is what’s new.

Variants

Ambassador as a sidecar (in-process)

# in_process_ambassador.py — ambassador logic embedded in the client
# Requires: requests, time, AmbassadorError from ambassador_proxy.py
import requests
import time
from ambassador_proxy import AmbassadorError

class InProcessAmbassador:
    """Ambassador logic embedded directly in the client process.
    No separate container needed — just a wrapper class."""

    def __init__(self, target_url, max_retries=3, timeout=10):
        self.target_url = target_url
        self.max_retries = max_retries
        self.timeout = timeout
        self._pool = requests.Session()
        # Configure connection pooling
        adapter = requests.adapters.HTTPAdapter(
            pool_connections=10,
            pool_maxsize=100,
            max_retries=0  # We handle retries ourselves
        )
        self._pool.mount("https://", adapter)
        self._pool.mount("http://", adapter)

    def get(self, path, **kwargs):
        return self._request("GET", path, **kwargs)

    def post(self, path, **kwargs):
        return self._request("POST", path, **kwargs)

    def _request(self, method, path, **kwargs):
        for attempt in range(self.max_retries):
            try:
                resp = self._pool.request(
                    method,
                    f"{self.target_url}{path}",
                    timeout=self.timeout,
                    **kwargs
                )
                if resp.status_code < 500:
                    return resp
            except requests.RequestException:
                pass
            if attempt < self.max_retries - 1:
                time.sleep(2 ** attempt)
        raise AmbassadorError(f"Failed after {self.max_retries} retries")

This sits halfway to the in-process ambassador variant: same policy chain, no extra hop, but the pooling and retry config lives in your codebase instead of your infra.

Ambassador for multiple external services

# multi_ambassador.py — one ambassador routing to several services
# Requires: AmbassadorProxy from ambassador_proxy.py
from ambassador_proxy import AmbassadorProxy

class MultiServiceAmbassador:
    """Single ambassador that routes to multiple external services."""

    def __init__(self):
        self.services = {}

    def register(self, name, url, config):
        self.services[name] = AmbassadorProxy(url)
        self.services[name].failure_threshold = config.get("failure_threshold", 5)

    def call(self, service, method, path, **kwargs):
        if service not in self.services:
            raise ValueError(f"Unknown service: {service}")
        return self.services[service].request(method, path, **kwargs)

    def all_stats(self):
        return {
            name: proxy.get_stats()
            for name, proxy in self.services.items()
        }

# Usage
ambassador = MultiServiceAmbassador()
ambassador.register("payment", "https://api.payment.com", {"failure_threshold": 3})
ambassador.register("shipping", "https://api.shipping.com", {"failure_threshold": 5})
ambassador.register("inventory", "https://api.inventory.com", {"failure_threshold": 10})

resp = ambassador.call("payment", "POST", "/charge", json={"amount": 99.99})

Best Practices

• Use connection pooling — the ambassador holds persistent connections to external services.
• Set timeouts at two levels — per-attempt and end-to-end — or retries turn a 5s failure into a 30s hang.
• Implement circuit breaking — stop calling a failing service so it can recover.
• Collect metrics — request count, error rate, and latency per endpoint.
• Use exponential backoff for retries — 1s, 2s, 4s, not fixed intervals.
• Keep the ambassador stateless — it’s a proxy, not a cache or a session store.
• Monitor the ambassador itself — when it fails, every client behind it fails.
• Prefer standard proxies — Envoy, Nginx, and HAProxy beat hand-rolled proxy code.

Common Mistakes

• No circuit breaking: the ambassador keeps hammering a dead service and makes the outage worse.
• Retrying non-idempotent operations: retrying POST requests can create duplicates. Only retry GET, PUT, DELETE.
• Retries without backoff: three instant retries are a thundering herd, not resilience. Use exponential backoff.
• One ambassador for everything: a single instance for all clients is a bottleneck and a single point of failure. Scale it horizontally.
• Blind ambassador: no health checks or metrics on the proxy itself, so you find out it’s down from your clients’ error logs.

See Also

• Ambassador pattern — Azure Architecture Center — the canonical catalog entry for this pattern.
• Envoy proxy documentation — retry policies, circuit breakers, and TLS termination upstream.
• PgBouncer — a production ambassador for PostgreSQL connection pooling.
• Ambassador Pattern: Resilient Remote Calls via Local Proxy — the in-process variant of this pattern.
• Compute Resource Consolidation Pattern — consolidating workloads onto shared infrastructure.