Async Patterns with Promises, Futures, and Coroutines

Overview

Synchronous code blocks the execution thread until an operation completes. When that operation is I/O — querying a database, fetching from an API, reading a file — the thread sits idle, wasting CPU cycles that could process other requests. Async programming fixes this by suspending the current task when it hits I/O, letting the runtime run other work, and resuming the task when the I/O completes. A single thread can then hold thousands of concurrent connections.

The hard part isn’t writing async and await — it’s understanding the event loop underneath, avoiding callback hell, propagating errors across suspension points, and bounding concurrency so shared resources don’t get exhausted. Each runtime does it differently: JavaScript runs an event loop with promises, Python has asyncio coroutines, Java has CompletableFuture on thread pools. Below are the patterns, anti-patterns, and working implementations for all three.

When to use it

Use this recipe when:

• Building APIs that handle hundreds of concurrent requests per process
• Fetching data from several independent services that can be called in parallel
• Processing I/O-bound workloads like web scraping, file uploads, or message queues
• Implementing real-time capabilities like WebSockets, chat, or live dashboards
• Replacing a thread-per-request model with an event-driven architecture to cut server count

Solution

Async/Await with Concurrent Requests (JavaScript / Node.js)

async function fetchUserDashboard(userId) {
  const [profile, orders, recommendations] = await Promise.all([
    getProfile(userId),
    getOrders(userId),
    getRecommendations(userId),
  ]);
  return { profile, orders, recommendations };
}

async function fetchDashboardResilient(userId) {
  const [profile, orders, recommendations] = await Promise.allSettled([
    getProfile(userId),
    getOrders(userId),
    getRecommendations(userId),
  ]);

  return {
    profile: profile.status === 'fulfilled' ? profile.value : null,
    orders: orders.status === 'fulfilled' ? orders.value : [],
    recommendations: recommendations.status === 'fulfilled' ? recommendations.value : [],
  };
}

Python asyncio with Task Groups

import asyncio
import aiohttp

async def fetch_url(session: aiohttp.ClientSession, url: str) -> dict:
    async with session.get(url) as response:
        return await response.json()

async def fetch_all_urls(urls: list[str]) -> list[dict]:
    async with aiohttp.ClientSession() as session:
        async with asyncio.TaskGroup() as tg:
            tasks = [tg.create_task(fetch_url(session, url)) for url in urls]
        return [task.result() for task in tasks]

async def fetch_with_limit(urls: list[str], max_concurrent: int = 10):
    semaphore = asyncio.Semaphore(max_concurrent)

    async def bounded_fetch(session, url):
        async with semaphore:
            return await fetch_url(session, url)

    async with aiohttp.ClientSession() as session:
        return await asyncio.gather(
            *[bounded_fetch(session, url) for url in urls]
        )

urls = ["https://api.example.com/users/1", "https://api.example.com/users/2"]
results = asyncio.run(fetch_all_urls(urls))

Java CompletableFuture Pipeline

import java.util.concurrent.CompletableFuture;

public class AsyncOrderService {
    public CompletableFuture<Order> processOrder(String orderId) {
        return validateOrder(orderId)
            .thenCompose(this::checkInventory)
            .thenCompose(this::processPayment)
            .thenCompose(this::createShipment)
            .exceptionally(ex -> {
                log.error("Order processing failed", ex);
                return Order.failed(orderId, ex.getMessage());
            });
    }

    private CompletableFuture<ValidatedOrder> validateOrder(String orderId) {
        return CompletableFuture.supplyAsync(() -> new ValidatedOrder(orderId));
    }

    public CompletableFuture<Dashboard> loadDashboard(String userId) {
        CompletableFuture<Profile> profileFuture = fetchProfile(userId);
        CompletableFuture<List<Order>> ordersFuture = fetchOrders(userId);
        return profileFuture.thenCombine(ordersFuture, Dashboard::new);
    }
}

Explanation

• Event loop: the mechanism behind JavaScript async and Python asyncio. It keeps a queue of ready tasks and runs them one at a time. When a task hits await, it yields; the loop picks up the next ready task; when the awaited I/O finishes, the suspended task goes back in the queue. One thread, no context-switch cost per task.
• Structured concurrency: in Python 3.11+, asyncio.TaskGroup cancels every sibling task if one child fails, so you never leak orphaned background work after a parent error. JavaScript has no built-in equivalent — Promise.all fails fast but doesn’t cancel the rest, so AbortController is the manual version.
• Promise composition: Promise.all() waits for every promise and rejects on the first failure; Promise.allSettled() waits for all and reports each outcome; Promise.race() returns the first to settle. The asyncio equivalents are gather, TaskGroup, and wait with FIRST_COMPLETED.
• Backpressure with semaphores: unbounded concurrency exhausts memory, file descriptors, and upstream quotas. A semaphore caps simultaneous operations — with a limit of 10, the 11th request waits for a slot instead of competing for sockets.

Variants

Pattern	Language	Concurrency model	Error handling	Best for
async/await	JS/Python	Event loop	try/catch	I/O-bound APIs
CompletableFuture	Java	Thread pool	exceptionally()	CPU + I/O mixed
Goroutines	Go	M:N threads	Channels	High-throughput services
RxJS/RxPY	JS/Python	Observables	onError	Event streams
Threads	All	OS threads	try/catch	CPU-bound tasks

The real split is event-loop concurrency (one thread, cooperative scheduling) versus thread-pool concurrency (preemptive, OS-scheduled). Goroutines sit in between: the Go runtime maps goroutines onto a small set of OS threads, which is why Go scales differently than Node.js under the same connection count.

What works

• Always await your promises: an unawaited promise is fire-and-forget — if it rejects and nothing is waiting on it, Node.js emits unhandledRejection, which crashes the process on Node 15+. In async functions, await or .catch() every promise you create.
• Promise.all for independent work, sequential awaits for dependencies: if task B needs task A’s output, they must run in order. Independent calls waste wall-clock time when serialized — Promise.all, asyncio.gather, or CompletableFuture.allOf run them concurrently.
• Timeout every external call: an API that never responds hangs the task forever. Wrap external calls in a timeout with retry logic so failures stay bounded and latencies predictable.
• Structured concurrency over fire-and-forget: a spawned task that outlives its parent leaks memory and races on shutdown. Use TaskGroup, Promise.allSettled scopes, or explicit cancellation tokens so every task’s lifetime is owned by a parent.
• Profile the event loop instead of guessing: in Node.js use perf_hooks.monitorEventLoopDelay() or clinic.js; in Python run asyncio with PYTHONASYNCIODEBUG=1. If CPU work blocks the loop, move it to a worker thread or process pool.

Common mistakes

• Blocking the event loop: fs.readFileSync or a CPU-heavy loop inside an async function stalls every other request on that process. Use fs.promises.readFile and offload CPU work to worker threads.
• Callback hell without async/await: deeply nested .then() chains are hard to read and debug. Modern JavaScript should use async/await for anything beyond a one-liner — flat code that reads synchronous but runs async.
• Race conditions on shared mutable state: two tasks incrementing a counter across await boundaries still race — await is a suspension point, not a lock. Use atomic operations, locks, or message passing instead of shared mutation.
• Ignoring backpressure: accepting work faster than you can drain it grows memory until the OOM killer steps in. A fast 503 beats a crashed server.

When Not to Use This Approach

• CPU-bound tasks: async doesn’t help when the CPU is the bottleneck; use worker threads or process pools.
• Simple sequential scripts: one HTTP call, wait, exit — a plain requests.get() is clearer than the asyncio equivalent.
• Hard real-time deadlines: async schedulers are cooperative and non-deterministic; keep them off the critical path of deadline-sensitive systems.
• Memory-constrained embedded targets: every pending task holds a closure and a state machine; thousands of suspended tasks add up.
• Legacy synchronous codebases: retrofitting async means touching every I/O call in the chain; the payoff rarely justifies the refactor.
• Single-request batch jobs: a nightly job that calls one endpoint and writes one row gains nothing from async.

Performance Benchmarks

Treat these as orders of magnitude, not guarantees — measure on your own stack and payload sizes:

• Event-loop capacity: a Node.js process on a small VM (2 cores / 4 GB) can hold on the order of 10k keep-alive connections; the limit is file descriptors and per-connection memory, not CPU.
• asyncio vs threads: for I/O-bound HTTP fan-out, asyncio with aiohttp typically sustains several times the throughput of a threaded WSGI stack on one core, because there’s no per-thread stack and no GIL contention while waiting.
• Goroutine overhead: goroutines start with ~2 KB stacks and grow on demand, so hundreds of thousands fit in a few hundred MB of memory.
• Spawn cost: a tokio task spawn costs tens of nanoseconds; an OS thread costs microseconds plus an MB-scale stack reservation.
• Java virtual threads: JDK 21 virtual threads live on the heap, not on OS stacks — a million of them is realistic where a million platform threads would exhaust address space.
• Context-switch cost: an OS thread switch takes ~1–10 µs; an async task switch is a function call, in the ~100–500 ns range.
• Latency under load: event-loop systems hold tighter tail latencies under I/O-heavy load because there’s no thread-pool queue wait; CPU saturation is where the model falls over.
• GC pressure: each pending task is a heap-allocated state machine. Sustained high task churn shows up as GC load — worth watching on the JVM and V8.

Testing Strategy

• Unit test async functions in isolation: mock the I/O boundary and assert on return values; use pytest.mark.asyncio (or anyio) in Python and your test runner’s async support in JavaScript.
• Integration test against real I/O: run a local HTTP server and database; end-to-end bugs hide exactly where the mocks end.
• Stress test with realistic concurrency: launch 1,000+ concurrent tasks and check for deadlocks, leaks, and wrong results — race conditions only surface under interleaving pressure.
• Test timeout behavior: assert that slow operations actually hit asyncio.wait_for timeouts or Promise.race-based deadline wrappers, and that the underlying I/O gets cancelled rather than abandoned.
• Test cancellation propagation: cancel a parent task and assert all children are cancelled; in asyncio, cancellation arrives as CancelledError at the next await.
• Test error propagation: child failures must reach the parent — TaskGroup surfaces them as ExceptionGroup, Promise.all rejects on the first failure.
• Race detection: run Python tests with asyncio debug mode (PYTHONASYNCIODEBUG=1) to catch unclosed resources and slow callbacks; use ThreadSanitizer where real threads are involved.
• Load test with production-sized payloads: toy payloads hide backpressure and GC problems.
• Test backpressure: push requests faster than the service drains them and verify it sheds load (503s, bounded queues) instead of eating memory.
• Chaos test the async path: kill tasks mid-await, inject network delay, drop connections — cleanup handlers and finally blocks are where async bugs live.

Cost Estimation

• Server sizing: async workloads need fewer, smaller boxes — a Node.js async server holding ~10k connections runs fine on a 2-core / 4 GB instance, which costs roughly $20–40/month on mainstream clouds.
• Connection pools: the usual async drivers (aiohttp, asyncpg, undici) are open source; the cost is operational, not licensing.
• Development cost: async code typically takes 20–30% longer to write and debug than the synchronous equivalent — error paths across suspension points are the expensive part.
• Monitoring overhead: you need event-loop lag, task-queue depth, and rejection-rate metrics that generic host monitoring doesn’t cover.
• Infrastructure savings: moving from thread-per-request to async commonly cuts required capacity by 3–5x on I/O-bound services.
• Memory: an async task costs KBs; an OS thread costs an MB-scale stack. At 10k concurrent units of work that’s the difference between tens of MB and tens of GB.
• Operational cost: fewer moving parts — no thread-pool tuning, no lock-contention debugging — at the price of a different bug class (stuck tasks, unawaited promises).

Monitoring and Observability

• Event-loop lag: the delay between a scheduled callback and its execution; sustained lag above ~50 ms means something is blocking. Measure it with perf_hooks.monitorEventLoopDelay() in Node.js or clinic.js for profiles.
• Task queue depth: pending tasks growing monotonically means producers outpace consumers — a backpressure problem, not a scaling problem.
• Active connections: track concurrent connection count against ulimit -n.
• Unhandled rejections and exceptions: count unhandledRejection events in Node.js and “Task exception was never retrieved” log lines in asyncio.
• GC pause time: high task churn allocates many small objects; watch GC pause and heap growth together.
• Memory: RSS and heap — slow growth under constant load is the signature of a leaked task or closure.
• Latency percentiles: p50/p95/p99 per endpoint; a widening p99 with flat p50 usually means a few code paths block the loop.

Deployment Checklist

• Set file-descriptor limits: ulimit -n 65536 or LimitNOFILE=65536 in the systemd unit
• Size connection pools for expected concurrency — for async, small pools suffice (roughly 2× cores), not 50+
• Set timeouts on all I/O: HTTP clients, database queries, cache reads; 5–30 s is a sane default band
• Enable structured logging with request IDs so async call chains stay traceable
• Configure health checks that probe event-loop responsiveness, not just process aliveness
• Set memory limits and OOM behavior — async tasks are cheap, unbounded queues aren’t
• Graceful shutdown: stop accepting work, drain in-flight tasks for 5–10 s, then exit

Security Considerations

• Task flooding: an attacker who can trigger task creation spawns unbounded work; bound concurrency and rate-limit the entry points.
• Callback injection: if user input selects which callback runs, it selects which code runs. Never dispatch on raw user strings.
• Unhandled rejection DoS: in Node.js 15+ an unhandled promise rejection crashes the process. Attach a .catch() on every promise or a process-level unhandledRejection handler.
• Event-loop blocking: one synchronous call (fs.readFileSync, time.sleep, crypto.pbkdf2Sync) stalls every request on the process. Audit dependencies for sync I/O.
• Shared state across awaits: await boundaries allow interleaving; a read-modify-write spread across an await races.
• Timeouts that don’t cancel: if the underlying I/O ignores cancellation, the task times out but the operation keeps burning resources — use transports that honor cancellation.
• Closure memory leaks: each task captures its scope; long-lived tasks pinning large objects prevent GC.
• Supply-chain risk: async drivers (aiohttp, tokio, Netty) have had CVEs; pin versions and watch advisories.
• Slow-client DoS: slowloris-style clients hold connections open — set read timeouts and per-IP connection caps.
• Unsafe deserialization: await response.json() on an unbounded payload buffers the whole body in memory before validation runs; enforce size limits first.
• Coroutine spoofing: in Python, any object implementing __await__ can be awaited — don’t await objects built from untrusted input.
• File-descriptor exhaustion: every connection costs an FD; without limits, a connection flood kills the process.
• Error-message leakage: async stack traces are deep and can expose internal paths, query strings, or credentials — sanitize what crosses the wire.
• TLS verification disabled: some HTTP clients ship permissive TLS defaults — verify certificate checking is actually on.
• ReDoS on the event loop: a pathological regex runs on the loop thread and stalls everything; move matching off the loop or use re2 (node-re2) for linear-time matching.
• Cancellation vs non-idempotent work: cancelling a task mid-operation (e.g., a card charge) can double-charge if the operation completed before the cancel landed; make handlers idempotent.
• async with discipline: skipping async with on sessions and connections leaks resources invisibly.
• Backpressure bypass: fast producer + slow consumer + unbounded queue = memory exhaustion.
• Task-priority manipulation: an attacker who controls task timing can starve critical tasks.
• Timing attacks in auth: compare secrets with secrets.compare_digest (Python) or crypto.timingSafeEqual (Node.js), never ==, so scheduling doesn’t leak byte prefixes.
• Promise prototype pollution: if Promise.prototype can be modified, everything downstream is compromised; Object.freeze it in hostile-plugin environments.
• Async middleware bypass: an unawaited middleware link can be skipped — always await next().
• Rate-limit check-then-act: checking and updating a counter in separate awaits lets concurrent requests slip through; make the check atomic.
• Forced shutdown skips cleanup: SIGKILL never runs async cleanup handlers; rely on SIGTERM plus a drain window.
• Shared pool starvation: consumers sharing a connection pool without quotas let one spike starve the rest.
• Synchronous logging on the loop: a blocking logger inside an async handler stalls the loop; use buffered or async logging.
• Locks held across cancellation: a coroutine cancelled while holding a lock may never release it; acquire locks with async with plus a timeout.
• Microtask flooding: recursive Promise.resolve().then() chains monopolize the microtask queue and starve macrotasks.

Troubleshooting

• Races that only appear under load: protect shared state with locks, atomics, or message passing; reproduce with targeted stress tests, not unit tests.
• Deadlock between workers: establish a global lock-acquisition order and keep critical sections free of await.
• Thread-pool saturation (CompletableFuture/executor code): watch queue length and rejection policy; grow the pool only if CPU and memory allow.
• Queues or actor mailboxes growing unbounded: apply backpressure, bounded queues, and load shedding upstream.
• Async task never completes: look for a missing await, an unhandled rejection swallowed upstream, or a loop that never yields to the scheduler.

Further Reading

• MDN — asynchronous JavaScript and the Promise reference
• Python asyncio documentation and the TaskGroup notes in What’s New in 3.11
• Java CompletableFuture javadoc and JEP 444: Virtual Threads
• Node.js — Don’t block the event loop
• TechEmpower Framework Benchmarks — public, reproducible throughput numbers if you want to compare stacks instead of trusting orders of magnitude

Production Notes

• Roll out gradually with canary or blue-green — event-loop bugs only surface under real concurrency.
• Alert on event-loop lag, unhandled rejection rate, and p99 latency before the rollout, not after.
• Keep a documented rollback in the runbook; going from async back to synchronous is the painful direction.
• Propagate correlation IDs end-to-end — a single request’s async hops are invisible without them.

Key Takeaways

• Async/await buys concurrency on a single thread; it’s an I/O tool, not a CPU tool.
• Bound concurrency (semaphores, pools) and timeout every external call — both are non-negotiable in production.
• Prefer structured concurrency (TaskGroup, scoped promise combinators) so task lifetimes are owned instead of orphaned.
• Measure the event loop itself — lag, queue depth, rejections — because its failure modes don’t show up in request logs.

Failure modes in production

• Shipping an unbounded Promise.all/gather fan-out that melts under real traffic.
• Timing out the task but not the underlying I/O, so the resource keeps leaking in the background.
• Awaiting user-triggered work without rate limits — a free denial-of-service vector.
• Holding a lock or a transaction across an await, which blocks the lock for the duration of the suspension.
• Fire-and-forget tasks that die silently on shutdown and leave in-flight state half-written.
• Assuming async removes races: shared state across an await boundary still races.
• Deploying without event-loop metrics — you fly blind on the one resource that matters.
• Watching p50 latency while p99 burns.