Secure API Key Authentication for Services and Clients

Overview

API keys are the simplest and most widely deployed form of machine-to-machine authentication. Where OAuth2 handles user delegation and JWTs carry claims, API keys are opaque strings exchanged between trusted services. Done right, keys give you fast auth, easy revocation, and per-key access control through scoped permissions.

The challenge with API keys isn’t generation — any random string will do — but lifecycle management. Keys need real entropy at generation, TLS in transit, fast validation, rotated periodically, and revoked immediately upon compromise. A leaked API key embedded in a mobile app or committed to a public repository grants attackers the same privileges as the legitimate service. Below is a practical approach to key generation with HMAC signatures, request validation, scope enforcement, and rotation strategies.

When to use it

Use this recipe when:

• Authenticating backend services, microservices, or serverless functions to each other
• Providing third-party developers access to a public API with usage limits
• Securing webhook endpoints that receive push notifications from external providers
• Replacing basic authentication (username/password) in service-to-service calls
• Implementing tiered API access with different keys for read-only vs write operations

Skip this recipe when the caller is a human user — sessions, OAuth2, or magic links fit better; an API key glued to a person is just a shared password nobody rotates.

Solution

Generating Secure API Keys (Python)

import secrets
import hmac
import hashlib
from datetime import datetime, timezone

class APIKeyManager:
    def __init__(self, master_secret: str):
        self.master_secret = master_secret.encode()

    def generate_key(self, owner_id: str, scopes: list[str]) -> dict:
        random_part = secrets.token_urlsafe(32)
        key_id = f"pk_{random_part[:16]}"

        # HMAC-SHA256 signature binds key to owner and scopes
        payload = f"{key_id}:{owner_id}:{':'.join(sorted(scopes))}"
        signature = hmac.new(
            self.master_secret,
            payload.encode(),
            hashlib.sha256
        ).hexdigest()[:16]

        api_key = f"{key_id}.{signature}"
        return {
            "key_id": key_id,
            "api_key": api_key,
            "owner_id": owner_id,
            "scopes": scopes,
            "created_at": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"),
            "last_used": None,
        }

    def validate_key(self, api_key: str, owner_id: str, scopes: list[str]) -> bool:
        parts = api_key.split('.')
        if len(parts) != 2:
            return False

        key_id, provided_sig = parts
        payload = f"{key_id}:{owner_id}:{':'.join(sorted(scopes))}"
        expected_sig = hmac.new(
            self.master_secret,
            payload.encode(),
            hashlib.sha256
        ).hexdigest()[:16]

        return hmac.compare_digest(provided_sig, expected_sig)

Validating API Keys in Middleware (Node.js / Express)

const crypto = require('crypto');

function apiKeyAuth(masterSecret) {
  return async (req, res, next) => {
    const authHeader = req.headers['x-api-key'];
    if (!authHeader) {
      return res.status(401).json({ error: 'API key required' });
    }

    // Lookup key metadata from cache or database
    const keyData = await redis.get(`apikey:${authHeader}`);
    if (!keyData) {
      return res.status(401).json({ error: 'Invalid API key' });
    }

    const key = JSON.parse(keyData);

    // Check expiration
    if (key.expires_at && new Date(key.expires_at) < new Date()) {
      return res.status(401).json({ error: 'API key expired' });
    }

    // Verify scopes for this endpoint
    const requiredScope = req.routeScope;
    if (requiredScope && !key.scopes.includes(requiredScope)) {
      return res.status(403).json({ error: 'Insufficient scope' });
    }

    // Update last used timestamp (fire and forget)
    redis.hset(`apikey:${authHeader}:meta`, 'last_used', Date.now());

    req.apiKey = key;
    next();
  };
}

// Usage
app.get('/api/v1/users', apiKeyAuth(process.env.API_SECRET), requireScope('users:read'), (req, res) => {
  res.json(users);
});

API Gateway Key Validation (AWS API Gateway)

openapi: 3.0.1
info:
  title: Secure API
paths:
  /users:
    get:
      x-amazon-apigateway-request-validator: params-only
      security:
        - api_key: []
      x-amazon-apigateway-integration:
        type: aws_proxy
        uri: arn:aws:apigateway:...:lambda:path/...

components:
  securitySchemes:
    api_key:
      type: apiKey
      name: x-api-key
      in: header
      x-amazon-apigateway-api-key-source: HEADER

Explanation

• Key structure: a well-designed API key contains a public identifier (key ID) and a secret signature. The key ID is logged and displayed in dashboards; the signature is validated server-side. Never store the full raw key in logs.
• HMAC validation: rather than a DB row per key and a lookup per request, you can validate keys using HMAC. The signature proves you issued the key — no database round-trip needed. However, storing metadata (owner, scopes, expiration) still requires a lookup.
• Scope-based access control: assign scopes like users:read, orders:write, admin:full to each key. Middleware then confirms the endpoint’s required scope sits in the key’s scope list before allowing access.
• Rate limiting per key: track request counts per API key in Redis with TTL windows. Apply tiered limits — a free-tier key gets 100 requests/hour while an enterprise key gets 100,000.

Storing Keys as Hashes

A plaintext key table turns any database leak into a credential giveaway. Hash at rest and compare digests on lookup — the raw key exists only in the client’s request:

import hashlib

def key_hash(api_key: str) -> str:
    return hashlib.sha256(api_key.encode()).hexdigest()

# On issue: store key_hash(api_key) in Redis/DB, never api_key itself
# On request: look up redis.get(f"apikey:{key_hash(incoming)}")

Keep a separate key_id prefix column so you can log pk_abc123 without exposing the digest either.

Rotating Keys Without Downtime

Rotation works when both keys validate during a grace window:

1. Issue key_new to the owner, record key_old: {valid_until: now + 7d}.
2. Middleware accepts either key until valid_until; log which one each request used.
3. After the window, delete key_old — stragglers get a clean 401 with a renewal link in documentation_url.

Automate the whole loop: a cron job that flags keys older than 90 days and emails the owner is worth more than any policy document.

Variants

Approach	Storage	Validation speed	Revocation	Best for
Stateless HMAC	None (signature)	Fast (no DB)	Impossible	Internal services with short-lived keys
Database lookup	SQL/NoSQL	Medium	Instant	Public APIs with user tiers
Redis cache	Redis	Fast	TTL-based	High-traffic APIs
API Gateway managed	Cloud provider	Fast	Via dashboard	AWS/GCP/Azure hosted APIs

What works

• Never commit keys to source control: use . gitignore for . env files and run secret-scanning tools (GitGuardian, TruffleHog) in CI pipelines. Rotate any key found in commit history immediately.
• HTTPS only: keys sent over plain HTTP are trivially intercepted by network sniffers. Reject plain HTTP requests at the load balancer or gateway level.
• Rotate keys proactively: set a maximum key age (90 days for production, 30 days for high-sensitivity) and notify owners before expiration. Provide a grace period where both old and new keys work.
• Log key IDs, never full keys: record only the key prefix (pk_abc123...) for debugging — the full signature never belongs in logs, error messages, or URLs.
• Usage alerts: ping key owners as they near 80% of their rate limit. This reduces surprise 429 errors and encourages upgrades for legitimate growth.

Common mistakes

• Predictable key formats: sequential IDs or UUIDv1 keys leak generation time — generate with secrets.token_urlsafe or /dev/urandom.
• Keys in client-side code: browsers and mobile apps can’t keep secrets — anything shipped is public.
• Not validating scopes: a read-only analytics key shouldn’t be able to delete records. Check scopes at the endpoint itself, not only at auth time.
• Keys hardcoded in config: production keys in config.json or env vars on shared servers leak to every process.

Security Considerations

• Keys in URLs: an ?api_key= query param lands in server logs, browser history, and Referer headers. Send keys in X-API-Key or Authorization headers only.
• Keys in client-side code: anything shipped to a browser or mobile app is public. Server-to-server only; use a backend proxy if a client needs the data.
• Unencrypted key storage: a database breach shouldn’t hand out usable keys. Store sha256(key) and compare hashes on lookup — the plaintext never rests.
• No rotation policy: keys that never expire stay compromised forever. Rotate on a schedule (90 days is a sane default) and warn owners before expiry.
• Timing leaks on validation: comparing keys with == leaks prefix length through response timing. Use a constant-time comparison — hmac.compare_digest in Python, crypto.timingSafeEqual in Node.
• Auth endpoints without rate limiting: unlimited tries means brute-forcing key prefixes. Rate-limit per IP and per key, and log failures for review.
• Silent failures: log every rejected key attempt with IP and timestamp — a burst of 401s is usually a brute-force or a leaked key.