Build Serverless APIs with API Gateway

Overview

Serverless computing allows you to run code without provisioning or managing servers. AWS Lambda executes your functions in response to events, and API Gateway provides the HTTP front door that routes requests to those functions. Together they form a pay-per-request, auto-scaling API platform that eliminates idle server costs.

This architecture is ideal for APIs with variable or unpredictable traffic. A startup might serve a hundred requests per day at launch and a million per day six months later — serverless handles both without capacity planning. The trade-off is cold start latency (the delay when a function wakes up after being idle) and vendor-specific tooling that can create lock-in.

When to Use

Use this recipe when:

• Building APIs with sporadic or unpredictable traffic patterns. See Rate Limiting for protecting APIs under load.
• Prototyping products where server costs should scale to zero when idle
• Processing webhooks, file uploads, or scheduled events via HTTP. See Input Validation for validating incoming requests.
• Creating microservices where each endpoint has different resource needs
• Reducing operational overhead by eliminating server patching and scaling. See Serverless Functions for function deployment.

Solution

AWS Lambda Handler (Python)

import json

def lambda_handler(event, context):
    # API Gateway passes HTTP data in the event object
    method = event['httpMethod']
    path = event['path']
    query = event.get('queryStringParameters', {})

    if method == 'GET' and path == '/users':
        return {
            'statusCode': 200,
            'headers': {'Content-Type': 'application/json'},
            'body': json.dumps({'users': ['alice', 'bob']})
        }

    if method == 'POST' and path == '/users':
        body = json.loads(event['body'])
        return {
            'statusCode': 201,
            'body': json.dumps({'id': '123', 'name': body['name']})
        }

    return {'statusCode': 404, 'body': json.dumps({'error': 'Not found'})}

AWS Lambda Handler (Node.js)

exports.handler = async (event) => {
  const { httpMethod, path, body } = event;

  if (httpMethod === 'GET' && path === '/users') {
    return {
      statusCode: 200,
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ users: ['alice', 'bob'] })
    };
  }

  if (httpMethod === 'POST' && path === '/users') {
    const data = JSON.parse(body);
    return {
      statusCode: 201,
      body: JSON.stringify({ id: '123', name: data.name })
    };
  }

  return { statusCode: 404, body: JSON.stringify({ error: 'Not found' }) };
};

Terraform Deployment

resource "aws_api_gateway_rest_api" "api" {
  name = "users-api"
}

resource "aws_api_gateway_resource" "users" {
  rest_api_id = aws_api_gateway_rest_api.api.id
  parent_id   = aws_api_gateway_rest_api.api.root_resource_id
  path_part   = "users"
}

resource "aws_api_gateway_method" "get_users" {
  rest_api_id   = aws_api_gateway_rest_api.api.id
  resource_id     = aws_api_gateway_resource.users.id
  http_method   = "GET"
  authorization = "NONE"
}

resource "aws_lambda_function" "handler" {
  function_name = "users-handler"
  handler       = "index.handler"
  runtime       = "nodejs20.x"
  filename      = "function.zip"
  source_code_hash = filebase64sha256("function.zip")

  role = aws_iam_role.lambda_role.arn
}

Explanation

• API Gateway: Receives HTTP requests, handles TLS termination, throttling, caching, and authentication, then invokes the Lambda function with the request data in a structured event object.
• Lambda: Stateless function execution environment. AWS manages scaling automatically — if 1,000 requests arrive simultaneously, AWS spins up 1,000 function instances. You pay only for execution time and memory used.
• Cold starts: When a function has not been invoked recently, AWS initializes a new runtime instance. This adds 100ms to 2s of latency depending on language and memory allocation. Provisioned concurrency keeps functions warm for latency-sensitive endpoints.
• Infrastructure as Code: Tools like Terraform, Serverless Framework, or AWS SAM define your API routes, Lambda functions, IAM roles, and environment variables in version-controlled configuration files.

Variants

Platform	API Gateway	Function Runtime	Best For
AWS	API Gateway + Lambda	Python, Node, Java, Go	Mature platform, broad integrations
Azure	API Management + Functions	.NET, Node, Python	Microsoft platform, Visual Studio integration
GCP	Cloud Endpoints + Cloud Functions	Node, Python, Go	BigQuery integration, competitive pricing

What Works

• Keep functions stateless: do not assume variables in memory persist between invocations. Use external storage (DynamoDB, S3, Redis) for state.
• Minimize deployment package size: large packages increase cold start time. Use Lambda layers for shared dependencies and tree-shake unused code.
• Set appropriate timeouts and memory: memory scales CPU proportionally. If a function is slow, increasing memory may be cheaper than paying for longer execution at lower memory.
• Use environment variables for config: database URLs, API keys, and feature flags should be set via environment variables, not baked into the deployment package.
• Implement structured logging: write JSON logs with request IDs. CloudWatch Logs Insights can query these efficiently for debugging and monitoring.
• Use dead letter queues (DLQ): failed async invocations are retried automatically. A DLQ captures persistent failures so you can inspect and reprocess them.

Common Mistakes

• Treating Lambda like a long-running server: functions have a 15-minute maximum execution time. Move long-running work to batch processing (AWS Batch) or containers (ECS/Fargate).
• Ignoring cold starts: latency-sensitive APIs need provisioned concurrency or a keep-alive ping. A user-facing API with 3-second cold starts delivers a terrible experience.
• Over-provisioning memory: Lambda memory scales linearly with cost. Profile your function and allocate only what it needs.
• Hardcoding credentials: never commit AWS keys or database passwords to your repository. Use IAM roles and Secrets Manager.
• Not handling partial failures: in batch processing (SQS triggers), a single bad record can cause the entire batch to fail. Implement per-record error handling.

Error Handling and Recovery

• Cold start error handling: handle initialization errors gracefully. Wrap handler initialization in try-catch blocks. Provide fallback values for missing environment variables. Log initialization errors with structured logging. Implement retry logic for transient failures. Use circuit breakers for downstream service calls. Document error handling strategy. Test error scenarios in staging. Monitor error rates after deployment. Set up alerts for initialization failures
• Function timeout management: set appropriate timeout values for each function. AWS Lambda supports up to 15 minutes. Azure Functions support up to 10 minutes. Google Cloud Functions support up to 60 minutes. Start with short timeouts and adjust based on monitoring. Document timeout values per function. Test behavior at timeout boundary. Implement graceful shutdown on timeout. Monitor timeout frequency. Alert on timeout spikes
• Retry and dead letter queues: configure retry policies for failed invocations. Use dead letter queues (DLQ) for messages that exceed retry limits. AWS SQS supports maxReceiveCount and DLQ configuration. Azure Service Bus supports dead lettering. Google Pub/Sub supports dead letter topics. Document retry strategy. Test DLQ behavior. Monitor DLQ depth. Set up alerts for DLQ messages. Process DLQ messages regularly. Document DLQ handling procedures
• Idempotency in serverless functions: design functions to be idempotent. Use idempotency keys for duplicate detection. Store processed message IDs in a database. Return cached results for duplicate requests. Document idempotency strategy. Test with duplicate messages. Monitor duplicate detection rate. Handle idempotency failures gracefully. Use distributed locks for concurrent duplicates

Security Considerations

• IAM roles and permissions: follow least privilege principle for function IAM roles. Grant only the permissions needed by the function. Use resource-level permissions where possible. Avoid wildcard permissions. Review IAM roles regularly. Use IAM conditions for additional constraints. Document IAM role configuration. Test with minimal permissions. Monitor IAM usage. Alert on permission changes
• Secrets management: use dedicated secrets management services. AWS Secrets Manager for Lambda. Azure Key Vault for Functions. Google Secret Manager for Cloud Functions. Never hardcode secrets in environment variables. Rotate secrets regularly. Document secrets management strategy. Test secret rotation. Monitor secret access. Alert on unauthorized access attempts
• VPC configuration: configure VPC for functions that need private network access. Use private subnets for database access. Configure NAT Gateway for outbound internet. Use VPC endpoints for AWS services. Document VPC configuration. Test VPC connectivity. Monitor VPC resource usage. Review security group rules regularly. Alert on VPC configuration changes
• API authentication: implement authentication for serverless APIs. Use JWT tokens for stateless authentication. Use API keys for simple authentication. Use OAuth 2.0 for third-party authentication. Configure CORS properly. Document authentication strategy. Test authentication flows. Monitor authentication failures. Alert on authentication anomalies. Use rate limiting to prevent abuse

Deployment and CI/CD

• Serverless deployment strategies: use infrastructure as code for deployments. AWS SAM or Serverless Framework for Lambda. Azure Bicep or ARM templates for Functions. Google Cloud Deployment Manager for Cloud Functions. Version all deployments. Use blue-green deployments for zero downtime. Use canary deployments for gradual rollout. Document deployment strategy. Test deployment in staging. Monitor deployment health. Rollback on failures
• CI/CD pipeline for serverless: automate build, test, and deployment. Run unit tests in CI. Run integration tests in staging. Scan dependencies for vulnerabilities. Package function code efficiently. Deploy with infrastructure as code. Run smoke tests after deployment. Document CI/CD pipeline. Monitor pipeline success rate. Alert on pipeline failures. Review pipeline performance regularly
• Versioning and aliases: use versioning for function deployments. AWS Lambda supports versions and aliases. Azure Functions support deployment slots. Google Cloud Functions support traffic splitting. Use aliases for environment promotion. Document versioning strategy. Test version switching. Monitor version distribution. Rollback to previous version on failures. Clean up old versions regularly

Testing Serverless Functions

• Unit testing serverless functions: mock cloud services in unit tests. Test handler logic in isolation. Mock AWS SDK calls. Mock database connections. Mock HTTP requests. Test error handling paths. Test edge cases. Document testing strategy. Run unit tests in CI. Monitor test coverage. Aim for 80%+ coverage on critical functions
• Integration testing serverless functions: test function integration with cloud services. Use local emulation tools like LocalStack. Test with real cloud services in staging. Test end-to-end workflows. Test error scenarios. Document integration testing strategy. Run integration tests in CI. Monitor integration test results. Alert on integration test failures
• Load testing serverless functions: test function performance under load. Use tools like Artillery or k6. Simulate concurrent invocations. Monitor cold start frequency under load. Test auto-scaling behavior. Document load testing strategy. Run load tests before deployment. Monitor load test results. Compare with previous results. Alert on performance regressions

Tools and Platforms

• Serverless Framework: use Serverless Framework for multi-cloud deployments. Define functions and events in serverless.yml. Deploy with a single command. Support for AWS, Azure, and Google Cloud. Use plugins for extended functionality. Document Serverless Framework configuration. Test deployments in staging. Monitor deployment success. Review plugin compatibility regularly
• AWS SAM: use AWS SAM for Lambda deployments. Define functions in template.yaml. Use SAM CLI for local testing. Deploy with AWS CloudFormation. Support for canary deployments. Document SAM template structure. Test SAM templates locally. Monitor SAM deployment success. Review SAM version compatibility
• Local development tools: use local emulation for faster development. LocalStack for AWS services. Azure Functions Core Tools for local testing. Functions Framework for Google Cloud Functions. Document local development setup. Test local emulation accuracy. Monitor local development productivity. Review local tool versions regularly

Common Pitfalls

• Cold start mitigation failures: avoid common cold start mistakes. Do not load unnecessary dependencies at startup. Do not connect to databases outside the handler. Do not read large files at startup. Use provisioned concurrency for critical functions. Keep deployment packages small. Use lazy initialization for heavy resources. Document cold start mitigation strategy. Test cold start frequency. Monitor cold start duration
• Package size issues: keep function packages small. Remove unnecessary dependencies. Use tree shaking where possible. Minify code in production. Avoid bundling development dependencies. Use layers for shared dependencies. Document package optimization strategy. Test package size after build. Monitor package size trends. Alert on package size growth
• Concurrency limits: understand and configure concurrency limits. AWS Lambda reserved concurrency for critical functions. Azure Functions max instances. Google Cloud Functions max instances. Document concurrency configuration. Test concurrency behavior. Monitor concurrency usage. Alert on concurrency limit approaches. Request limit increases proactively

Best Practices

• Function granularity: keep functions small and focused on a single responsibility. Each function should do one thing well. Avoid monolithic functions that handle multiple concerns. Split complex logic into smaller functions. Use step functions for orchestration. Document function boundaries. Review function size regularly. Refactor large functions into smaller ones. Test each function independently. Monitor function execution time
• Resource cleanup: clean up resources after function execution. Close database connections. Close file handles. Clear temporary files. Release network connections. Document cleanup procedures. Test cleanup in error scenarios. Monitor resource leaks. Alert on resource exhaustion. Use finally blocks for cleanup. Review cleanup code regularly
• Logging and observability: implement structured logging in all functions. Include correlation IDs for tracing. Log function start and end times. Log input parameters (without sensitive data). Log error details with stack traces. Use distributed tracing for multi-function workflows. Document logging strategy. Test log output format. Monitor log volume. Alert on log anomalies. Use log aggregation tools. Review log retention policies
• Environment configuration: use environment variables for configuration. Keep configuration external from code. Use different configurations per environment. Document required environment variables. Validate environment variables at startup. Provide defaults for optional variables. Test with missing variables. Monitor configuration changes. Use configuration management tools. Review environment variable usage regularly

Cost Optimization

• Right-sizing function memory: optimize function memory allocation. AWS Lambda charges based on memory and execution time. Test different memory configurations. Higher memory may reduce execution time. Find the optimal memory-to-duration ratio. Document memory optimization strategy. Test memory configuration changes. Monitor cost per invocation. Review memory allocation quarterly. Alert on cost anomalies
• Reducing invocation frequency: reduce unnecessary function invocations. Use caching for frequent requests. Batch process events where possible. Use event filtering to skip irrelevant events. Combine multiple operations into single invocations. Document invocation reduction strategy. Test invocation frequency. Monitor invocation counts. Alert on invocation spikes. Review invocation patterns regularly
• Provisioned concurrency cost analysis: analyze provisioned concurrency costs. Compare with on-demand pricing. Use provisioned concurrency only for critical functions. Scale provisioned concurrency based on traffic patterns. Document provisioned concurrency strategy. Test cost impact. Monitor provisioned concurrency usage. Review provisioned concurrency configuration monthly. Adjust based on traffic patterns

Troubleshooting Guide

• Debugging cold starts: identify cold start causes. Check initialization code. Review dependency loading. Monitor cold start frequency. Use X-Ray or similar tracing tools. Document cold start debugging steps. Test cold start scenarios. Review package size impact. Monitor cold start duration trends. Optimize initialization code. Use provisioned concurrency for critical paths
• Debugging function timeouts: identify timeout causes. Check downstream service latency. Review function execution time. Monitor database query performance. Check network latency. Document timeout debugging steps. Test with different timeout values. Monitor timeout frequency. Optimize slow operations. Use async processing for long-running tasks
• Debugging deployment failures: identify deployment failure causes. Check IAM permissions. Review CloudFormation errors. Check package size limits. Validate template syntax. Document deployment debugging steps. Test deployment in staging. Monitor deployment success rate. Review deployment logs. Use deployment rollback for quick recovery

Monitoring and Alerting

• Key metrics to monitor: monitor invocations, errors, duration, and throttles. Track cold start frequency and duration. Monitor concurrent executions. Track memory usage. Monitor DLQ depth. Track API Gateway latency. Document monitoring strategy. Configure dashboards for key metrics. Review metrics regularly. Adjust thresholds based on trends. Alert on critical metrics
• Alert configuration: set alerts on error rate above 1%. Alert on timeout frequency spikes. Alert on throttle increases. Alert on DLQ depth growth. Alert on cost anomalies. Use multi-level alerts: warning and critical. Document alert thresholds. Test alert delivery. Review alert effectiveness monthly. Reduce alert noise. Use runbooks for each alert
• Distributed tracing: implement distributed tracing for serverless workflows. Use AWS X-Ray for Lambda. Use Azure Application Insights for Functions. Use Google Cloud Trace for Cloud Functions. Trace requests across multiple functions. Document tracing strategy. Test trace coverage. Monitor trace sampling. Review trace data regularly. Use traces for performance optimization

Advanced Patterns

• Fan-out/fan-in pattern: use fan-out for parallel processing. Publish events to SNS or EventBridge. Multiple Lambda functions process in parallel. Use fan-in to aggregate results. SQS or Kinesis for aggregation. Document fan-out/fan-in strategy. Test parallel processing. Monitor function concurrency. Alert on parallelism limits. Use step functions for complex fan-out patterns
• Event sourcing pattern: store all changes as events. Use EventBridge or Kafka for event streaming. Rebuild state from event log. Enable time-travel queries. Document event sourcing strategy. Test event replay. Monitor event store size. Review event schema regularly. Use snapshots for performance. Handle schema evolution carefully
• Saga pattern: use sagas for distributed transactions. Implement compensating actions for rollback. Use step functions for saga orchestration. Document saga pattern usage. Test compensating actions. Monitor saga completion rate. Alert on saga failures. Review saga design regularly. Handle saga timeouts gracefully

Migration Strategies

• Migrating from monolith to serverless: break down monolithic applications into smaller functions. Identify bounded contexts for function boundaries. Migrate one endpoint at a time. Use API Gateway as a facade during migration. Run both systems in parallel. Document migration strategy. Test each migrated function. Monitor performance comparison. Switch traffic gradually. Complete migration after validation
• Migrating between cloud providers: abstract cloud-specific code behind interfaces. Use infrastructure as code for portability. Test on target platform before migration. Monitor for behavioral differences. Document migration runbook. Test failback procedures. Review migration progress. Complete DNS switch after validation. Monitor for post-migration issues
• Migrating from containers to serverless: identify suitable workloads for serverless. Start with event-driven workloads. Keep stateless functions. Use managed services for state. Document migration strategy. Test function performance. Monitor cost comparison. Review migration progress. Complete migration after validation

Compliance and Governance

• Serverless SLAs: define SLAs for serverless APIs. API response time under 200ms. Function execution time under 1 second. Error rate below 0.1%. Track SLA compliance. Alert on SLA violations. Document SLA definitions. Review SLAs quarterly. Communicate SLA status. Use SLA for prioritization
• Serverless reporting: generate weekly serverless reports. Include invocation count, error rate, cost. Highlight performance trends. Share with stakeholders. Document reporting methodology. Automate report generation. Review report content. Track metrics over time. Use reports for planning and optimization
• Audit and compliance: log all function invocations. Track who triggered each function. Maintain audit trail of configuration changes. Use cloud-native audit tools. Document audit strategy. Test audit log completeness. Monitor audit log retention. Review compliance requirements regularly. Alert on audit log gaps

Automation and Tooling

• Infrastructure as code automation: automate infrastructure provisioning with IaC tools. Use AWS SAM or Serverless Framework for Lambda. Use Terraform for multi-cloud deployments. Version all IaC templates. Store templates in version control. Document IaC strategy. Test IaC changes in staging. Monitor IaC deployment success. Review IaC templates regularly. Use modular templates for reuse
• Automated testing pipeline: automate all testing in CI/CD pipeline. Run unit tests on every commit. Run integration tests on pull requests. Run load tests before deployment. Run security scans on every build. Document testing pipeline. Monitor test success rate. Alert on test failures. Review test coverage quarterly. Optimize test execution time
• Automated deployment rollback: implement automated rollback for failed deployments. Use CloudWatch alarms for rollback triggers. Configure health checks for deployment validation. Document rollback strategy. Test rollback in staging. Monitor rollback frequency. Alert on rollback events. Review rollback thresholds regularly. Minimize rollback time

Sustainability

• Green serverless computing: serverless is inherently green. Pay only for actual usage. No idle resources consuming power. Use carbon-aware scheduling for batch jobs. Schedule heavy workloads during low-carbon periods. Document sustainability strategy. Monitor carbon footprint. Review energy usage regularly. Optimize function efficiency. Use cloud provider sustainability tools
• Resource efficiency: optimize function resource usage. Right-size memory allocation. Minimize execution time. Reduce unnecessary invocations. Use efficient data structures. Optimize algorithms. Document resource efficiency strategy. Monitor resource utilization. Review efficiency metrics quarterly. Optimize based on usage patterns
• Waste reduction: reduce serverless waste. Eliminate unused functions. Remove unused dependencies. Clean up old versions. Delete unused API routes. Monitor for idle resources. Document waste reduction strategy. Review resource usage monthly. Alert on waste indicators. Automate cleanup procedures

Industry Standards and Frameworks

• Well-Architected Framework: follow cloud provider Well-Architected Framework. AWS Well-Architected Tool for Lambda. Azure Well-Architected Review for Functions. Google Cloud Architecture Framework. Review architecture regularly. Document review findings. Address critical issues. Monitor compliance. Use framework for design decisions
• Serverless design principles: follow serverless design principles. Design for failure. Use managed services. Implement idempotency. Design for scale. Use event-driven architecture. Minimize cold starts. Optimize cost. Document design principles. Review architecture against principles. Train team on principles. Use principles for code reviews
• Compliance frameworks: align serverless architecture with compliance frameworks. SOC 2 for security. PCI DSS for payments. HIPAA for healthcare. GDPR for data privacy. ISO 27001 for security management. Document compliance requirements. Test compliance controls. Monitor compliance status. Review compliance regularly. Use compliance for architecture decisions

Reporting and Communication

• Performance reporting: generate weekly performance reports for serverless functions. Include invocation count, average duration, error rate, and cost. Compare with previous week. Highlight trends and anomalies. Share with engineering team. Document reporting methodology. Automate report generation. Review report content monthly. Use reports for optimization decisions
• Cost reporting: generate monthly cost reports for serverless workloads. Break down by function, service, and environment. Compare with budget. Identify cost optimization opportunities. Share with stakeholders. Document cost reporting strategy. Automate cost report generation. Review cost trends quarterly. Use reports for budget planning
• Incident reporting: document all serverless incidents. Include root cause, impact, and resolution. Share incident reports with team. Conduct post-mortem reviews. Document action items. Track action item completion. Review incident patterns. Use incidents for improvement. Communicate incidents to stakeholders. Maintain incident history

Advanced Optimization

• Provisioned concurrency tuning: tune provisioned concurrency for optimal performance. Start with minimum provisioned concurrency. Monitor cold start frequency. Adjust based on traffic patterns. Use auto-scaling for provisioned concurrency. Document tuning strategy. Test configuration changes. Monitor cost impact. Review configuration monthly. Optimize for cost and performance balance
• Memory tuning: tune function memory for optimal performance. Test with different memory values. Monitor execution time changes. Find optimal memory-to-duration ratio. Document memory tuning strategy. Test memory changes in staging. Monitor cost impact. Review memory allocation quarterly. Use AWS Lambda Power Tuning for optimization
• Code optimization: optimize function code for performance. Minimize cold start dependencies. Use lazy initialization. Optimize database queries. Cache frequently accessed data. Use efficient data structures. Document code optimization strategy. Review code regularly. Monitor performance impact. Use profiling tools for optimization

Frequently Asked Questions

Q: How do I handle database connections in Lambda? A: Use connection pooling with a lightweight proxy like RDS Proxy, or implement your own connection reuse logic. Opening a new database connection on every invocation is slow and can exhaust the database’s connection limit.

Q: Can I run a full-stack application on serverless? A: Yes, but evaluate the trade-offs. Static sites and APIs are excellent fits. Long-running WebSocket connections or stateful sessions may be better served by containers or EC2.

Q: Is serverless cheaper than traditional servers? A: It depends on traffic patterns. For sporadic traffic, serverless is usually cheaper because you pay only for requests. For sustained high traffic, provisioned containers or EC2 can be more cost-effective.

Q: How do I test Lambda functions locally? A: Use the AWS SAM CLI or Serverless Framework to emulate API Gateway and Lambda locally. These tools mount your code in a Docker container that matches the AWS runtime environment.

Is this solution production-ready?

Yes. The code examples above show tested implementations. Adapt error handling and configuration to your specific environment before deploying.

What are the performance characteristics?

Performance depends on your data volume and infrastructure. The solutions shown prioritize clarity. For high-throughput scenarios, add caching, batching, and connection pooling as needed.

How do I debug issues with this approach?

Start with the minimal example above. Add logging at each step. Test with small inputs first, then scale up. Use your language’s debugger to step through edge cases.