Post-Deployment Verification Checklist Template
A checklist template for verifying deployments: health checks, smoke tests, metric validation, and rollback readiness before declaring all-clear.
Run this checklist before you call a deployment done. Pair it with the Release Notes Template for communication and the CI/CD Pipeline Guide for automation.
Overview
A deployment isn’t complete when the code reaches production — it’s complete when you’ve verified that production behaves as expected. Post-deployment verification is where you catch what CI tests can’t: config mismatches, missing env vars, broken integrations, and the regressions that only show up once real users hit the release.
This template covers six areas:
- Health checks — application, database, and dependency connectivity
- Smoke tests — critical user paths under real conditions
- Metrics validation — error rates, latency, resource usage
- Security and logs — headers, certificates, and new error patterns
- Rollback readiness — whether you can revert quickly if something breaks
- Sign-off — who verified the deployment and when
Copy the template below, keep the sections that apply to your stack, and wire the automated checks into CI so the checklist runs the same way every time — a checklist that depends on memory is a checklist that gets skipped.
When to Use
- Every production deployment — no exceptions, even for small changes
- Staging deployments — verify before promoting to production
- Infrastructure changes — config updates, scaling events, DNS changes
- Database migrations — confirm schema changes didn’t break queries
- Rollback verification — confirm the rollback restored expected behavior
- After hotfixes — urgent fixes need verification before sign-off too
The only case where this doesn’t apply is a change that never reaches production — a docs commit, a README fix, a test-only branch. Everything else gets verified.
Template
# Post-Deployment Verification: [Service] v[X.Y.Z]
## Deployment Info
| Field | Value |
|-------|-------|
| **Deployer** | [name] |
| **Timestamp** | [YYYY-MM-DD HH:MM UTC] |
| **Environment** | [staging / production] |
| **Rollback commit** | [SHA] |
## Health Checks
- [ ] Application starts without errors
- [ ] Health endpoint returns 200: `GET /health`
- [ ] Readiness probe passes
- [ ] Liveness probe passes
- [ ] Database connectivity confirmed
- [ ] External dependency connectivity confirmed
## Smoke Tests
- [ ] Core user flow: [login → action → logout]
- [ ] API returns expected status codes
- [ ] Critical path payment flow
- [ ] Admin dashboard loads
## Metrics Validation
| Metric | Pre-Deploy | Post-Deploy | Delta | Alert? |
|--------|-----------|-------------|-------|--------|
| Error rate | 0.05% | ___ | ___ | ___ |
| Latency p95 | 120ms | ___ | ___ | ___ |
| CPU usage | 45% | ___ | ___ | ___ |
## Security & Logs
- [ ] Security headers present on key routes (`curl -I` shows CSP, HSTS)
- [ ] TLS certificate valid, expiry > 30 days
- [ ] No spike in 401/403 responses vs pre-deploy baseline
- [ ] No new exception patterns in application logs
- [ ] No secrets or credentials leaked in logs
## Rollback Readiness
- [ ] Rollback script tested in last 30 days
- [ ] Previous version artifacts available
- [ ] Database migration is backward-compatible
- [ ] Feature flags can disable new code instantly
## Sign-Off
| Role | Name | Time |
|------|------|------|
| Deployer | | |
| On-call | | |
Lifecycle
The Security & Logs block is newer than the others — add it even if your team has never filled it in before. Deployments that break TLS, strip a header, or start logging tokens don’t show up in a /health response; they show up in a breach report a month later.
Pre-deploy preparation
Fill out the deployment info before anything leaves CI. Confirm rollback readiness — a rollback that’s never been tested isn’t a rollback, it’s a hope. Make sure someone’s on call and snapshot your metric baselines so you’ve got something to compare against.
Deploy and immediate verification
Run health checks the moment the deploy finishes. When a check fails, stop there and look into it — don’t move on to smoke tests, because a service that isn’t healthy will fail every downstream check anyway.
Post-deploy monitoring
Run smoke tests at 5 minutes and metric validation at 15 minutes. Keep watching for at least 1 hour; extend to 24 hours for high-risk changes like schema migrations or payment flows. What you’re watching for in that window isn’t just “does it still work” — it’s the slow signals: memory climbing steadily, error rate drifting upward, queues backing up. Those don’t trip a health check, but they show up in the graphs before they show up in user complaints.
Sign-off and archiving
Once all checks pass, collect sign-offs from the deployer and the on-call engineer. Archive the completed checklist in the deployment log and link it to the release notes and any related ADRs — it becomes evidence if an incident review asks “did we verify this?”
Adapting the checklist to risk
Not every deploy deserves the full ceremony. Scale the checklist to what could actually break:
| Risk level | Examples | Scope |
|---|---|---|
| Low | Content changes, CSS tweaks, config flags | Health checks + spot smoke test |
| Medium | New endpoints, dependency bumps, infra changes | Full checklist |
| High | Schema migrations, payment flows, auth changes | Full checklist + 24 h monitoring + third sign-off |
When in doubt, run the full version — the checklist costs ten minutes, while a missed production issue costs the on-call’s night.
Verification Commands
The checklist items above are the what; these snippets are the how. Adapt the URLs and thresholds to your stack.
Health endpoint
curl -fsS --retry 3 --retry-delay 2 \
https://api.example.com/health || echo "HEALTH CHECK FAILED"
Smoke test with k6
// smoke.js — run with: k6 run smoke.js
import http from 'k6/http';
import { check } from 'k6';
export const options = { vus: 1, duration: '30s' };
export default function () {
const res = http.get('https://api.example.com/health');
check(res, { 'health is 200': (r) => r.status === 200 });
const login = http.post('https://api.example.com/login',
JSON.stringify({ user: 'smoke@test.dev', pass: '***' }),
{ headers: { 'Content-Type': 'application/json' } });
check(login, { 'login works': (r) => r.status === 200 });
}
Error rate from access logs
# 5xx share in the last 15 minutes of nginx logs
awk -v cutoff="$(date -d '15 minutes ago' '+%d/%b/%Y:%H:%M')" \
'$4 > "["cutoff { total++; if ($9 >= 500) errors++ }
END { printf "%.3f%% (%d/%d)\n", 100*errors/total, errors, total }' \
/var/log/nginx/access.log
New exceptions in application logs
# Errors seen post-deploy that didn't appear in the pre-deploy window
grep -i "exception\|error" /var/log/app/current.log | \
awk '{print $NF}' | sort | uniq -c | sort -rn | head -10
If any of these fail, the checklist row fails — mark it, investigate, and decide between rollback and fix-forward before sign-off.
Filled Example
# Post-Deployment Verification: payments-api v3.2.1
## Deployment Info
| Field | Value |
|-------|-------|
| **Deployer** | Jane Doe |
| **Timestamp** | 2026-07-15 14:30 UTC |
| **Environment** | production |
| **Rollback commit** | a1b2c3d4e5f6 |
## Health Checks
- [x] Application starts without errors
- [x] Health endpoint returns 200: `GET /health`
- [x] Readiness probe passes
- [x] Liveness probe passes
- [x] Database connectivity confirmed
- [x] External dependency connectivity confirmed (Stripe, PayPal)
## Smoke Tests
- [x] Core user flow: login → create order → checkout → logout
- [x] API returns expected status codes (200, 201, 400, 401, 500)
- [x] Critical path: credit card payment flow
- [x] Admin dashboard loads with new metrics
## Metrics Validation
| Metric | Pre-Deploy | Post-Deploy | Delta | Alert? |
|--------|-----------|-------------|-------|--------|
| Error rate | 0.05% | 0.04% | -0.01% | No |
| Latency p95 | 120ms | 118ms | -2ms | No |
| CPU usage | 45% | 48% | +3% | No |
| Memory | 512MB | 520MB | +8MB | No |
| Throughput | 850 req/s | 860 req/s | +10 | No |
## Security & Logs
- [x] Security headers present (`curl -I https://api.example.com` → CSP + HSTS)
- [x] TLS certificate valid until 2027-01-20
- [x] 401/403 rate unchanged (0.8% pre, 0.7% post)
- [x] No new exception patterns (checked `payments-api` error log)
- [x] No secrets in logs
## Rollback Readiness
- [x] Rollback script tested in last 30 days
- [x] Previous version artifacts available (v3.2.0)
- [x] Database migration is backward-compatible
- [x] Feature flags can disable new code instantly
## Sign-Off
| Role | Name | Time |
|------|------|------|
| Deployer | Jane Doe | 14:35 UTC |
| On-call | Bob Smith | 14:40 UTC |
Timing Guidelines
| Check Type | When | Duration |
|---|---|---|
| Health checks | Immediately after deploy | 2 minutes |
| Smoke tests | 5 minutes post-deploy | 10 minutes |
| Metric validation | 15 minutes post-deploy | 10 minutes |
| Security & log review | 15 minutes post-deploy | 5 minutes |
| Full validation | 1 hour post-deploy | Ongoing monitoring |
Variants
Canary deployment verification
For canary deployments, compare metrics between the canary group and the stable group. Watch the error-rate delta, latency delta, and user feedback. Roll the canary back if any metric degrades beyond its threshold, and scale gradually: 1% → 5% → 25% → 50% → 100%.
Blue-green deployment verification
For blue-green deployments, run the new version (green) alongside the old (blue). Route a portion of traffic to green, verify, then switch the router. Keep blue running so you can roll back instantly if green shows issues. For the rollout mechanics, see the Blue-Green Deployment recipe.
Database migration verification
For deployments with schema changes, verify: migration completed, backward compatibility, query performance, and index usage. Have a rollback migration script tested in staging. See the Database Schema Documentation Template for migration tracking.
Static site or CDN deployment
For static sites and CDN-backed frontends, the checklist shrinks but doesn’t disappear. Verify the new asset versions are actually being served (check a cache-busted URL and a fresh one), confirm cache headers are correct, spot-check redirects and the 404 page, and purge the CDN cache if stale content lingers. Skip database and probe checks entirely.
API service behind a load balancer
For APIs sitting behind a load balancer, drain connections per node during the rollout and hit each node’s health check individually — only when every node passes should the pool go back to full traffic. Watch for nodes that pass the LB health check but fail real requests: a /health that only checks process liveness won’t catch a broken database pool.
Scheduled jobs and workers
For cron jobs, queues, and background workers, “is it up” means something different: the process runs but there’s no traffic to smoke-test. Verify instead that the job actually executed — check its last-run timestamp, its run duration against the baseline, and that the queue depth is draining rather than growing. A worker that’s alive but silently skipping its schedule is the classic failure here.
What Works
- Automate the checklist — CI should fail the deploy if health checks don’t pass. See the CI/CD Pipeline Guide for integration.
- Test rollback before you need it — an untested rollback is a gamble. See the Disaster Recovery Plan Template for broader planning.
- Keep the previous version warm — blue-green deployments let you switch back instantly. See the Deployment Strategies Guide for patterns.
- Use synthetic monitoring — external probes catch issues your internal checks can’t see.
- Log correlation IDs — tag deploys with a correlation ID so you can trace post-deploy errors back to the release in structured logs.
- Document actual vs expected — deviations become your incident response data.
- Set alert thresholds before deploying — know what “normal” looks like for the new version.
- Have a rollback decision timer — if metrics degrade within 15 minutes, roll back automatically.
- Keep the checklist lean — a 50-item checklist gets skimmed; a 15-item one gets done. Cut anything nobody would ever act on.
Common Mistakes
- Skipping verification because “the tests passed” — production traffic is the real test.
- Not checking error rates after deploy — a deploy that increases errors by 0.1% is a failed deploy.
- Assuming rollback is trivial — test your rollback procedure quarterly. If you’re building the wider recovery plan around it, the Disaster Recovery Plan Template covers that.
- Deploying without on-call coverage — if verification fails, someone must be available to respond.
- Checking too few metrics — CPU and error rate aren’t enough; check business metrics too.
- Filling the checklist once and never archiving it — if the completed checklist isn’t stored with the deployment log, the sign-off means nothing in a later review.
- No rollback decision criteria — if nobody wrote down what triggers a rollback, nobody will pull the trigger in time.
Automation
CI/CD integration
post-deploy-verification:
stage: post-deploy
script:
- curl -fsS --retry 3 https://api.example.com/health || exit 1
- k6 run smoke.js --env production
- npm run check-metrics -- --threshold=baseline.json
- node scripts/scan-error-log.js --since-deploy
after_script:
- npm run notify-team -- --status=$CI_JOB_STATUS
only:
- main
Automated health probes
Set up synthetic monitoring that hits your health endpoints every 60 seconds and alerts on any non-200 response. Pingdom, UptimeRobot, and AWS CloudWatch Synthetics all handle this without extra infrastructure.
Metric-based auto-rollback
Wire your deploy tool to revert automatically if the error rate crosses a threshold inside the first 15 minutes. Both Argo Rollouts on Kubernetes and AWS CodeDeploy do this out of the box.
Troubleshooting
- Pipeline fails silently: turn on verbose logging and keep artifacts between stages, or you’ll have nothing to inspect when it happens again.
- Container crashes on startup: check that environment variables, secrets, and config files are mounted correctly, and read the first 50 lines of logs before scaling replicas.
- Deployment rolls back repeatedly: verify health checks, resource limits, and startup probes — a readiness probe that never passes is a common cause of rolling restarts.
- Slow CI builds: cache dependencies and Docker layers. Split large test suites into parallel jobs to reduce wall-clock time.
- Drift between environments: use infrastructure-as-code and immutable artifacts.
- Health check passes but users see errors: your health endpoint is too shallow. Add dependency checks (database, cache, queues) to a
/health/deepvariant and run the smoke tests against it.
Further Reading
- Google SRE Workbook — Release Engineering — how Google structures release and post-release verification.
- Argo Rollouts documentation — automated canary analysis and metric-based rollback for Kubernetes.
- AWS CodeDeploy deployment documentation — auto-rollback triggers and traffic shifting options.
- Blue-green deployment — Martin Fowler — the canonical write-up on the pattern behind instant rollback.
Key Takeaways
- Verification is what makes a deployment “done” — code reaching production isn’t the finish line.
- Health checks first, smoke tests at 5 minutes, metrics at 15 — and roll back on degradation instead of debugging live.
- The checklist only works if it’s archived with the deploy record and the rollback path is tested before it’s needed.
Frequently Asked Questions
How long should I monitor after deployment?
At minimum: health checks immediately, smoke tests at 5 minutes, metrics at 15 minutes, and business metrics at 1 hour. For high-risk changes, extend to 24 hours with a follow-up review.
What if smoke tests fail but metrics look fine?
Investigate immediately. Smoke tests cover critical user paths; metric dashboards may not catch functional regressions. Don't sign off until the smoke tests are green.
Should I automate the checklist or run it manually?
Automate health checks and smoke tests in CI. See the Monitoring and Alerting Guide for probes. Manual verification is for business-critical judgment calls ("does the checkout flow feel right?"). The goal is automated gates with human oversight.
What metrics should I track post-deployment?
Track four categories: infrastructure (CPU, memory, disk), application (error rate, latency, throughput), business (conversion rate, revenue, active users), and external (third-party API response time, webhook delivery rate). Compare every one of them to the pre-deploy baseline.
When should I roll back vs fix forward?
Roll back when the error rate increases considerably, critical user flows break, or security is compromised. Fix forward when the issue is cosmetic, a minor bug with a quick fix available, or when rollback would cause data loss. When in doubt, roll back — you can always redeploy once you know what broke; you can't un-burn a bad release.
How do I handle deployments with breaking database changes?
Use the expand-contract pattern: deploy new code that supports both old and new schema, migrate the data, then deploy code that uses only the new schema. Never ship breaking schema changes and code changes in the same release. See the Database Schema Documentation Template for migration tracking.
Who should fill out the checklist?
Whoever deploys fills in the info block and runs the health checks; the on-call engineer then checks the metrics and countersigns. For high-risk changes, a third person (release manager or tech lead) should review the completed checklist before declaring success.
What if I don't have a rollback script?
Build one before your next deploy — it doesn't need to be fancy. It should stop the new version, restore the previous artifact, revert database migrations if needed, and restart the service. Test it in staging first; without a tested rollback, you're deploying without a safety net.
How do I verify external dependencies post-deploy?
Probe each external dependency yourself: hit the payment gateway's health endpoint, check the email service responds, confirm the CDN serves assets. Don't assume external services are healthy because your app started — they can be degraded in ways that only surface under specific operations.
What if the deployment succeeds but performance degrades?
Check whether the degradation sits within your thresholds. If p95 moved less than 10% and the error rate is flat, keep a close eye on it. If p95 moved more than 20% or errors spiked, roll back right away — degradation that sharp usually means a missing index, an N+1 query, or resource contention.
Related Resources
Deployment Checklist Template
A copy-paste verification checklist for safe production deployments, from pre-merge tests to post-release monitoring.
DocRelease Notes Template
A release notes template that communicates changes clearly to users, operators, and stakeholders with categories, upgrade instructions, and known issues.
GuideCI/CD Pipeline Guide
A practical guide to building CI/CD pipelines with GitHub Actions, testing, deployment strategies, and rollback procedures.
GuideComplete Guide to Monitoring and Alerting
Build a production monitoring stack with Prometheus, Grafana, and AlertManager — plus the RED/USE methods, alert design that avoids fatigue, SLOs with error budgets, runbooks, and dashboards that answer in five seconds.
RecipeBlue-Green Deployment Recipe
Deploy with zero downtime using blue-green environments, instant traffic switching, and automated rollback capabilities.
RecipeImplement Graceful Shutdown and Zero-Downtime Restarts
How to implement graceful shutdown and zero-downtime restarts for web servers, workers, and containers