Service Level Objective (SLO) Document Template

Use this template to define reliability targets that balance user happiness with engineering velocity. See Monitoring and Alerting Guide for metric collection and On-Call Incident Response Guide for escalation procedures.

How to Implement SLOs Step by Step

Don’t try to roll out SLOs to every service at once. Pick one service, work through these eight steps, learn what works for your team, then expand. Most teams fail at SLOs because they try to do too much too fast — start small, prove the value, then scale.

1. Pick one service — start with the service that hurts most when it’s down. Don’t roll out SLOs to 20 services at once. Pick one, learn, then expand.
2. Identify the user journey — what does a user do with this service? “User makes a payment”, “user logs in”, “user uploads a file”. Write it down.
3. Define what “successful” means — for each step in the journey, what counts as success? “Transaction completed in < 500ms”, “login returns 200 in < 200ms”. These are your SLIs.
4. Set the SLO target — look at current performance. If you’re at 99.5% today, don’t promise 99.99%. Set the target slightly above current performance to leave room for improvement.
5. Calculate the error budget — 100% - SLO target. For 99.9% availability over 30 days, the budget is 0.1% = 43.2 minutes of allowed downtime.
6. Define burn rate alerts — alert when the burn rate is fast enough to exhaust the budget before the window resets. Fast burn (2% in 1h) pages; slow burn (5% in 6h) creates a ticket.
7. Write the error budget policy — what happens when the budget is low? Deploy freeze? All hands on reliability? Write it down before you need it.
8. Review quarterly — put the review on the calendar. Check if the SLO is still achievable, if the SLIs still match user experience, and if the policy is still working.

SLO Flow: SLI to Alert

Template

# SLO: [Service Name]

## Overview
| Field | Value |
|-------|-------|
| **Service** | [name] |
| **Owner** | [team or individual] |
| **Review date** | [quarterly] |

## SLIs (Service Level Indicators)

| SLI | Description | Measurement |
|-----|-------------|-------------|
| **Availability** | Ratio of successful requests | (total - errors) / total |
| **Latency** | Response time distribution | p95, p99 per endpoint |
| **Throughput** | Requests per second | RPS at peak |

## SLOs (Targets)

| Objective | Target | Measurement Window |
|-----------|--------|-------------------|
| Availability | 99.9% | Rolling 30 days |
| Latency p95 | < 200ms | Rolling 7 days |
| Error rate | < 0.1% | Rolling 24 hours |

## Error Budget

- **Budget:** 100% - SLO target (e.g., 0.1% for 99.9% availability)
- **Period:** 30 days
- **Policy:** When error budget is > 50% consumed in < 50% of period, freeze non-critical deploys

## Alerting Thresholds

| Severity | Threshold | Response |
|----------|-----------|----------|
| Page | Error budget 10% consumed in 1 hour | On-call responds immediately |
| Ticket | Error budget 50% consumed in 7 days | Team reviews in next sprint |

## Dependencies

| Dependency | Their SLO | Impact If They Miss |
|------------|-----------|-------------------|
| Payment API | 99.95% | Our checkout SLO drops |
| Identity Provider | 99.9% | Login failures affect availability |

Choosing the Right SLO

User Impact	Typical SLO	Rationale
Critical path (payments, login)	99.99% (4 nines)	Downtime directly blocks revenue
Important but not critical	99.9% (3 nines)	~43 min downtime/month acceptable
Internal tools	99% (2 nines)	~7 hours downtime/month acceptable

Error Budget Policy

Budget remaining | Policy
-------------------|--------
> 50%              | Normal operations
25-50%             | Deploy freeze for risky changes
< 25%              | Deploy freeze except critical fixes
< 10%              | All hands on reliability; halt feature work

SLI Selection Guide

Service Type	Good SLI	Bad SLI	Why
HTTP API	Request success rate (2xx+3xx / total)	CPU usage	Users care about responses, not CPU
gRPC service	grpc_io_server_completed_rpcs (OK / total)	Memory usage	Users care about RPCs succeeding
Batch job	Freshness (time since last successful run)	Disk I/O	Users care about data being current
Queue consumer	Lag (messages behind head)	Network throughput	Users care about processing keeping up
Database	Query success rate + p99 latency	Connection count	Users care about queries working fast

RED vs USE model

• RED (for services): Rate, Errors, Duration. Use for HTTP APIs, gRPC services, message consumers.
• USE (for infrastructure): Utilization, Saturation, Errors. Use for databases, caches, load balancers.
• Don’t mix them: a RED SLI on a database misses saturation; a USE SLI on an API misses user experience.

SLI anti-patterns

• CPU usage as SLI — users don’t care about CPU. High CPU might be fine (busy but fast); low CPU might be bad (deadlocked).
• Uptime as SLI — a service can return errors and still be “up”. Measure success rate, not reachability.
• Average latency as SLI — averages hide tail latency. Use p95 or p99 percentiles.
• Error count (not rate) as SLI — 100 errors out of 1M requests is fine; 100 errors out of 200 isn’t. Use rates, not counts.

What Works

• Start with user-visible metrics — “CPU usage” isn’t an SLI; “request success rate” is. See Monitoring and Alerting Guide for metric selection.
• Set SLOs based on current performance — if you’re at 99.5% today, don’t promise 99.99%
• Review quarterly — adjust targets based on user feedback and engineering capacity
• Distinguish SLI, SLO, and SLA — SLI is the metric, SLO is the target, SLA is the contractual promise to customers

Common Mistakes

• SLOs that are too loose — 99% for a payment API means 7 hours of downtime is “acceptable”
• SLOs that are too tight — 99.999% requires expensive infrastructure for marginal user benefit
• Tracking SLIs no one looks at — every SLI needs an owner and a review cadence
• Ignoring error budget burn — the budget exists to protect engineering velocity, not to be ignored. See Incident Postmortem Template for when SLOs are breached.

SLO Troubleshooting

• Error budget burns too fast: check for a recent deploy that introduced a regression. Look at the error rate before and after the deploy. If the deploy caused it, roll back. Don’t adjust the SLO — fix the deploy.
• SLO is met but users complain: your SLI doesn’t capture what users experience. Add a user-journey-based SLI (e.g., “checkout completes in < 2s”) instead of an infrastructure metric.
• Alert fatigue from burn rate alerts: tune the alert windows. Fast burn (2% in 1h) catches outages; slow burn (5% in 6h) catches chronic issues. If you only have one, prefer slow burn — it’s more actionable.
• SLO review never happens: assign a named owner and put the review on the team calendar. A SLO without a review cadence is a document, not a practice.
• Error budget is always at 0: your SLO is too aggressive. Track 2-3 months of data, then adjust to a realistic target. Don’t lower it after a single bad month.

Further Reading

• Google SRE Workbook — SLO Engineering: the canonical reference for SLO design and error budget policies.
• Google SRE Book — Service Level Objectives: foundational concepts, SLI/SLO/SLA distinctions.
• Sloth — SLO Generator for Prometheus: open-source tool to generate Prometheus SLO rules from declarative specs.
• Nobl9 — SLO Platform: managed SLO platform with error budget tracking and burn rate alerts.
• OpenTelemetry — SLI Metrics: semantic conventions for service-level metrics.

Production Notes

• Start with one SLO — pick the service that hurts most when it’s down. Don’t roll out SLOs to 20 services at once.
• Track error budget in a dashboard — Grafana with a stat panel showing “budget remaining” makes the SLO visible to the whole team.
• Alert on burn rate, not on SLO breach — a 99.9% SLO doesn’t mean “page when availability drops below 99.9%”. It means “page when the burn rate is fast enough to exhaust the budget before the window resets”.
• Review SLOs quarterly — user expectations change, infrastructure improves, and SLOs that were right 6 months ago may be too loose or too tight now.

Key Takeaways

• SLOs are a contract between the team and users — they define what “reliable enough” means and give you a budget for taking risks.
• Error budget is the whole point — without it, SLOs are just numbers. The budget tells you when to slow down and when you can move fast.
• SLI selection matters more than SLO targets — a wrong SLI (CPU usage) makes the SLO useless; a right SLI (request success rate) makes it actionable.
• Review or retire — a SLO nobody looks at is worse than no SLO. Review quarterly or delete it.

SLO Maturity Model

Level	What you’ve got	What you’re working on
1 — Ad hoc	No SLOs; “is it up?” is the only metric	Defining what users care about
2 — Basic	1-2 SLOs (availability, latency); manual tracking	Automating SLI collection and dashboards
3 — Managed	2-4 SLOs per service; error budget tracked; burn rate alerts	Error budget policy enforcement
4 — Optimized	Multi-window SLOs; automated policy enforcement; quarterly reviews	Predictive budget tracking, SLO-based release gating

Most teams sit at Level 2. Getting to Level 3 requires: (1) error budget dashboards, (2) burn rate alerts wired to PagerDuty, (3) a written policy the team agrees to follow. Level 4 is advanced — only invest when you’ve got 10+ services with SLOs and the tooling to support automated enforcement.

Variants

Context	Approach	Notes
Startup	Basic SLOs: availability and latency	Target Level 2 first
Enterprise	Multi-window SLOs + error budget policies	Level 3-4 with quarterly reviews
Microservices	Per-service SLOs + dependency SLOs	Include upstream service SLOs
Batch processing	Freshness and throughput SLOs	Latency doesn’t apply the same way

SLO Examples by Industry

E-commerce checkout

SLI	Target	Window	Alert
Checkout success rate	99.95%	30 days	Page at 10x burn
Checkout p95 latency	< 800ms	7 days	Ticket at 5x burn
Payment gateway availability	99.9%	30 days	Page at 10x burn

Checkout is on the critical path. A 99.95% target gives you ~22 minutes of allowed downtime per month. Track the full journey: “add to cart” → “enter payment” → “confirmation”.

SaaS API platform

SLI	Target	Window	Alert
API success rate (per endpoint)	99.9%	30 days	Page at 10x burn
API p99 latency	< 300ms	30 days	Ticket at 5x burn
Auth token issuance	99.95%	30 days	Page at 10x burn

For API platforms, track SLOs per endpoint, not just per service. A slow /health endpoint is fine; a slow /checkout endpoint isn’t.

Internal batch pipeline

SLI	Target	Window	Alert
Job freshness (time since last success)	< 4 hours	Rolling	Page if > 6 hours
Job success rate	99%	30 days	Ticket at 5x burn
Data completeness	100%	Per run	Page if < 100%

Batch jobs don’t need latency SLOs. Freshness and completeness matter more than speed.

Complete SLO Example

Here’s a filled-in SLO for a payment service. It shows two SLIs (success rate and latency), the SLO targets, error budget math, and burn rate alert thresholds. Copy this structure and adapt the numbers to your service.

=== SLO: payment-service ===

SLI: Transaction success rate
  Definition: HTTP 2xx+3xx requests / total HTTP requests
  Source: Prometheus (http_requests_total)
  Query: sum(rate(http_requests_total{service="payment",status!~"5.."}[5m])) / sum(rate(http_requests_total{service="payment"}[5m]))

SLO: 99.9% success over 30-day window
  Error budget: 0.1% = 43.2 minutes of errors in 30 days
  Fast burn alert: 2% of budget in 1 hour
  Slow burn alert: 5% of budget in 6 hours

SLI: p95 Latency
  Definition: 95th percentile of response time
  Source: Prometheus (http_request_duration_seconds)
  Query: histogram_quantile(0.95, rate(http_request_duration_seconds_bucket{service="payment"}[5m]))

SLO: p95 < 500ms over 30-day window
  Error budget: 0.1% of requests may exceed 500ms

Review: Quarterly
Owner: Team Payments

Error Budget Policy Enforcement

=== Error Budget Policy ===

Budget remaining | Action
-----------------|-------
75-100%          | Normal development; feature work prioritized
50-75%           | Normal development; monitor trends
25-50%           | Deploy freeze for risky changes
< 25%            | Freeze except critical fixes
< 10%            | All hands on reliability; halt feature work

Exceptions:
- Security hotfixes: always allowed
- Fixes that restore budget: always allowed
- Disabled feature flags: allowed (don't affect budget)

How do we define SLIs that matter to users?

A good SLI measures what users experience, not what the infrastructure does. “CPU usage” isn’t an SLI — users don’t care about CPU. “Request success rate” is an SLI — users notice when their requests fail. To define SLIs: identify the user journey (e.g., “user makes a payment”). For each step in the journey, define what “successful” means (e.g., “transaction completed in < 500ms”). Measure that. Avoid infrastructure SLIs (CPU, memory, disk) unless they’re directly correlated with user experience. Use the RED model (Rate, Errors, Duration) for services and USE (Utilization, Saturation, Errors) for infrastructure.

How do we adjust SLOs that are too aggressive?

If an SLO is too aggressive (the team consistently burns the error budget): don’t lower the SLO immediately. First, investigate why the SLO isn’t met: are there recurring bugs? unstable dependencies? technical debt? If the cause is systemic: create a reliability improvement plan before adjusting the SLO. If after 1-2 quarters of improvement the SLO still isn’t met: adjust the SLO to an achievable level. Document the change and the reason. Communicate to stakeholders that the SLO is adjusted to be realistic. Never adjust an SLO after a single bad month — it may be an anomaly. Use 2-3 months of data for SLO adjustment decisions.

How do we communicate SLOs to non-technical stakeholders?

Translate SLOs to business language: “99.9% availability means the service can be down 43 minutes per month.” Instead of “p95 < 500ms”, use “95% of requests complete in under half a second.” Connect SLOs to business impact: “every minute of payment service downtime costs $10,000 in lost transactions.” Use visual dashboards with traffic lights (green/yellow/red) instead of numbers. Report SLOs monthly to leadership. Include remaining error budget and actions taken when it’s exhausted. Don’t use jargon (SLI, SLO, burn rate) with non-technical stakeholders — translate to impact.

What tools do we need to implement SLOs?

To implement SLOs you need: a monitoring system (Prometheus, Datadog) to collect metrics. A dashboard system (Grafana) to visualize SLOs and error budgets. An alerting system (Alertmanager, PagerDuty) for burn rate-based alerts. Optionally: an SLO tool (Sloth, Prometheus Operator, Nobl9) to automate error budget calculation and alerting. For services without traditional metrics (batch, event-driven): use tools like OpenTelemetry to generate custom metrics. Start simple — Prometheus + Grafana is sufficient for Level 2. Add specialized tools when you reach Level 3-4.

Common Production Pitfalls

• Leaving required fields blank or using vague one-word answers.
• Filling the document once and never updating it after scope or decisions change.
• Storing the document where the team doesn’t look during incidents or reviews.
• Not assigning an owner, due date, or review cadence.
• Copying boilerplate without removing sections that don’t apply.
• Skipping version control, which makes rollback and accountability impossible.
• Failing to link the document to related decisions or follow-up actions.
• Avoiding quarterly reviews that would retire stale or unused sections.