Platform Engineering — Building Internal Developer Platforms

Overview

Platform engineering is the discipline of building and maintaining Internal Developer Platforms (IDPs): self-service layers that abstract infrastructure complexity and allow developers to deploy, operate, and observe their applications without deep platform expertise. Instead of every team reinventing CI/CD, observability, and security patterns, a platform team curates “golden paths” — paved roads with guardrails that make the right thing the easy thing. The goal isn’t to restrict developers but to accelerate them by removing cognitive load.

A well-built IDP treats the platform as a product. The platform team has a roadmap, collects feedback from internal developers, and iterates. When this works, a developer can scaffold a new service, wire up CI/CD, provision a database, and ship to production in under an hour — without filing a single ticket.

When to Use

You should consider platform engineering when:

• You’ve got 10+ engineering teams with duplicated infrastructure effort
• Developer onboarding takes days because environments are hand-crafted
• Teams spend more time on YAML than on business logic
• Security and compliance requirements are inconsistently applied
• You want to scale Kubernetes adoption without every team becoming a cluster admin

You probably don’t need an IDP yet if you’ve got fewer than 5 teams, a single deployment target, or a monolithic architecture with no plans to split. In those cases, a shared CI/CD pipeline and a good README will get you further than a full platform team.

The tipping point is when the cost of duplicated effort exceeds the cost of running a platform team. If three teams are each spending two days a week on infrastructure setup, a platform team that eliminates that duplication pays for itself in the first quarter.

For infrastructure-as-code alternatives, see Complete Guide to Terraform Modules.

IDP Core Components

Component	Purpose	Example tools
Developer portal	Service catalog, documentation, scaffolds	Backstage, Port, Cortex
Self-service infrastructure	On-demand environments, databases	Crossplane, Terraform Cloud, Pulumi
Golden path CI/CD	Standardized deployment pipelines	ArgoCD, GitHub Actions, Tekton
Observability stack	Metrics, logs, traces per service	Prometheus, Grafana, Tempo, Loki
Security guardrails	Policy-as-code, secret management	OPA, Kyverno, Vault

The Golden Path

A golden path is a well-supported, documented, and templated workflow for a common task:

┌─────────────────────────────────────────────────────────┐
│  Developer wants: "Deploy a new REST API"                │
│                                                         │
│  → Backstage scaffold: API template                      │
│  → Auto-generated: CI/CD pipeline, monitoring, TLS       │
│  → ArgoCD deploys to staging with policy checks          │
│  → PR to main → canary to production                     │
│  → Grafana dashboard and alerts auto-provisioned         │
└─────────────────────────────────────────────────────────┘

The developer doesn’t choose the ingress controller, the log format, or the metric naming convention. The platform made those decisions — and enforces them.

How It Works

An IDP sits between the developer and the raw infrastructure. The platform team builds abstractions (templates, CRDs, pipelines) that hide cloud-specific complexity. Developers interact with the platform through a portal (Backstage), a CLI, or GitOps — never directly with AWS, GCP, or Azure consoles.

The architecture has three layers:

Layer	Who builds it	Who uses it	Example
Portal	Platform team	Developers, managers	Backstage service catalog, scaffolder
Platform API	Platform team	CI/CD, GitOps	Crossplane CRDs, Terraform modules, Helm charts
Infrastructure	Cloud provider	Platform team (not developers)	RDS, EKS, S3, VPC

The key insight: developers never touch the infrastructure layer. They declare what they need (a database, a queue, a cache) through the platform API, and the platform team’s automation provisions it. This separation is what makes self-service possible at scale.

GitOps is the connective tissue between the portal and the infrastructure. Backstage templates generate a repo with a catalog-info.yaml and a Helm chart. ArgoCD watches the repo and syncs the Helm chart to the cluster. When a developer needs a database, they don’t run aws rds create-db-instance — they commit a DatabaseClaim manifest and Crossplane handles the rest. The entire workflow is auditable through Git history.

For observability of the platform itself — metrics, logs, traces for the IDP components — see the Observability Guide. For SRE practices that the platform team should adopt, see the SRE Practices Guide.

Backstage Configuration Example

Backstage is the most widely adopted open-source developer portal — Spotify built it, then donated it to the CNCF. It gives you a service catalog, a scaffolder for golden-path templates, TechDocs for documentation, and a plugin ecosystem.

Here is a minimal app-config.yaml that registers an external catalog and a scaffolder template:

# app-config.yaml
app:
  title: Internal Developer Portal
  baseUrl: http://localhost:3000

backend:
  baseUrl: http://localhost:7007
  listen:
    port: 7007

catalog:
  rules:
    - allow: [Component, System, API, Resource, Location]
  locations:
    - type: url
      target: https://github.com/acme/catalog-info.yaml

scaffolder:
  # Template locations
  locations:
    - type: url
      target: https://github.com/acme/backstage-templates/nodejs-api/template.yaml

techdocs:
  builder: 'local'
  generator:
    runIn: 'docker'

The catalog section tells Backstage where to find service metadata. Each service repo contains a catalog-info.yaml that declares the component type, owner, and dependencies. The scaffolder section points to template repos — these are the golden paths. When a developer clicks “Create component” in Backstage, the scaffolder runs the template, which can generate a repo, a CI/CD pipeline, Helm charts, and monitoring dashboards.

For CI/CD security practices that the platform team should enforce on these pipelines, see the CI/CD Security Guide.

Crossplane for Self-Service Infrastructure

Crossplane is a Kubernetes-native control plane that lets the platform team define custom resources (CRDs) representing infrastructure. Developers request resources through Kubernetes manifests — no AWS console, no Terraform CLI, no tickets.

apiVersion: platform.example.com/v1alpha1
kind: DatabaseClaim
metadata:
  name: payment-db
  namespace: payments
spec:
  engine: postgres
  version: "15"
  size: small
  backupRetentionDays: 7

The platform team defines this CRD. Crossplane composites it into RDS instances, VPC security groups, and backup policies. The developer requests a database without knowing AWS exists.

The power here is that the platform team controls the composition. If the company migrates from RDS to Aurora, or from AWS to GCP, the developer’s DatabaseClaim manifest doesn’t change — only the Crossplane composition behind it does. This is what makes the abstraction real rather than cosmetic.

Comparing IDP Portals: Backstage vs Port vs Cortex

Feature	Backstage	Port	Cortex
License	Open source (CNCF)	SaaS	SaaS
Service catalog	Yes (built-in)	Yes	Yes
Scaffolder	Yes (built-in)	Yes (no-code builder)	No
Plugin ecosystem	200+ plugins	API-first	API-first
Self-host vs managed	Self-host	Managed	Managed
Learning curve	Steep (React, YAML)	Low (no-code UI)	Low
Best for	Teams with platform engineers	Teams that want fast setup	Teams focused on scorecards

Backstage gives you the most flexibility and the largest ecosystem, but you need platform engineers to maintain it. Port and Cortex are managed SaaS — faster to adopt, but you pay per seat and have less control over the scaffolder. Most teams start with Backstage because it’s free and the CNCF community behind Backstage is active. If you don’t have the bandwidth to maintain a React app, Port is the pragmatic choice.

Measuring Platform Success

A platform team that doesn’t measure adoption is flying blind. These are the metrics that matter:

Metric	How to measure	Target
Time to provision environment	Backstage analytics	< 10 minutes
Developer onboarding time	HR + platform surveys	< 1 day
Deployment frequency	DORA metrics	2+ per developer per day
Platform NPS	Quarterly developer survey	> 50
Ticket volume to platform team	ITSM data	Decreasing trend

Track these in a Grafana dashboard. The platform team should review them monthly and report to engineering leadership quarterly. If ticket volume isn’t dropping after 6 months, the self-service layer isn’t working — developers are still asking the platform team for manual help.

A common mistake is tracking only platform uptime. A platform that’s 99.99% available but takes 2 weeks to provision a database is a failed platform. The right metrics measure developer productivity, not infrastructure health.

Platform Team Anti-Patterns

Anti-pattern	Fix
Ticket ops platform team	Build self-service APIs, not ticket queues
One-size-fits-all mandates	Golden paths should be defaults, not requirements. Allow escape hatches.
Platform without users	Treat internal teams as customers. Do user research.
Over-abstracting	If the platform is harder than the underlying tool, it’s failed.
No product management	Platform teams need roadmaps, OKRs, and feedback loops like any product team.

Common Mistakes

• Building before understanding pain — interview teams before writing any platform code. The most common failure is a platform built for a problem nobody has.
• Rebuilding the platform every year — Backstage plugins go stale, Crossplane compositions drift, golden path templates rot. Budget 20-30% of platform team capacity for maintenance, not just new features.
• Locking down instead of paving paths — a platform that forces every team through the same pipeline without escape hatches creates shadow platforms. Teams will route around you.
• No documentation — a platform without docs is a black box that generates tickets. Every golden path needs a README, a troubleshooting section, and at least one working example.
• Treating platform as cost center — measure developer productivity gains, not just platform uptime. If you can’t show a before/after metric, you can’t justify the team.
• Ignoring the long tail — the 80% case is easy; the platform must handle the 20% without breaking. A team with a Rust monorepo or an ML training pipeline shouldn’t have to leave the platform entirely.
• No migration path — existing services need a path onto the platform, not just greenfield templates. Migrating 200 legacy services is the hard part; scaffolding a new one is easy.

Troubleshooting

• Backstage scaffolder template fails silently: check the scaffolder task logs in Backstage backend (/api/scaffolder/tasks). Most failures are missing template parameters or a misconfigured catalog location in app-config.yaml.
• Crossplane composition stuck in SyncError: run kubectl describe composition to see which managed resource failed. The most common cause is an IAM role without permission to create the underlying cloud resource.
• Golden path pipeline generates broken Helm chart: validate generated templates with helm template and helm lint in CI before merging template changes. A bad template pushes broken charts to every service that uses it.
• ArgoCD sync loop on new service: check that the generated catalog-info.yaml doesn’t conflict with an existing component name. Duplicate catalog entities cause ArgoCD to flip between desired states.
• Developer complains the platform is slower than manual: measure end-to-end time from scaffold to first deploy. If it’s over an hour, the problem is usually a manual approval step hiding in the pipeline — remove it or automate it with policy-as-code.

Quick Reference

• Scaffold a service: npx @backstage/cli create-app for the portal; in production, developers use the Backstage UI scaffolder.
• Register a component: add a catalog-info.yaml to the repo root with kind: Component, then register the URL in Backstage.
• Provision infrastructure: kubectl apply -f database-claim.yaml — Crossplane composes the claim into cloud resources.
• Check golden path health: argocd app list for deploy status; Backstage catalog for service health.
• Rollback: argocd app rollback <app> or revert the Git commit that triggered the sync.

Further Reading

• Backstage documentation — portal setup, plugins, and scaffolder authoring.
• Crossplane documentation — compositions, managed resources, and provider configuration.
• OPA policy language — Rego reference for writing policy-as-code guardrails.
• DORA State of DevOps reports — benchmark data for deployment frequency and lead time.
• Team Topologies — the book that defined the platform-as-product operating model.

Production Notes

• Version your golden paths — a breaking change to a scaffolder template affects every service that uses it. Tag template releases and test them against a staging catalog before rolling out.
• Track Crossplane drift — a kubectl get managed showing resources out of sync usually means someone changed cloud resources manually. Enforce GitOps-only changes with IAM policies.
• Alert on platform SLOs, not just service SLOs — if the scaffolder or the catalog goes down, every developer is blocked. Monitor Backstage uptime and Crossplane reconciliation latency like any production service.
• Rotate secrets used by templates — scaffolder templates often embed tokens for repo creation and CI/CD provisioning. Store them in Vault or External Secrets, never in the template repo.

Key Takeaways

• Golden paths beat mandates — make the right thing easy instead of making the wrong thing impossible.
• Platform teams are product teams — they need a PM, a roadmap, user research, and adoption metrics.
• Self-service is the only scaling strategy — every manual step in a golden path is a bottleneck waiting to happen.
• Measure adoption, not uptime — a platform nobody uses is worse than no platform at all.

Advanced Topics

Scenario: Internal Platform for 50 Teams

System: Company with 50 product teams, 200 services
Problem: Each team builds their own CI/CD, monitoring, auth
Solution: Platform team provides golden paths and shared tooling

Model: Platform as a Product (PaaP)
  | Component | Tool | Consumers |
  |-----------|------|-----------|
  | CI/CD | GitHub Actions + templates | 50 teams |
  | Deploy | ArgoCD + Helm charts | 50 teams |
  | Monitoring | Prometheus + Grafana | 50 teams |
  | Logging | Loki + Promtail | 50 teams |
  | Tracing | Jaeger + OpenTelemetry | 50 teams |
  | Auth | OAuth2 + SPIFFE | 50 teams |
  | Secrets | External Secrets Operator | 50 teams |
  | Service catalog | Backstage | 50 teams |

Golden paths (opinionated templates):
  1. New microservice:
     - Template: npm create @platform/microservice
     - Generates: Dockerfile, Helm chart, CI/CD pipeline,
       monitoring dashboards, alert rules, service catalog entry
     - Time: from 2 days to 15 minutes

  2. New API endpoint:
     - Template generates: OpenAPI spec, handler, tests,
       documentation, client SDK
     - Automatic validation: linter, schema, tests

  3. New team onboarding:
     - Backstage plugin: provisions repos, namespaces,
       dashboards, permissions
     - Time: from 1 week to 1 hour

Platform metrics (internal SLOs):
  | SLO | Target | Metric |
  |-----|--------|--------|
  | Build time | < 5 min | p50 pipeline duration |
  | Deploy availability | 99.9% | ArgoCD uptime |
  | Golden path adoption | > 80% | % services with template |
  | Onboarding time | < 1 day | Provision hours |
  | Team satisfaction | > 4/5 | Quarterly survey |

Organization:
  Platform team (8 people):
    - 3 platform engineers (CI/CD, deploy)
    - 2 observability engineers (monitoring, tracing)
    - 2 developer experience (Backstage, templates)
    - 1 product manager (prioritizes roadmap)

  Engagement model:
    - Weekly office hours (consulting)
    - Slack channel #platform-help
    - Quarterly roadmap (feedback -> priorities)
    - Open RFCs for major changes

Lessons:
  - Treat the platform as a product, not infrastructure
  - Measure adoption and satisfaction, not just uptime
  - Golden paths reduce onboarding time dramatically
  - The platform team needs a PM to prioritize
  - Documentation and examples > 1:1 support

Scaling Self-Service Beyond 50 Teams

Once the platform serves 50+ teams, the bottleneck shifts from provisioning to support. Templates generate everything automatically, Backstage gives you the catalog and scaffolding, and documentation is detailed. Office hours are for consulting, not tickets. If teams wait on the platform for something, automate it. Measure wait time and reduce it.

Common Production Pitfalls

• Treating the platform as infrastructure, not product — the platform needs a PM, a roadmap, and adoption metrics like any product.
• Rolling out to all teams at once — start with 2-3 willing teams, fix their feedback, then expand.
• Letting golden paths go stale — Backstage plugins and Crossplane compositions need maintenance. Budget 20-30% of capacity for it.
• Ignoring the 20% long tail — teams with unusual needs (ML training, monorepos, legacy systems) will route around the platform if it can’t accommodate them.
• No escape hatches — forcing every team through the same pipeline without alternatives creates shadow platforms.
• Measuring uptime instead of adoption — a platform nobody uses is a cost center, not an accelerator.