Complete Guide to GitOps in Production
Set up GitOps with ArgoCD or Flux in production: pull-based reconciliation, multi-cluster management, progressive delivery, SOPS secrets and CI/CD integration.
Introduction
Someone hotfixes a production Deployment with kubectl edit at 2 AM, the change is never written down anywhere, and the next scheduled deploy silently wipes it out. GitOps exists to kill that entire class of problems: Git becomes the only place where desired state lives, and an agent inside the cluster pulls and applies it — nobody touches the cluster directly.
This guide walks through a production GitOps setup end to end: installing and configuring ArgoCD or Flux, structuring the GitOps repository, managing secrets with SOPS, progressive delivery with canary rollouts, automated rollback, and wiring it all into your CI pipeline. Everything below is working YAML you can adapt — for ArgoCD-specific depth, see the Complete Guide to GitOps with ArgoCD.
GitOps Principles
GitOps, as defined by the OpenGitOps working group, rests on four properties. Everything else in this guide follows from them:
GitOps Principles:
1. Declarative — System state is described declaratively in Git
2. Versioned — Git stores the complete history of desired state
3. Pulled — Agents pull changes from Git and apply them (not push)
4. Continuously reconciled — Agents continuously compare actual vs desired state
Benefits:
- Audit trail: Every change is a Git commit with author and timestamp
- Rollback: Revert to any previous state with git revert
- Drift detection: Agents detect and fix manual changes
- Security: No direct cluster access needed for deployments
- Collaboration: PRs enable code review for infrastructure changes
The pull model is the part teams underestimate at first. A push-based pipeline needs cluster credentials in CI — one CI compromise means cluster compromise. With pull, credentials never leave the cluster; the agent only needs read access to a Git repo.
When to Use GitOps
GitOps pays off when:
- You run multiple clusters or environments. Templating the same app across us-east-1, eu-west-1 and staging with one ApplicationSet beats scripting
kubectlloops. - Audit and compliance matter. Every production change becomes a reviewed PR with an author and a timestamp — that’s your audit log.
- You need reliable rollback.
git revertplus auto-sync is a rollback mechanism you don’t have to build or test separately. - Manual changes keep causing incidents.
selfHealturns “who changed this?” into a non-event: the agent reverts it and reports it. - Teams deploy independently. Each team owns its manifests via PRs without holding cluster credentials.
When GitOps does not fit: single-developer hobby clusters where the agent overhead exceeds the value, and highly imperative workflows (databases with manual failover steps, one-off jobs) that don’t map to declarative desired state. Forcing those into GitOps creates more problems than it solves.
How GitOps Works
The pipeline splits cleanly in two: CI owns the application repo and produces immutable artifacts; the GitOps agent owns cluster state and reads a separate config repo.
The reconciliation loop is the key mechanism: the agent fetches the desired state from the GitOps repo on an interval (30s–5m depending on configuration), diffs it against live cluster state, and applies the difference. A manual kubectl edit shows up as drift on the next reconcile — with self-heal enabled it’s reverted automatically, with it disabled it’s reported as OutOfSync.
Keeping two repos — one for application source, one for deployment manifests — is deliberate. App code churns on every feature; cluster config should only change on intentional releases. It also lets CI write to the config repo without giving the cluster agent write access to anything.
ArgoCD Setup
ArgoCD is a controller that runs inside the cluster and exposes a web UI, an API and a CLI. It’s the better fit when several teams need visibility into sync state, or when you want one control plane managing many clusters.
ArgoCD Installation
# Install ArgoCD
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
# Get admin password
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d
# Port forward to access UI
kubectl port-forward svc/argocd-server -n argocd 8080:443
# Install ArgoCD CLI
brew install argocd # macOS
# or
curl -sSL -o argocd https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64
chmod +x argocd && mv argocd /usr/local/bin/
The stable manifest URL always points at the latest release — pin to a versioned tag (v3.x.y/manifests/install.yaml) in production so upgrades are deliberate, not a surprise side-effect of a reinstall.
Application Manifest
An Application is ArgoCD’s unit of work: one repo path, one destination, one sync policy.
# ArgoCD Application for API service
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: api-service
namespace: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io # Cascade deletion
spec:
project: production
source:
repoURL: https://github.com/myorg/k8s-manifests
targetRevision: main
path: production/api-service
destination:
server: https://kubernetes.default.svc
namespace: production
syncPolicy:
automated:
prune: true # Delete resources removed from Git
selfHeal: true # Revert manual changes
allowEmpty: false # Prevent deleting all resources
syncOptions:
- CreateNamespace=true
- PruneLast=true
- ApplyOutOfSyncOnly=true
retry:
limit: 5
backoff:
duration: 5s
factor: 2
maxDuration: 3m
revisionHistoryLimit: 10 # Keep last 10 sync history
Three fields do the heavy lifting:
prune: truedeletes live resources that no longer exist in Git — without it, deleted manifests linger forever.selfHeal: trueis the actual drift reconciliation; without it ArgoCD reportsOutOfSyncbut leaves the manual change in place.allowEmpty: falseis your seatbelt: if the repo path is accidentally emptied (bad path rename, empty commit), ArgoCD refuses to wipe the namespace instead of pruning everything.
revisionHistoryLimit: 10 matters for rollback — it bounds how far back argocd app rollback can reach without touching Git.
AppProject for Multi-Team
AppProject is the RBAC boundary: which repos a team may deploy from, which namespaces they may target, and which cluster-level objects they may touch.
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: production
namespace: argocd
spec:
description: Production environment applications
# Allowed source repositories
sourceRepos:
- https://github.com/myorg/k8s-manifests
- https://github.com/myorg/api-config
# Allowed destination namespaces
destinations:
- server: https://kubernetes.default.svc
namespace: production
- server: https://kubernetes.default.svc
namespace: production-*
# Allowed cluster resources
clusterResourceWhitelist:
- group: ''
kind: Namespace
- group: rbac.authorization.k8s.io
kind: RoleBinding
# Denied resources
clusterResourceBlacklist:
- group: ''
kind: Node
- group: ''
kind: PersistentVolume
# Roles for team access
roles:
- name: platform-team
description: Platform team access
policies:
- p, proj:production:platform-team, applications, get, production/*, allow
- p, proj:production:platform-team, applications, sync, production/*, allow
groups:
- platform-team
The blacklist/whitelist split is the important part: platform teams can manage their own namespaces and RBAC, but can’t create Nodes or PersistentVolumes — the blast radius of a bad Application stays inside the project.
ApplicationSet for Multi-Cluster
ApplicationSet generates one Application per target so “deploy api-service to every prod cluster” is a single manifest instead of N copies.
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: api-all-clusters
namespace: argocd
spec:
generators:
- list:
elements:
- cluster: us-east-1
url: https://1.2.3.4
- cluster: eu-west-1
url: https://5.6.7.8
- cluster: ap-southeast-1
url: https://9.10.11.12
template:
metadata:
name: 'api-{{cluster}}'
spec:
project: production
source:
repoURL: https://github.com/myorg/k8s-manifests
targetRevision: main
path: production/api-service
destination:
server: '{{url}}'
namespace: production
syncPolicy:
automated:
prune: true
selfHeal: true
The list generator is the simplest; cluster and git generators cover dynamic cases (one Application per registered cluster secret, or per directory in the repo). For per-cluster value overrides, pair ApplicationSet with Kustomize overlays rather than forking the manifest tree.
Flux Setup
Flux is a set of composable controllers (source, kustomize, helm, notification) with no UI. It fits teams that live in the CLI, want the smallest possible footprint, or already standardize on Helm and Kustomize.
Flux Installation
# Install Flux CLI
brew install fluxcd/tap/flux # macOS
# Bootstrap Flux with GitHub
flux bootstrap github \
--owner=myorg \
--repository=fleet-infra \
--branch=main \
--path=clusters/production \
--personal=false
# Check Flux installation
flux check
flux get kustomizations
flux get helmreleases
flux bootstrap is itself GitOps: it commits the Flux manifests into your repo, so Flux upgrades are PRs like everything else. The command is idempotent — run it again to upgrade Flux in place.
Flux Kustomization
A Kustomization ties a repo path to a reconcile loop — the Flux equivalent of ArgoCD’s Application.
# clusters/production/api-service.yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: api-service
namespace: flux-system
spec:
interval: 1m # Reconcile every minute
path: ./production/api-service
sourceRef:
kind: GitRepository
name: k8s-manifests
prune: true # Delete resources removed from Git
wait: true # Wait for resources to be ready
timeout: 5m
targetNamespace: production
# Health checks
healthChecks:
- apiVersion: apps/v1
kind: Deployment
name: api
namespace: production
- apiVersion: v1
kind: Service
name: api-service
namespace: production
# Post-build variable substitution
postBuild:
substitute:
cluster_name: production
cluster_region: us-east-1
wait: true plus healthChecks changes the failure mode usefully: a sync isn’t “applied” until the Deployment actually reports ready, so a bad image surfaces as a failed Kustomization instead of silently degraded pods. postBuild.substitute injects per-cluster values without duplicating manifests — combine it with ${cluster_name} placeholders in the source YAML.
Flux GitRepository
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
name: k8s-manifests
namespace: flux-system
spec:
interval: 30s
url: https://github.com/myorg/k8s-manifests
ref:
branch: main
secretRef:
name: github-deploy-key # For private repos
Flux HelmRelease
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: redis
namespace: production
spec:
interval: 5m
chart:
spec:
chart: redis
version: "18.x"
sourceRef:
kind: HelmRepository
name: bitnami
namespace: flux-system
values:
architecture: replication
auth:
enabled: true
existingSecret: redis-secret
existingSecretPasswordKey: redis-password
replica:
replicaCount: 3
persistence:
enabled: true
size: 10Gi
metrics:
enabled: true
serviceMonitor:
enabled: true
Pin the chart to a major-version range (18.x) rather than a bare * — you still get patch releases through Git history, but a breaking major bump can’t roll out unreviewed.
ArgoCD or Flux
| ArgoCD | Flux | |
|---|---|---|
| UI | Full web UI + API + CLI | CLI only (UIs are third-party) |
| Multi-cluster | Built-in, single control plane | Per-cluster install (or via Flux.sharding) |
| App model | Application / ApplicationSet CRDs | Kustomization / HelmRelease / GitRepository CRDs |
| Progressive delivery | Argo Rollouts | Flagger |
| Footprint | Heavier (API server, UI, repo server) | Minimal controllers |
| RBAC | Project-level, fine-grained | Kubernetes RBAC only |
Pick ArgoCD when several teams need shared visibility and per-app RBAC, or when you manage many clusters from one pane. Pick Flux for CLI-first teams, tight Helm integration, or minimal control-plane overhead. Many organizations run both — Flux for platform infrastructure, ArgoCD for app teams that want the UI.
GitOps Repository Layout
How you lay out the config repo determines how painful multi-environment work gets. A structure that scales:
k8s-manifests/
├── base/ # Shared manifests (kustomize base)
│ └── api-service/
│ ├── deployment.yaml
│ └── service.yaml
├── production/ # Environment overlays
│ ├── api-service/
│ │ └── kustomization.yaml # image tag, replicas, ingress host
│ └── secrets/
│ └── api-secrets.yaml # SOPS-encrypted
├── staging/
│ └── api-service/
│ └── kustomization.yaml
└── clusters/ # Flux bootstrap / per-cluster wiring
└── production/
└── api-service.yaml
Two rules hold up in practice:
- One GitOps repo per environment (or cluster), not per app. A single source of truth only works if “the truth” is in one place; per-app repos scatter it.
- Overlays over branches. A
stagingbranch that merges tomainfor promotion sounds appealing and produces constant drift between branches. Directory-per-environment with Kustomize overlays keeps every environment visible at once.
Secret Management with SOPS
Plaintext secrets in Git are a hard no — Git history is forever. The GitOps-compatible options:
- SOPS + age/KMS: encrypt secret files client-side, commit them, let the agent decrypt in-cluster. Works with Flux natively and ArgoCD via plugins.
- External Secrets Operator: keep secrets in AWS Secrets Manager, Vault or Key Vault; ESO syncs them into Kubernetes Secrets. Best when a secrets manager already exists.
- Sealed Secrets: encrypt against a cluster-specific public key; the sealed file is safe to commit. Simple, but rotation is awkward.
SOPS is the lightest starting point:
# Install SOPS and age
brew install sops age
# Generate age key pair
age-keygen -o sops-key.txt
# Public key: age1xxxxx...
# Private key: AGE-SECRET-KEY-1xxxxx...
# Encrypt a secret file
sops --encrypt --age age1xxxxx... --in-place secret.yaml
# Decrypt a secret file
sops --decrypt secret.yaml
# Edit an encrypted file in-place
sops secret.yaml
# Rotate keys
sops updatekeys secret.yaml
# Encrypted secret (SOPS)
apiVersion: v1
kind: Secret
metadata:
name: api-secrets
namespace: production
type: Opaque
stringData:
DB_PASSWORD: ENC[AES256_GCM,data:abc123,iv:xxx,tag:yyy,type:str]
API_KEY: ENC[AES256_GCM,data:def456,iv:xxx,tag:yyy,type:str]
sops:
age:
- recipient: age1xxxxx...
enc: |
-----BEGIN AGE ENCRYPTED FILE-----
...
-----END AGE ENCRYPTED FILE-----
lastmodified: "2026-07-04T12:00:00Z"
mac: ENC[AES256_GCM,data:mac,iv:xxx,tag:yyy,type:str]
# Flux Kustomization with SOPS decryption
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: api-secrets
namespace: flux-system
spec:
interval: 1m
path: ./production/secrets
sourceRef:
kind: GitRepository
name: k8s-manifests
prune: true
decryption:
provider: sops
secretRef:
name: sops-age-key # Secret containing the private key
The private age key lives as a cluster Secret (sops-age-key) — never in Git. Rotating it means sops updatekeys on every encrypted file plus a new cluster Secret; script that before you need it at 3 AM. If you need centralized secret storage beyond Kubernetes, the External Configuration Store Pattern covers that side.
Progressive Delivery
Progressive delivery caps the blast radius of a bad release: route a fraction of traffic to the new version, watch error rate and latency, and either promote or roll back automatically based on metrics.
Argo Rollouts
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: api
namespace: production
spec:
replicas: 10
strategy:
canary:
canaryService: api-canary
stableService: api-stable
trafficRouting:
nginx:
stableIngress: api-stable-ingress
steps:
- setWeight: 10
- pause: { duration: 2m }
- setWeight: 25
- pause: { duration: 2m }
- setWeight: 50
- pause: { duration: 5m }
- setWeight: 75
- pause: { duration: 5m }
analysis:
templates:
- templateName: success-rate
startingStep: 2 # Start analysis after 25% canary
selector:
matchLabels:
app: api
template:
metadata:
labels:
app: api
spec:
containers:
- name: api
# CI sets the immutable tag on every release — never "latest"
image: ghcr.io/myorg/api:v1.4.2
ports:
- containerPort: 8000
readinessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 5
periodSeconds: 10
# Analysis template for canary
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
name: success-rate
namespace: production
spec:
metrics:
- name: success-rate
interval: 1m
successCondition: result[0] >= 0.99
failureLimit: 3 # Abort after 3 failures
provider:
prometheus:
address: http://prometheus.monitoring:9090
query: |
sum(rate(http_requests_total{status!~"5.."}[2m]))
/
sum(rate(http_requests_total[2m]))
The AnalysisTemplate is what makes this a quality gate instead of a fancy load balancer: Prometheus is queried every minute, and three failed checks abort the rollout. Tune successCondition to a signal you actually alert on — a 99% success threshold with a 1m window catches obvious breakage fast; subtler regressions need longer steps and a p95 latency metric alongside.
Flux Flagger
apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
name: api
namespace: production
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: api
service:
port: 80
targetPort: 8000
gateways:
- public-gateway.istio-system.svc.cluster.local
hosts:
- api.stackpractices.com
analysis:
interval: 1m
threshold: 5
maxWeight: 50
stepWeight: 10
metrics:
- name: request-success-rate
thresholdRange:
min: 99
interval: 1m
- name: request-duration
thresholdRange:
max: 500
interval: 30s
webhooks:
- name: load-test
type: rollout
url: http://flagger-loadtester.test/
timeout: 5s
metadata:
cmd: "hey -z 1m -q 10 -c 2 http://api.production:80/"
Flagger shifts stepWeight (10%) each interval (1m) until maxWeight, checking success rate and p99 duration each step. The load-test webhook matters more than it looks: without baseline traffic, a canary that gets 2 requests a minute produces metrics noise, not signal.
Automated Rollback
With GitOps, rollback is a Git operation — that’s the point. Two routes, and you should know when each applies:
git revert+ push (preferred). The agent syncs the cluster to the reverted state. The rollback itself is auditable, reviewable history — same workflow as any change.- ArgoCD point-in-time rollback (emergency).
argocd app rollbackpins the app to a previous in-cluster revision without a Git commit. Faster, but it bypasses the audit trail — follow up with agit revertso Git and cluster converge again.
# ArgoCD Application — history settings that enable rollback
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: api-service
namespace: argocd
spec:
revisionHistoryLimit: 10 # argocd app rollback can reach back this many revisions
syncPolicy:
automated:
prune: true
selfHeal: true
# ArgoCD rollback — uses the history ID, not a Git SHA
argocd app history api-service # find the revision ID to roll back to
argocd app rollback api-service <id> # pin to that revision
# Flux: rollback by reverting the Git commit
git revert <commit-sha>
git push origin main
# Flux automatically syncs to the reverted state on the next reconcile
Watch out for the rollback trap with selfHeal: if you roll back by pinning a revision while selfHeal is on, ArgoCD sees the pinned state as drift versus Git and fights you. Disable auto-sync first (argocd app set api-service --sync-policy none), or do the git revert and let normal sync handle it.
CI/CD Integration
CI’s job in GitOps is to build the artifact and update the GitOps repo — never to touch the cluster.
# GitHub Actions: Build and update GitOps repo
name: Deploy
on:
push:
branches: [main]
jobs:
build-and-deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build and push image
run: |
docker build -t ghcr.io/myorg/api:${{ github.sha }} .
echo ${{ secrets.GITHUB_TOKEN }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin
docker push ghcr.io/myorg/api:${{ github.sha }}
- name: Update GitOps repo
run: |
git clone https://github.com/myorg/k8s-manifests.git
cd k8s-manifests
# Update image tag in manifest
sed -i "s|ghcr.io/myorg/api:.*|ghcr.io/myorg/api:${{ github.sha }}|" production/api-service/deployment.yaml
git config user.name "CI Bot"
git config user.email "ci@stackpractices.com"
git add .
git commit -m "Deploy api:${{ github.sha }}"
git push
Production-hardened notes on that workflow:
- Tag images with
${{ github.sha }}(or semver), neverlatest. GitOps only works if the desired state in Git is exact —latestis mutable state hiding outside Git. - Prefer
kustomize edit set image api=ghcr.io/myorg/api:${{ github.sha }}oversedonce your manifests are Kustomize-based; it’s syntax-aware and can’t corrupt a manifest. - For many apps, skip hand-rolled
sedentirely and use ArgoCD Image Updater or Flux’sImagePolicy/ImageRepositoryCRDs — the agent writes tag commits back to the repo itself.
Companion code: GitOps production manifests: every YAML file from this guide — ArgoCD, Flux, SOPS, rollouts and the CI workflow.
Best Practices
- Separate app repo from config repo. CI pushes image tags to the config repo; the agent only reads it. Mixing them creates a commit storm in the repo your auditors read.
- Immutable image tags everywhere. A
v1.4.2tag in Git is a fact;latestis a rumor. Rollback, drift detection and promotion all assume exact desired state. - Gate syncs on health, not just apply. Flux
wait: true+healthChecks, ArgoCD sync waves + health assessment — a sync that returns success while pods crash is worse than a visible failure. - Use sync waves for ordered dependencies. Annotate resources with
argocd.argoproj.io/sync-wave(CRDs and RBAC first, apps last) so a fresh environment converges in the right order. - Cap
revisionHistoryLimitdeliberately. 10 is a sane default — enough rollback runway, small etcd footprint. - Rehearse key rotation.
sops updatekeys, new cluster Secret, re-encrypt — if you’ve never done it, the first time will be during an incident.
Common Mistakes
selfHeal: true+allowEmptyunset. One empty-path accident and the agent prunes the namespace. Always setallowEmpty: falseon automated sync policies.- Secrets committed in plaintext “temporarily”. Git remembers. If it ever touched the repo unencrypted, rotate it — the leak already happened.
- Reconcile intervals too aggressive.
interval: 10s× 200 applications = constant API-server and Git-provider load. Start at 1–5m and tighten only where latency matters. - Pruning CRDs or shared namespaces.
prune: trueon an Application that owns cluster-level objects can delete resources other apps depend on. Keep cluster-scoped objects in a separate, non-pruned Application. - Letting CI
kubectl apply“just this once”. Every direct apply is drift the agent will either revert or hide. No exceptions — that’s how the audit trail stays meaningful. - Environment promotion via branch merges. Staging→prod via merged branches produces two diverging histories. Promote by copying the image tag (or overlay change) into the prod path via PR instead.
Troubleshooting
Sync failures that keep coming back in production, and how to read them:
Application stuck OutOfSync after sync runs. Run argocd app diff api-service — the diff usually shows an immutable field (spec.selector on a Deployment can’t change in place) or an admission webhook rejecting the manifest. Fix by deleting the object so the agent recreates it, or adding the Replace=true sync option for that resource.
Cascade delete hangs forever. The resources-finalizer blocks Application deletion while a managed resource is itself stuck (finalizer loop, unreachable namespace). As a last resort, strip the finalizer: kubectl -n argocd patch app api-service --type=json -p='[{"op":"remove","path":"/metadata/finalizers"}]' — you’ll have to clean up the orphaned resources yourself.
Flux Kustomization shows Ready=False but pods are Running. wait: true plus healthChecks verifies the exact objects listed — a renamed Deployment fails the check even when the workload is healthy. kubectl -n flux-system describe kustomization api-service shows which object timed out.
SOPS decryption errors after key rotation. Either the cluster Secret holding the age private key wasn’t updated, or the Kustomization’s secretRef still points at the old key name. Compare kubectl -n flux-system get secret sops-age-key against the recipient line in the encrypted file — they must match.
ApplicationSet generated apps for the wrong clusters. The list or cluster generator matched more than intended. Inspect the generated names with kubectl -n argocd get applications before enabling prune — a misconfigured generator at scale doesn’t just create apps, it deletes them.
See Also
- Complete Guide to GitOps with ArgoCD
- Complete Guide to Kubernetes Networking
- Complete Guide to Terraform in Production
- External Configuration Store Pattern
- Multi-Cloud Strategies — Benefits, Pitfalls
Further Reading
- OpenGitOps specification — the four principles this guide implements
- ArgoCD documentation — Applications, Projects, sync options, Image Updater
- Flux documentation — bootstrap, Kustomization, SOPS decryption, Image Automation
- Argo Rollouts documentation — canary steps, AnalysisTemplates, traffic routing
- Flagger documentation — Canary CRD, metric checks, webhooks
- SOPS — encryption formats, age/KMS backends, key rotation
Frequently Asked Questions
What is the difference between ArgoCD and Flux?
Both implement GitOps for Kubernetes. ArgoCD ships a web UI, multi-cluster from a single control plane, and ApplicationSets for templating. Flux is lighter, is pure CRDs, and has tighter native Helm and Kustomize integration. Choose ArgoCD if teams need a UI and centralized multi-cluster. Choose Flux for CLI-driven workflows and minimal footprint.
How does drift reconciliation work?
The agent continuously compares live cluster state with the desired state in Git. When it detects a difference — a manual kubectl edit, a deleted resource — it re-applies the Git state. With selfHeal: true (ArgoCD) or prune: true + automated reconciliation (Flux), manual changes are reverted automatically; without self-heal, drift is only reported as OutOfSync.
How do I handle secrets in GitOps?
Never commit plaintext secrets. Encrypt secret files client-side with SOPS (age or AWS KMS) and commit the ciphertext — the agent decrypts in-cluster with a private key stored as a cluster Secret. Or keep secrets outside Git entirely with External Secrets Operator (pulls from AWS Secrets Manager, Vault, Key Vault) or Sealed Secrets. Full details in Secret Management with SOPS.
What is progressive delivery in GitOps?
Releasing a new version to a fraction of traffic — 10%, 25%, 50% — while automated metric checks run at each step, instead of updating all pods at once. Argo Rollouts and Flagger implement this: if the success-rate or latency check fails, traffic snaps back to the stable version. It's a quality gate, not just traffic shaping.
Should I use one Git repo or multiple repos for GitOps?
One config repo per cluster or environment, separate repos for application source. A single GitOps repo keeps the complete desired state visible and makes changes atomic; app code lives elsewhere so CI can update image tags without the agent needing write access to app code. See GitOps Repository Layout.
How do I roll back a deployment in GitOps?
Prefer git revert + push — the agent syncs to the reverted state and the rollback is itself audited history. For emergencies, argocd app rollback <app> <id> pins to a previous revision instantly (get the ID from argocd app history), but follow up with a revert so Git catches up. Details and the self-heal gotcha in Automated Rollback.
How does CI update the GitOps repo?
CI builds and pushes the image, clones the config repo, updates the image tag (kustomize edit set image or a templated sed), commits and pushes. The agent picks up the change on its next reconcile. ArgoCD Image Updater or Flux ImagePolicy automate the tag-commit step entirely.
Does GitOps work for databases and stateful workloads?
Partially. Deploying the StatefulSet and its config via GitOps works fine; schema migrations and failover decisions don't map well to declarative sync and usually stay as CI jobs or operators. The rule of thumb: if the operation is "converge to this state", GitOps fits; if it's "run this procedure", keep it out of the sync loop.
Related Resources
Complete Guide to GitOps with ArgoCD
Deploy Kubernetes applications with GitOps using ArgoCD. Covers installation, ApplicationSets, sync strategies, Helm/Kustomize, RBAC, and multi-cluster management.
GuideComplete Guide to Kubernetes Networking
Master Kubernetes networking. Covers Services, Ingress, NetworkPolicies, CNI plugins, DNS, service mesh, load balancing, external traffic, mTLS, and troubleshooting with practical YAML manifests and configuration examples.
GuideComplete Guide to Terraform in Production
Manage infrastructure as code with Terraform in production. Covers modules, state management, workspaces, drift detection, remote backends, variable validation, sentinel policies, and CI/CD integration with practical HCL examples.
GuideComplete Guide to Monitoring and Alerting
Build a production monitoring stack. Covers Prometheus, Grafana, AlertManager, metrics instrumentation, alert rules, runbooks, SLI/SLO/SLA, distributed tracing with Jaeger, log aggregation, and on-call best practices with practical configuration examples.
PatternExternal Configuration Store Pattern
Centralize application configuration outside deployment artifacts to support live updates, environment separation, and audited changes.