StackPractices
advanced By Mathias Paulenko

Complete Guide to GitOps in Production

Set up GitOps with ArgoCD or Flux in production: pull-based reconciliation, multi-cluster management, progressive delivery, SOPS secrets and CI/CD integration.

Introduction

Someone hotfixes a production Deployment with kubectl edit at 2 AM, the change is never written down anywhere, and the next scheduled deploy silently wipes it out. GitOps exists to kill that entire class of problems: Git becomes the only place where desired state lives, and an agent inside the cluster pulls and applies it — nobody touches the cluster directly.

This guide walks through a production GitOps setup end to end: installing and configuring ArgoCD or Flux, structuring the GitOps repository, managing secrets with SOPS, progressive delivery with canary rollouts, automated rollback, and wiring it all into your CI pipeline. Everything below is working YAML you can adapt — for ArgoCD-specific depth, see the Complete Guide to GitOps with ArgoCD.

GitOps Principles

GitOps, as defined by the OpenGitOps working group, rests on four properties. Everything else in this guide follows from them:

GitOps Principles:

1. Declarative — System state is described declaratively in Git
2. Versioned — Git stores the complete history of desired state
3. Pulled — Agents pull changes from Git and apply them (not push)
4. Continuously reconciled — Agents continuously compare actual vs desired state

Benefits:
  - Audit trail: Every change is a Git commit with author and timestamp
  - Rollback: Revert to any previous state with git revert
  - Drift detection: Agents detect and fix manual changes
  - Security: No direct cluster access needed for deployments
  - Collaboration: PRs enable code review for infrastructure changes

The pull model is the part teams underestimate at first. A push-based pipeline needs cluster credentials in CI — one CI compromise means cluster compromise. With pull, credentials never leave the cluster; the agent only needs read access to a Git repo.

When to Use GitOps

GitOps pays off when:

  • You run multiple clusters or environments. Templating the same app across us-east-1, eu-west-1 and staging with one ApplicationSet beats scripting kubectl loops.
  • Audit and compliance matter. Every production change becomes a reviewed PR with an author and a timestamp — that’s your audit log.
  • You need reliable rollback. git revert plus auto-sync is a rollback mechanism you don’t have to build or test separately.
  • Manual changes keep causing incidents. selfHeal turns “who changed this?” into a non-event: the agent reverts it and reports it.
  • Teams deploy independently. Each team owns its manifests via PRs without holding cluster credentials.

When GitOps does not fit: single-developer hobby clusters where the agent overhead exceeds the value, and highly imperative workflows (databases with manual failover steps, one-off jobs) that don’t map to declarative desired state. Forcing those into GitOps creates more problems than it solves.

How GitOps Works

The pipeline splits cleanly in two: CI owns the application repo and produces immutable artifacts; the GitOps agent owns cluster state and reads a separate config repo.

flowchart diagram: App repo<br/>source code

The reconciliation loop is the key mechanism: the agent fetches the desired state from the GitOps repo on an interval (30s–5m depending on configuration), diffs it against live cluster state, and applies the difference. A manual kubectl edit shows up as drift on the next reconcile — with self-heal enabled it’s reverted automatically, with it disabled it’s reported as OutOfSync.

Keeping two repos — one for application source, one for deployment manifests — is deliberate. App code churns on every feature; cluster config should only change on intentional releases. It also lets CI write to the config repo without giving the cluster agent write access to anything.

ArgoCD Setup

ArgoCD is a controller that runs inside the cluster and exposes a web UI, an API and a CLI. It’s the better fit when several teams need visibility into sync state, or when you want one control plane managing many clusters.

ArgoCD Installation

# Install ArgoCD
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# Get admin password
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

# Port forward to access UI
kubectl port-forward svc/argocd-server -n argocd 8080:443

# Install ArgoCD CLI
brew install argocd  # macOS
# or
curl -sSL -o argocd https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64
chmod +x argocd && mv argocd /usr/local/bin/

The stable manifest URL always points at the latest release — pin to a versioned tag (v3.x.y/manifests/install.yaml) in production so upgrades are deliberate, not a surprise side-effect of a reinstall.

Application Manifest

An Application is ArgoCD’s unit of work: one repo path, one destination, one sync policy.

# ArgoCD Application for API service
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: api-service
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io  # Cascade deletion
spec:
  project: production

  source:
    repoURL: https://github.com/myorg/k8s-manifests
    targetRevision: main
    path: production/api-service

  destination:
    server: https://kubernetes.default.svc
    namespace: production

  syncPolicy:
    automated:
      prune: true       # Delete resources removed from Git
      selfHeal: true    # Revert manual changes
      allowEmpty: false # Prevent deleting all resources
    syncOptions:
      - CreateNamespace=true
      - PruneLast=true
      - ApplyOutOfSyncOnly=true
    retry:
      limit: 5
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 3m

  revisionHistoryLimit: 10  # Keep last 10 sync history

Three fields do the heavy lifting:

  • prune: true deletes live resources that no longer exist in Git — without it, deleted manifests linger forever.
  • selfHeal: true is the actual drift reconciliation; without it ArgoCD reports OutOfSync but leaves the manual change in place.
  • allowEmpty: false is your seatbelt: if the repo path is accidentally emptied (bad path rename, empty commit), ArgoCD refuses to wipe the namespace instead of pruning everything.

revisionHistoryLimit: 10 matters for rollback — it bounds how far back argocd app rollback can reach without touching Git.

AppProject for Multi-Team

AppProject is the RBAC boundary: which repos a team may deploy from, which namespaces they may target, and which cluster-level objects they may touch.

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: production
  namespace: argocd
spec:
  description: Production environment applications

  # Allowed source repositories
  sourceRepos:
    - https://github.com/myorg/k8s-manifests
    - https://github.com/myorg/api-config

  # Allowed destination namespaces
  destinations:
    - server: https://kubernetes.default.svc
      namespace: production
    - server: https://kubernetes.default.svc
      namespace: production-*

  # Allowed cluster resources
  clusterResourceWhitelist:
    - group: ''
      kind: Namespace
    - group: rbac.authorization.k8s.io
      kind: RoleBinding

  # Denied resources
  clusterResourceBlacklist:
    - group: ''
      kind: Node
    - group: ''
      kind: PersistentVolume

  # Roles for team access
  roles:
    - name: platform-team
      description: Platform team access
      policies:
        - p, proj:production:platform-team, applications, get, production/*, allow
        - p, proj:production:platform-team, applications, sync, production/*, allow
      groups:
        - platform-team

The blacklist/whitelist split is the important part: platform teams can manage their own namespaces and RBAC, but can’t create Nodes or PersistentVolumes — the blast radius of a bad Application stays inside the project.

ApplicationSet for Multi-Cluster

ApplicationSet generates one Application per target so “deploy api-service to every prod cluster” is a single manifest instead of N copies.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: api-all-clusters
  namespace: argocd
spec:
  generators:
    - list:
        elements:
          - cluster: us-east-1
            url: https://1.2.3.4
          - cluster: eu-west-1
            url: https://5.6.7.8
          - cluster: ap-southeast-1
            url: https://9.10.11.12
  template:
    metadata:
      name: 'api-{{cluster}}'
    spec:
      project: production
      source:
        repoURL: https://github.com/myorg/k8s-manifests
        targetRevision: main
        path: production/api-service
      destination:
        server: '{{url}}'
        namespace: production
      syncPolicy:
        automated:
          prune: true
          selfHeal: true

The list generator is the simplest; cluster and git generators cover dynamic cases (one Application per registered cluster secret, or per directory in the repo). For per-cluster value overrides, pair ApplicationSet with Kustomize overlays rather than forking the manifest tree.

Flux Setup

Flux is a set of composable controllers (source, kustomize, helm, notification) with no UI. It fits teams that live in the CLI, want the smallest possible footprint, or already standardize on Helm and Kustomize.

Flux Installation

# Install Flux CLI
brew install fluxcd/tap/flux  # macOS

# Bootstrap Flux with GitHub
flux bootstrap github \
  --owner=myorg \
  --repository=fleet-infra \
  --branch=main \
  --path=clusters/production \
  --personal=false

# Check Flux installation
flux check
flux get kustomizations
flux get helmreleases

flux bootstrap is itself GitOps: it commits the Flux manifests into your repo, so Flux upgrades are PRs like everything else. The command is idempotent — run it again to upgrade Flux in place.

Flux Kustomization

A Kustomization ties a repo path to a reconcile loop — the Flux equivalent of ArgoCD’s Application.

# clusters/production/api-service.yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: api-service
  namespace: flux-system
spec:
  interval: 1m  # Reconcile every minute
  path: ./production/api-service
  sourceRef:
    kind: GitRepository
    name: k8s-manifests
  prune: true  # Delete resources removed from Git
  wait: true   # Wait for resources to be ready
  timeout: 5m
  targetNamespace: production

  # Health checks
  healthChecks:
    - apiVersion: apps/v1
      kind: Deployment
      name: api
      namespace: production
    - apiVersion: v1
      kind: Service
      name: api-service
      namespace: production

  # Post-build variable substitution
  postBuild:
    substitute:
      cluster_name: production
      cluster_region: us-east-1

wait: true plus healthChecks changes the failure mode usefully: a sync isn’t “applied” until the Deployment actually reports ready, so a bad image surfaces as a failed Kustomization instead of silently degraded pods. postBuild.substitute injects per-cluster values without duplicating manifests — combine it with ${cluster_name} placeholders in the source YAML.

Flux GitRepository

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: k8s-manifests
  namespace: flux-system
spec:
  interval: 30s
  url: https://github.com/myorg/k8s-manifests
  ref:
    branch: main
  secretRef:
    name: github-deploy-key  # For private repos

Flux HelmRelease

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: redis
  namespace: production
spec:
  interval: 5m
  chart:
    spec:
      chart: redis
      version: "18.x"
      sourceRef:
        kind: HelmRepository
        name: bitnami
        namespace: flux-system
  values:
    architecture: replication
    auth:
      enabled: true
      existingSecret: redis-secret
      existingSecretPasswordKey: redis-password
    replica:
      replicaCount: 3
      persistence:
        enabled: true
        size: 10Gi
    metrics:
      enabled: true
      serviceMonitor:
        enabled: true

Pin the chart to a major-version range (18.x) rather than a bare * — you still get patch releases through Git history, but a breaking major bump can’t roll out unreviewed.

ArgoCD or Flux

ArgoCDFlux
UIFull web UI + API + CLICLI only (UIs are third-party)
Multi-clusterBuilt-in, single control planePer-cluster install (or via Flux.sharding)
App modelApplication / ApplicationSet CRDsKustomization / HelmRelease / GitRepository CRDs
Progressive deliveryArgo RolloutsFlagger
FootprintHeavier (API server, UI, repo server)Minimal controllers
RBACProject-level, fine-grainedKubernetes RBAC only

Pick ArgoCD when several teams need shared visibility and per-app RBAC, or when you manage many clusters from one pane. Pick Flux for CLI-first teams, tight Helm integration, or minimal control-plane overhead. Many organizations run both — Flux for platform infrastructure, ArgoCD for app teams that want the UI.

GitOps Repository Layout

How you lay out the config repo determines how painful multi-environment work gets. A structure that scales:

k8s-manifests/
├── base/                    # Shared manifests (kustomize base)
│   └── api-service/
│       ├── deployment.yaml
│       └── service.yaml
├── production/              # Environment overlays
│   ├── api-service/
│   │   └── kustomization.yaml   # image tag, replicas, ingress host
│   └── secrets/
│       └── api-secrets.yaml     # SOPS-encrypted
├── staging/
│   └── api-service/
│       └── kustomization.yaml
└── clusters/                # Flux bootstrap / per-cluster wiring
    └── production/
        └── api-service.yaml

Two rules hold up in practice:

  • One GitOps repo per environment (or cluster), not per app. A single source of truth only works if “the truth” is in one place; per-app repos scatter it.
  • Overlays over branches. A staging branch that merges to main for promotion sounds appealing and produces constant drift between branches. Directory-per-environment with Kustomize overlays keeps every environment visible at once.

Secret Management with SOPS

Plaintext secrets in Git are a hard no — Git history is forever. The GitOps-compatible options:

  • SOPS + age/KMS: encrypt secret files client-side, commit them, let the agent decrypt in-cluster. Works with Flux natively and ArgoCD via plugins.
  • External Secrets Operator: keep secrets in AWS Secrets Manager, Vault or Key Vault; ESO syncs them into Kubernetes Secrets. Best when a secrets manager already exists.
  • Sealed Secrets: encrypt against a cluster-specific public key; the sealed file is safe to commit. Simple, but rotation is awkward.

SOPS is the lightest starting point:

# Install SOPS and age
brew install sops age

# Generate age key pair
age-keygen -o sops-key.txt
# Public key: age1xxxxx...
# Private key: AGE-SECRET-KEY-1xxxxx...

# Encrypt a secret file
sops --encrypt --age age1xxxxx... --in-place secret.yaml

# Decrypt a secret file
sops --decrypt secret.yaml

# Edit an encrypted file in-place
sops secret.yaml

# Rotate keys
sops updatekeys secret.yaml
# Encrypted secret (SOPS)
apiVersion: v1
kind: Secret
metadata:
  name: api-secrets
  namespace: production
type: Opaque
stringData:
  DB_PASSWORD: ENC[AES256_GCM,data:abc123,iv:xxx,tag:yyy,type:str]
  API_KEY: ENC[AES256_GCM,data:def456,iv:xxx,tag:yyy,type:str]
sops:
  age:
    - recipient: age1xxxxx...
      enc: |
        -----BEGIN AGE ENCRYPTED FILE-----
        ...
        -----END AGE ENCRYPTED FILE-----
  lastmodified: "2026-07-04T12:00:00Z"
  mac: ENC[AES256_GCM,data:mac,iv:xxx,tag:yyy,type:str]
# Flux Kustomization with SOPS decryption
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: api-secrets
  namespace: flux-system
spec:
  interval: 1m
  path: ./production/secrets
  sourceRef:
    kind: GitRepository
    name: k8s-manifests
  prune: true
  decryption:
    provider: sops
    secretRef:
      name: sops-age-key  # Secret containing the private key

The private age key lives as a cluster Secret (sops-age-key) — never in Git. Rotating it means sops updatekeys on every encrypted file plus a new cluster Secret; script that before you need it at 3 AM. If you need centralized secret storage beyond Kubernetes, the External Configuration Store Pattern covers that side.

Progressive Delivery

Progressive delivery caps the blast radius of a bad release: route a fraction of traffic to the new version, watch error rate and latency, and either promote or roll back automatically based on metrics.

flowchart diagram: Deploy v2

Argo Rollouts

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: api
  namespace: production
spec:
  replicas: 10
  strategy:
    canary:
      canaryService: api-canary
      stableService: api-stable
      trafficRouting:
        nginx:
          stableIngress: api-stable-ingress
      steps:
        - setWeight: 10
        - pause: { duration: 2m }
        - setWeight: 25
        - pause: { duration: 2m }
        - setWeight: 50
        - pause: { duration: 5m }
        - setWeight: 75
        - pause: { duration: 5m }
      analysis:
        templates:
          - templateName: success-rate
        startingStep: 2  # Start analysis after 25% canary
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      containers:
        - name: api
          # CI sets the immutable tag on every release — never "latest"
          image: ghcr.io/myorg/api:v1.4.2
          ports:
            - containerPort: 8000
          readinessProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 5
            periodSeconds: 10
# Analysis template for canary
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: success-rate
  namespace: production
spec:
  metrics:
    - name: success-rate
      interval: 1m
      successCondition: result[0] >= 0.99
      failureLimit: 3  # Abort after 3 failures
      provider:
        prometheus:
          address: http://prometheus.monitoring:9090
          query: |
            sum(rate(http_requests_total{status!~"5.."}[2m]))
            /
            sum(rate(http_requests_total[2m]))

The AnalysisTemplate is what makes this a quality gate instead of a fancy load balancer: Prometheus is queried every minute, and three failed checks abort the rollout. Tune successCondition to a signal you actually alert on — a 99% success threshold with a 1m window catches obvious breakage fast; subtler regressions need longer steps and a p95 latency metric alongside.

Flux Flagger

apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
  name: api
  namespace: production
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  service:
    port: 80
    targetPort: 8000
    gateways:
      - public-gateway.istio-system.svc.cluster.local
    hosts:
      - api.stackpractices.com
  analysis:
    interval: 1m
    threshold: 5
    maxWeight: 50
    stepWeight: 10
    metrics:
      - name: request-success-rate
        thresholdRange:
          min: 99
        interval: 1m
      - name: request-duration
        thresholdRange:
          max: 500
        interval: 30s
    webhooks:
      - name: load-test
        type: rollout
        url: http://flagger-loadtester.test/
        timeout: 5s
        metadata:
          cmd: "hey -z 1m -q 10 -c 2 http://api.production:80/"

Flagger shifts stepWeight (10%) each interval (1m) until maxWeight, checking success rate and p99 duration each step. The load-test webhook matters more than it looks: without baseline traffic, a canary that gets 2 requests a minute produces metrics noise, not signal.

Automated Rollback

With GitOps, rollback is a Git operation — that’s the point. Two routes, and you should know when each applies:

  • git revert + push (preferred). The agent syncs the cluster to the reverted state. The rollback itself is auditable, reviewable history — same workflow as any change.
  • ArgoCD point-in-time rollback (emergency). argocd app rollback pins the app to a previous in-cluster revision without a Git commit. Faster, but it bypasses the audit trail — follow up with a git revert so Git and cluster converge again.
# ArgoCD Application — history settings that enable rollback
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: api-service
  namespace: argocd
spec:
  revisionHistoryLimit: 10  # argocd app rollback can reach back this many revisions
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
# ArgoCD rollback — uses the history ID, not a Git SHA
argocd app history api-service            # find the revision ID to roll back to
argocd app rollback api-service <id>      # pin to that revision

# Flux: rollback by reverting the Git commit
git revert <commit-sha>
git push origin main
# Flux automatically syncs to the reverted state on the next reconcile

Watch out for the rollback trap with selfHeal: if you roll back by pinning a revision while selfHeal is on, ArgoCD sees the pinned state as drift versus Git and fights you. Disable auto-sync first (argocd app set api-service --sync-policy none), or do the git revert and let normal sync handle it.

CI/CD Integration

CI’s job in GitOps is to build the artifact and update the GitOps repo — never to touch the cluster.

# GitHub Actions: Build and update GitOps repo
name: Deploy
on:
  push:
    branches: [main]

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build and push image
        run: |
          docker build -t ghcr.io/myorg/api:${{ github.sha }} .
          echo ${{ secrets.GITHUB_TOKEN }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin
          docker push ghcr.io/myorg/api:${{ github.sha }}

      - name: Update GitOps repo
        run: |
          git clone https://github.com/myorg/k8s-manifests.git
          cd k8s-manifests

          # Update image tag in manifest
          sed -i "s|ghcr.io/myorg/api:.*|ghcr.io/myorg/api:${{ github.sha }}|" production/api-service/deployment.yaml

          git config user.name "CI Bot"
          git config user.email "ci@stackpractices.com"
          git add .
          git commit -m "Deploy api:${{ github.sha }}"
          git push

Production-hardened notes on that workflow:

  • Tag images with ${{ github.sha }} (or semver), never latest. GitOps only works if the desired state in Git is exact — latest is mutable state hiding outside Git.
  • Prefer kustomize edit set image api=ghcr.io/myorg/api:${{ github.sha }} over sed once your manifests are Kustomize-based; it’s syntax-aware and can’t corrupt a manifest.
  • For many apps, skip hand-rolled sed entirely and use ArgoCD Image Updater or Flux’s ImagePolicy/ImageRepository CRDs — the agent writes tag commits back to the repo itself.

Companion code: GitOps production manifests: every YAML file from this guide — ArgoCD, Flux, SOPS, rollouts and the CI workflow.

Best Practices

  • Separate app repo from config repo. CI pushes image tags to the config repo; the agent only reads it. Mixing them creates a commit storm in the repo your auditors read.
  • Immutable image tags everywhere. A v1.4.2 tag in Git is a fact; latest is a rumor. Rollback, drift detection and promotion all assume exact desired state.
  • Gate syncs on health, not just apply. Flux wait: true + healthChecks, ArgoCD sync waves + health assessment — a sync that returns success while pods crash is worse than a visible failure.
  • Use sync waves for ordered dependencies. Annotate resources with argocd.argoproj.io/sync-wave (CRDs and RBAC first, apps last) so a fresh environment converges in the right order.
  • Cap revisionHistoryLimit deliberately. 10 is a sane default — enough rollback runway, small etcd footprint.
  • Rehearse key rotation. sops updatekeys, new cluster Secret, re-encrypt — if you’ve never done it, the first time will be during an incident.

Common Mistakes

  • selfHeal: true + allowEmpty unset. One empty-path accident and the agent prunes the namespace. Always set allowEmpty: false on automated sync policies.
  • Secrets committed in plaintext “temporarily”. Git remembers. If it ever touched the repo unencrypted, rotate it — the leak already happened.
  • Reconcile intervals too aggressive. interval: 10s × 200 applications = constant API-server and Git-provider load. Start at 1–5m and tighten only where latency matters.
  • Pruning CRDs or shared namespaces. prune: true on an Application that owns cluster-level objects can delete resources other apps depend on. Keep cluster-scoped objects in a separate, non-pruned Application.
  • Letting CI kubectl apply “just this once”. Every direct apply is drift the agent will either revert or hide. No exceptions — that’s how the audit trail stays meaningful.
  • Environment promotion via branch merges. Staging→prod via merged branches produces two diverging histories. Promote by copying the image tag (or overlay change) into the prod path via PR instead.

Troubleshooting

Sync failures that keep coming back in production, and how to read them:

Application stuck OutOfSync after sync runs. Run argocd app diff api-service — the diff usually shows an immutable field (spec.selector on a Deployment can’t change in place) or an admission webhook rejecting the manifest. Fix by deleting the object so the agent recreates it, or adding the Replace=true sync option for that resource.

Cascade delete hangs forever. The resources-finalizer blocks Application deletion while a managed resource is itself stuck (finalizer loop, unreachable namespace). As a last resort, strip the finalizer: kubectl -n argocd patch app api-service --type=json -p='[{"op":"remove","path":"/metadata/finalizers"}]' — you’ll have to clean up the orphaned resources yourself.

Flux Kustomization shows Ready=False but pods are Running. wait: true plus healthChecks verifies the exact objects listed — a renamed Deployment fails the check even when the workload is healthy. kubectl -n flux-system describe kustomization api-service shows which object timed out.

SOPS decryption errors after key rotation. Either the cluster Secret holding the age private key wasn’t updated, or the Kustomization’s secretRef still points at the old key name. Compare kubectl -n flux-system get secret sops-age-key against the recipient line in the encrypted file — they must match.

ApplicationSet generated apps for the wrong clusters. The list or cluster generator matched more than intended. Inspect the generated names with kubectl -n argocd get applications before enabling prune — a misconfigured generator at scale doesn’t just create apps, it deletes them.

See Also

Further Reading

Frequently Asked Questions

What is the difference between ArgoCD and Flux?

Both implement GitOps for Kubernetes. ArgoCD ships a web UI, multi-cluster from a single control plane, and ApplicationSets for templating. Flux is lighter, is pure CRDs, and has tighter native Helm and Kustomize integration. Choose ArgoCD if teams need a UI and centralized multi-cluster. Choose Flux for CLI-driven workflows and minimal footprint.

How does drift reconciliation work?

The agent continuously compares live cluster state with the desired state in Git. When it detects a difference — a manual kubectl edit, a deleted resource — it re-applies the Git state. With selfHeal: true (ArgoCD) or prune: true + automated reconciliation (Flux), manual changes are reverted automatically; without self-heal, drift is only reported as OutOfSync.

How do I handle secrets in GitOps?

Never commit plaintext secrets. Encrypt secret files client-side with SOPS (age or AWS KMS) and commit the ciphertext — the agent decrypts in-cluster with a private key stored as a cluster Secret. Or keep secrets outside Git entirely with External Secrets Operator (pulls from AWS Secrets Manager, Vault, Key Vault) or Sealed Secrets. Full details in Secret Management with SOPS.

What is progressive delivery in GitOps?

Releasing a new version to a fraction of traffic — 10%, 25%, 50% — while automated metric checks run at each step, instead of updating all pods at once. Argo Rollouts and Flagger implement this: if the success-rate or latency check fails, traffic snaps back to the stable version. It's a quality gate, not just traffic shaping.

Should I use one Git repo or multiple repos for GitOps?

One config repo per cluster or environment, separate repos for application source. A single GitOps repo keeps the complete desired state visible and makes changes atomic; app code lives elsewhere so CI can update image tags without the agent needing write access to app code. See GitOps Repository Layout.

How do I roll back a deployment in GitOps?

Prefer git revert + push — the agent syncs to the reverted state and the rollback is itself audited history. For emergencies, argocd app rollback <app> <id> pins to a previous revision instantly (get the ID from argocd app history), but follow up with a revert so Git catches up. Details and the self-heal gotcha in Automated Rollback.

How does CI update the GitOps repo?

CI builds and pushes the image, clones the config repo, updates the image tag (kustomize edit set image or a templated sed), commits and pushes. The agent picks up the change on its next reconcile. ArgoCD Image Updater or Flux ImagePolicy automate the tag-commit step entirely.

Does GitOps work for databases and stateful workloads?

Partially. Deploying the StatefulSet and its config via GitOps works fine; schema migrations and failover decisions don't map well to declarative sync and usually stay as CI jobs or operators. The rule of thumb: if the operation is "converge to this state", GitOps fits; if it's "run this procedure", keep it out of the sync loop.