GKE - Part 7: Build Once, Promote Forward with Kargo

gkekargoargocdgitopskubernetesplatform-engineeringcicdgcp

GKE - Part 7: Build Once, Promote Forward with Kargo

One of the principles behind my delivery platform is that an application artifact should be built once and then promoted through environments without being rebuilt along the way. CI is responsible for producing and validating the artifact, while dev, staging, and production are destinations that consume that same artifact. I wrote about this approach previously in Build Once. Promote Forward. Ship Daily., where the main goal was to remove rebuild-per-environment pipelines and make promotion a lightweight, predictable operation instead of another build cycle.

That model already worked well with GitOps. GitHub Actions built and published the container, Git stored the desired image version for each environment, and Argo CD reconciled those changes into Kubernetes. What was missing was a dedicated layer that understood the delivery path itself. The mechanics of promotion existed, but the meaning of promotion was mostly encoded in workflow steps, scripts, repository conventions, and permissions spread across several systems.

Kargo fits into that gap.

Kargo Logo

The GitOps Workflow Before Kargo

The original flow was straightforward and intentionally simple. CI built the application, ran the required checks, pushed the resulting image to the registry, and then automation updated the Git repository used by Argo CD.

GitHub
   |
   v
GitHub Actions
   |
   | Build, test, scan
   v
Container Registry
   |
   v
Deployment Repository
   |
   v
Argo CD
   |
   v
Kubernetes

Promotion was usually implemented as a small Git change. A workflow could update the image tag in a Helm values file, commit the result, and let Argo CD reconcile the environment. That pattern was reliable and aligned well with GitOps because the deployment repository remained the source of truth.

I still think this is a good design for many platforms. It is easy to understand, uses familiar tools, and does not require a large delivery framework. The challenge appears later, when the promotion logic begins to carry more responsibility than simply changing a value in YAML.

As more environments and applications are added, the workflow starts needing to understand which artifact is currently in dev, which artifact should move to staging, whether the deployment completed successfully, whether the same artifact was already verified somewhere else, and who is allowed to move it forward. None of those problems are especially difficult by themselves, but together they turn a small automation script into a growing release system.

Kargo Fits the Build-Once Model

What interested me about Kargo was not a new deployment mechanism. Its delivery model matches an architecture I already wanted: create an artifact once, give it a stable identity, and move that artifact through a defined sequence of environments.

The resulting flow looks more like this:

GitHub Actions
      |
      | Build once
      v
Container Registry
      |
      v
    Kargo
      |
      | Promote
      v
 Dev ---> Staging ---> Production
      |
      v
     Git
      |
      v
   Argo CD
      |
      v
 Kubernetes

CI continues to own the creation of software artifacts. Argo CD continues to own reconciliation between Git and Kubernetes. Kargo sits between those two responsibilities and models how an artifact moves through the delivery process.

That division is useful because it keeps build and promotion as separate concerns. The build pipeline answers whether an artifact is valid enough to publish. The promotion system decides where that exact artifact should run next. Environment-specific configuration can still change, but the application artifact itself remains unchanged as it moves forward.

Warehouse, Freight, and Stage

Kargo introduces its own terminology, but the three concepts that mattered most in my setup were Warehouse, Freight, and Stage.

A Warehouse watches for artifacts that are eligible to enter the delivery process. In my case, that means watching a container registry for newly published application images. The Warehouse can also filter what it considers valid, which is useful when a registry contains several image families, helper tags, or artifacts that should never enter the promotion path.

Once Kargo discovers a valid artifact, it represents that artifact as Freight. This is one of the parts of the model I ended up liking more than I expected. In a traditional GitOps workflow, the identity of a release is often just an image tag sitting inside a values file. With Freight, the artifact becomes something the delivery system can track as it moves between environments.

A Stage represents one of those environments. Dev, staging, and production can therefore be modeled as part of the delivery system instead of existing only as naming conventions inside repositories and CI jobs.

New Freight
     |
     v
    Dev
     |
     v
  Staging
     |
     v
Production

This maps cleanly to the build-once model because the Freight stays the same while its destination changes. The image that reaches staging is the same image that previously ran in dev, and the image that eventually reaches production can be traced through the same promotion path.

Promotion Becomes an Explicit Platform Operation

Before Kargo, promotion was represented indirectly by a sequence of lower-level actions. A workflow cloned the deployment repository, changed a Helm value, committed the update, and waited for Argo CD to react. The end result was a promotion, but the systems performing those steps mostly saw file changes and workflow executions.

Kargo gives promotion a first-class representation. Instead of expressing the process only as a set of Git operations, the platform understands that a particular Freight is being moved into a particular Stage.

This provides context that would otherwise need to be reconstructed from Git history, CI logs, and Argo CD state. The platform can show which artifact is associated with each environment, how it moved there, and what happened during the promotion.

The underlying GitOps model still stays intact. Kargo can perform the configuration change needed for the target environment, while Argo CD continues to reconcile that desired state into Kubernetes. The difference is that the operation now carries release intent instead of being treated as a generic repository update.

Reusable Promotion Logic

One area where Kargo immediately helped was reducing repeated delivery logic across applications.

Without a shared abstraction, each project can gradually accumulate its own version of the same workflow: clone the repository, locate the correct values file, update the image reference, commit the change, trigger reconciliation, and wait for the application to become healthy.

That repetition is manageable at first, but it becomes maintenance overhead when the delivery convention changes. A new validation step, repository structure, commit convention, or Argo CD behavior may require updating many separate workflows.

Kargo’s promotion tasks provide a better place to keep that logic. Common promotion behavior can be defined once and reused by multiple applications, while application-specific configuration remains relatively small.

This is closer to how I want a platform boundary to work. Application configuration should describe the artifact and the destination, while the platform owns the mechanics used to move that artifact safely between environments.

Waiting for the Deployment Matters

A Git commit completing successfully does not mean an application finished deploying successfully.

Argo CD may still be synchronizing resources while new pods start, readiness probes run, controllers reconcile dependencies, or infrastructure finishes provisioning. My custom automation could account for that, but it required explicitly querying Argo CD or adding polling logic around the workflow.

Kargo already treats Argo CD state as part of the promotion process. A promotion can update the desired configuration and then wait for the corresponding Argo CD application to reach the expected state before considering the operation complete.

This makes the delivery flow closer to what actually happens in Kubernetes:

Promote artifact
       |
       v
Update Git
       |
       v
Argo CD syncs
       |
       v
Application becomes healthy
       |
       v
Promotion completes

That small distinction removes an important source of ambiguity. The release system is no longer treating “configuration changed” and “application deployed successfully” as the same event.

Visibility Is Part of the Value

The operational view of the delivery pipeline is another area where Kargo improves the experience.

With a custom GitOps workflow, the information already exists, but it is spread across several places. Git can show configuration history, CI can show automation runs, and Argo CD can show the current application state. An engineer who understands the system can usually piece together the release history from those sources.

Kargo adds a view centered specifically on promotion. It becomes much easier to see which artifact is in dev, which one is in staging, what is waiting to move forward, and whether a previous promotion failed.

Kargo UI

That matters beyond convenience. A delivery platform should make its state understandable to the people using it. Developers should not need to understand the internal structure of every deployment repository or workflow just to determine which build is running in an environment.

The promotion model also provides a natural place for RBAC. Teams can be given visibility or controlled promotion permissions without granting broad administrative access to Kubernetes or the GitOps controllers. That creates a cleaner separation between operating the platform and using the delivery capabilities provided by it.

Kargo Versus the Custom Workflow

The custom workflow and Kargo solve many of the same practical problems, so I see them as two stages of the same platform evolution rather than competing architectures.

My original automation was intentionally small. It used tools I already had and added just enough logic to move an image forward through GitOps. As the release process expanded, more delivery concepts had to be implemented around that workflow.

Kargo already has names and abstractions for many of those concepts.

ConcernCustom WorkflowKargo
Discover deployable imagesCI or registry scriptWarehouse
Represent a release artifactImage tagFreight
Model environmentsRepository or workflow conventionStage
Move an artifact forwardScript or CI jobPromotion
Reuse common promotion logicShared script or composite actionPromotionTask
Wait for Argo CDCustom pollingArgo-aware promotion steps
Track promotion stateCI, Git, and Argo CDPromotion model
Control promotion accessCI and repository permissionsRBAC
View delivery progressMultiple systemsPromotion-focused UI

The important difference is not whether Kargo can execute something that a shell script cannot. Most delivery operations can be scripted.

The difference is how much release behavior the platform team wants to design, implement, document, and maintain. Artifact discovery, environment relationships, promotion history, health awareness, access control, and delivery visualization are all things that can be built incrementally around a custom workflow. Eventually, however, that collection of automation starts looking like a small internal release product.

Kargo gives those concerns a common model before the custom system grows that far.

A Better Fit for Multi-Cluster and Multi-Cloud Delivery

This also becomes more useful as the platform grows beyond a single cluster.

My current infrastructure includes workloads across more than one Kubernetes environment and more than one cloud provider. From the application delivery perspective, I want that difference to matter less than it does at the infrastructure layer.

A build should not need to know whether its eventual destination is GKE or EKS. The artifact has already been created and validated. The delivery system should decide where it moves, while the GitOps controller associated with that destination handles deployment.

That gives the platform a cleaner abstraction:

Artifact
   |
   v
Promotion
   |
   +------> GKE environment
   |
   +------> EKS environment

The cluster implementation still matters operationally, but it does not need to redefine the software delivery model. Kargo helps keep artifact promotion above that infrastructure boundary.

For a platform moving toward multi-cluster and multi-cloud operation, that is a useful property. The release process can remain consistent even when the destinations underneath it are different.

Build Once Remains the Important Part

The main reason Kargo fits this platform is that it reinforces the delivery principle that was already there.

The artifact produced by CI should have a stable identity. Testing, scanning, and validation should apply to that artifact. Moving from dev to staging should not create a new build, and reaching production should not depend on reproducing the same artifact later under slightly different conditions.

Promotion should change where the artifact runs, not what the artifact is.

That distinction is the foundation of the whole design. GitOps made promotion lightweight because the environment could be changed by updating desired state. Kargo adds a system that understands the progression behind those changes and can represent the artifact as it moves through the platform.

Reflection

Building the original GitOps automation was valuable because it exposed the mechanics behind promotion. The process is not mysterious: identify the artifact, change the desired configuration, let Argo CD reconcile it, and confirm that the application becomes healthy.

The harder problem appears when that pattern must be standardized across many applications and environments. At that point, the platform needs more than a command that changes an image tag. It needs a consistent way to represent artifacts, environments, promotion history, permissions, and delivery state.

Kargo gives me those abstractions while preserving the GitOps workflow already in place.

GitHub Actions continues to create the artifact. The registry stores it. Kargo tracks and promotes it. Git records the desired state. Argo CD reconciles that state into Kubernetes.

The architecture remains simple, but the release process becomes much easier to reason about.

That is the part I care about most. The goal of the platform is not to replace working automation simply because a new tool exists. It is to recognize when a pattern has become important enough that it deserves a proper abstraction.

For build-once, promote-forward delivery, Kargo has become that abstraction.