RETREAT TO DASHBOARD
QUEST ARCHIVE: FORGEOPS
VIEW REPO
MISSION BRIEFING // ForgeOps
● SYSTEM CASE STUDY

ForgeOps

A Go control plane that reconciles pull requests into ephemeral Kubernetes preview environments, mirroring platforms like Vercel and Railway.

TECH DEPLOYED:GoKubernetesHelmDockerclient-goKind
SPIN-UP TIME
< 45s
Per-PR Preview Namespace
STATE STORE
0-DB
K8s Labels as Source of Truth
CLEANUP
TTL-BASED
Automated Namespace Reclaim

Executive Summary

Engineering teams shipping microservices often face a bottleneck: staging environments become bloated shared single-points-of-failure where PRs block each other. While platforms like Vercel solved this for frontends, replicating isolated, ephemeral preview environments for complex Kubernetes services usually requires expensive enterprise SaaS or bloated CI scripts.

ForgeOps is an autonomous Go control plane engineered to bridge this gap. It intercepts GitHub PR webhooks, dynamically builds service images, and provisions fully isolated, ephemeral Kubernetes preview environments using Helm and client-go—then tears them down automatically when the PR merges or expires.

                  +-----------------------------------+
                  |        GitHub Webhook Event       |
                  |     (PR Opened / Synchronized)    |
                  +-----------------+-----------------+
                                    | (HMAC Verified)
                                    v
                  +-----------------+-----------------+
                  |      ForgeOps Go Controller       |
                  +--------+-----------------+--------+
                           |                 |
         +-----------------+                 +------------------+
         |                                                      |
         v                                                      v
+--------+---------------+                             +--------+---------------+
| containerd / Registry  |                             |    Helm Reconciler     |
| (Layer Mirroring)      |                             | (Dynamic Values Gen)   |
+--------+---------------+                             +--------+---------------+
         |                                                      |
         +--------------------------+---------------------------+
                                    |
                                    v
                  +-----------------+-----------------+
                  |   Ephemeral Preview Namespace     |
                  |   (pr-123-payment-service)        |
                  +-----------------+-----------------+
                                    |
                                    v (On Close / Expire)
                  +-----------------+-----------------+
                  |      TTL Janitor Worker           |
                  |   (Reclaims CPU/RAM & Ingress)    |
                  +-----------------------------------+

Architecture & Design Decisions

1. Stateless Reconciliation: Zero-Database Architecture

Most CI/CD orchestrators introduce an external PostgreSQL or Redis database to track environment state. If the database drifts from actual cluster state, manual cleanup is required.

ForgeOps eliminates external databases entirely by adopting the Kubernetes Controller Pattern:

  • Every ephemeral environment is tagged with structured labels: 
    forgeops.io/managed-by: "forgeops"
forgeops.io/pr-id: "42"
forgeops.io/repo: "core-platform"
forgeops.io/expires-at: "1725380000"
  • Kubernetes itself is the sole source of truth. If the ForgeOps controller crashes or restarts, its reconciliation loop simply queries the Kubernetes API via client-go, instantly reconstructing its state from live namespace metadata.

2. High-Speed Layer Caching with containerd Mirroring

To achieve under-45-second provisioning times without hammering public registries:

  • ForgeOps integrates directly with a local containerd image registry mirror inside the cluster.
  • Common base images and intermediate build layers are cached locally, eliminating external network pull bottlenecks across frequent PR commits.

3. Automated Lifecycle & TTL Janitor Worker

Cluster resources are finite. Preview environments that developers abandon can quickly cause CPU and memory starvation.

  • Dynamic Helm Releases: Each preview environment is encapsulated in a dedicated Helm release with scoped service accounts, resource limits, and unique ingress rules.
  • TTL-Based Garbage Collection: A background worker runs continuous reconciliation sweeps. If a pull request is closed or reaches its maximum configured TTL, ForgeOps performs atomic namespace teardowns, purging services, pods, and ingress routes.
  • Deduplication & Backoff: Incoming webhook bursts (e.g. rapid git commits) are debounced and deduplicated per PR to prevent redundant parallel builds.

Live Controller Telemetry Trace

When a developer opens a pull request, the ForgeOps control plane outputs structured structured telemetry traces:

[2026-09-03 14:02:11] INF webhook received event=pull_request action=opened pr=42 repo=core-api
[2026-09-03 14:02:11] INF hmac signature verified delivery_id=7fa82-bf21-99
[2026-09-03 14:02:12] INF reconciler triggered target_ns=pr-42-core-api status=scaffolding
[2026-09-03 14:02:14] INF registry cache hit layers=4/6 build_time=4.12s
[2026-09-03 14:02:18] INF helm install release=pr-42-core-api namespace=pr-42-core-api chart=./charts/service
[2026-09-03 14:02:26] INF ingress provisioned url=https://pr-42.preview.cluster.local
[2026-09-03 14:02:27] INF status updated github_pr=42 state=success duration=16.3s ttl=4h

Engineering Impact

  • Decoupled Team Velocity: Developers test isolated microservice changes against real cluster APIs without blocking or waiting on shared staging environments.
  • 100% Resource Recovery: Automated TTL janitor routines guarantee zero lingering zombie workloads, preventing unexpected infrastructure cloud costs.
  • Zero-Maintenance Footprint: A single compiled Go binary deployed into Kubernetes with zero external database dependencies.