Skip to content

Repository files navigation

ServiceProject Operator

A Kubernetes operator, built with controller-runtime, that turns a single cluster-scoped ServiceProject custom resource into a ready-to-use, quota-bounded namespace — optionally pre-loaded with a Helm release.

apiVersion: infra.sudhir.dev/v1alpha1
kind: ServiceProject
metadata:
  name: team-a
spec:
  tier: small

...produces a team-a Namespace with a tiered ResourceQuota and LimitRange already in place.

Why

Platform teams that hand out namespaces to application teams usually end up scripting the same four steps by hand: create the namespace, apply a quota, apply a limit range, and (often) install some baseline chart. This operator turns that into a single declarative object with a fixed, opinionated sizing menu (small / medium / large), so requesting a namespace looks like requesting any other Kubernetes resource.

Architecture

flowchart TB
    subgraph User
        SP[ServiceProject CR<br/>cluster-scoped]
    end

    subgraph "serviceproject-operator"
        R[ServiceProjectReconciler]
        Q["internal/quotas<br/>tier → ResourceQuota/LimitRange"]
        H["internal/helm<br/>Installer (helm.sh/helm/v3 SDK)"]
    end

    subgraph "Provisioned Namespace"
        NS[Namespace]
        RQ[ResourceQuota]
        LR[LimitRange]
        REL[Helm Release<br/>optional]
    end

    SP -- watch --> R
    R -- create/update --> NS
    R -- uses --> Q
    Q -- desired spec --> RQ
    Q -- desired spec --> LR
    R -- create/update --> RQ
    R -- create/update --> LR
    R -- EnsureRelease --> H
    H -- install/upgrade --> REL
    R -- status: phase, specHash,<br/>namespaceName, helmRevision --> SP
Loading

Reconcile loop

  1. Finalizer. On first sight of a ServiceProject, the controller adds infra.sudhir.dev/finalizer before doing anything else, so deletion can always be intercepted for ordered teardown.
  2. Namespace. Creates (or labels/annotates) a Namespace named after spec.namespaceName, falling back to the ServiceProject's own name.
  3. Quota / LimitRange. Builds the desired ResourceQuota and LimitRange for spec.tier (see Tiers) and reconciles them. Comparisons use resource.Quantity.Cmp, not reflect.DeepEqual / apiequality.Semantic, so "1000m" and "1" are correctly treated as equal and never cause a spurious update.
  4. Helm (optional). If spec.helm is set, the controller computes the sha256 of the canonical JSON encoding of spec and compares it against status.specHash. A Helm install/upgrade only runs when the hash changed — this is the content-hash gate that keeps reconciles cheap when nothing meaningful changed (Helm upgrades are comparatively expensive: they diff manifests and talk to the API server for every resource in the chart).
  5. Status. Publishes phase, specHash, namespaceName, helmRevision, and a standard Ready condition, then requeues on a 5-minute timer as a drift-correction safety net.

Deletion (finalizer-ordered teardown)

sequenceDiagram
    participant K as kube-apiserver
    participant C as Controller
    participant Helm as Helm release
    participant Q as Quota/LimitRange
    participant NS as Namespace

    K->>C: DeletionTimestamp set (finalizer blocks removal)
    C->>Helm: uninstall release (if spec.helm set)
    C->>Q: delete ResourceQuota, LimitRange
    C->>NS: delete Namespace
    C->>K: remove finalizer
    K->>K: ServiceProject object removed
Loading

The Helm release is torn down first — while its namespace still exists — so any pre-stop hooks or graceful-shutdown behavior in the chart's workloads still have a functioning namespace to run in. The namespace is deleted last, which also transitively cleans up anything the chart created that the controller doesn't track individually.

Tiers

Tier CPU (requests & limits) Memory (requests & limits)
small 2 4Gi
medium 4 8Gi
large 8 16Gi

spec.tier defaults to small when omitted. The generated LimitRange sets a per-container default limit equal to the full tier allotment and a default request of half that, so a namespace can host more than one container without immediately exhausting its quota. See internal/quotas/tiers.go.

CRD reference

apiVersion: infra.sudhir.dev/v1alpha1
kind: ServiceProject
metadata:
  name: team-b            # also the default namespace name
spec:
  tier: medium             # small | medium | large (default: small)
  namespaceName: team-b    # optional override
  namespaceLabels: {}      # optional, merged onto the Namespace
  namespaceAnnotations: {} # optional, merged onto the Namespace
  helm:                    # optional
    chart: deploy/charts/sample   # local path or repo chart name
    repo: ""                       # chart repository URL (repo charts only)
    version: ""                     # chart version (repo charts only)
    releaseName: team-b-sample     # defaults to metadata.name
    values:                        # freeform, same shape as values.yaml
      replicaCount: 2
status:
  phase: Ready
  observedGeneration: 1
  specHash: <sha256 hex>
  namespaceName: team-b
  helmRevision: 1
  conditions:
    - type: Ready
      status: "True"
      reason: ReconcileSucceeded

ServiceProject is cluster-scoped (spec.scope: Cluster): it's the platform team's request object, not something that lives inside the namespace it creates.

Repository layout

api/v1alpha1/          CRD Go types + hand-written DeepCopy (no controller-gen)
controllers/           Reconciler + unit tests (envtest-free, fake client)
internal/quotas/       Tier → ResourceQuota/LimitRange builders + semantic equality
internal/helm/         Thin wrapper around the helm.sh/helm/v3 SDK
deploy/                CRD, RBAC, operator Deployment manifests
deploy/charts/sample/  Local Deployment+Service chart used by config/samples
config/samples/        Example ServiceProject objects

Quickstart

Prerequisites: a Kubernetes cluster and kubectl pointed at it (e.g. kind create cluster), and Go 1.26+ to build locally.

# 1. Build
make build

# 2. Install the CRD
make install

# 3. Deploy the operator
make deploy

# 4. Try it out
make sample
kubectl get serviceprojects
kubectl get ns team-a team-b
kubectl -n team-b get resourcequota,limitrange
kubectl -n team-b get pods            # team-b's Helm release (sample chart)

# 5. Tear down
kubectl delete -f config/samples/serviceproject.yaml
make undeploy
make uninstall

To iterate on the controller against whatever cluster your kubeconfig points at, without building an image:

make install
go run . --leader-elect=false

Building the image

make docker-build IMG=ghcr.io/coder-in-a-shell/serviceproject-operator:dev
make docker-push  IMG=ghcr.io/coder-in-a-shell/serviceproject-operator:dev

Testing

go mod tidy
go build ./...
go vet ./...
go test ./...

All tests run against sigs.k8s.io/controller-runtime/pkg/client/fake and a hand-rolled fake helm.Interface — no real cluster or Tiller/Helm storage backend is required. internal/helm additionally loads and renders deploy/charts/sample through the real Helm chart loader/engine, so a broken template is caught at go test time.

Design notes / trade-offs

  • No controller-gen. Types, DeepCopy, and the CRD YAML are hand-written per the project's constraints. api/v1alpha1/deepcopy.go must be kept in sync by hand when types.go changes — a normal kubebuilder project would regenerate this instead.
  • No owner references on child objects. The Namespace/ResourceQuota/ LimitRange/Helm release are intentionally not linked via ownerReferences, so Kubernetes garbage collection can never race with the finalizer-ordered teardown described above. Drift is instead corrected by the 5-minute periodic resync.
  • Content-hash gate, not generation-only gate. Comparing status.observedGeneration to metadata.generation would also detect "did the spec change," but storing a spec hash makes the gate explicit and testable in isolation, and leaves room to hash additional inputs (e.g. a referenced chart's digest) later without changing the comparison mechanism.

About

Kubernetes operator: one YAML -> namespace + tiered ResourceQuota/LimitRange + optional Helm release. Idempotent reconciliation, content-hash gating, finalizer-ordered teardown.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages