This is a modern GitOps repository managing a complete Observability and CD stack using Argo CD, Kargo, Grafana, and Mimir. It follows a declarative approach where infrastructure configuration lives in Git, while sensitive data is decoupled from the codebase.
appsets/: Argo CD ApplicationSets (one per service). Managed by the Root App.apps/: Helm charts and stage-specific values.base/: Common configuration shared across all environments.prod/: Production overrides (primary focus).
0day-deployment-manifests/: Templates for manual bootstrap (Secrets, Repo-Access). SeeBOOTSTRAP.mdinside this folder for setup instructions.manifests/: Static Kubernetes manifests (e.g., Kargo Stages).
| Application | Role | Access URL |
|---|---|---|
| Argo CD | Continuous Delivery (GitOps) | argocd.saadisfy.me |
| Grafana | Visualization & Alerting | grafana.saadisfy.me |
| Kargo | Multi-Stage Promotion | kargo.saadisfy.me |
| Mimir | Long-term Metric Storage | mimir.saadisfy.me |
| Alloy | Telemetry Collection | (Internal Cluster DaemonSet) |
| OpenTelemetry Demo | Microservices demo workload | opentelemetry-demo.saadisfy.me |
This repository is designed to be public. We use two mechanisms to keep it secure:
- External Secrets: Applications like Grafana use
existingSecretreferences. The Kubernetes Secret is created once manually; the Helm chart only references it by name. - Selective ignoreDifferences: For Argo CD and Kargo, the Helm charts manage the Secret structure (including the auto-generated
server.secretkey), but we useignoreDifferencesin the ApplicationSet to prevent Git placeholders from overwriting the manually set admin password hashes.
- Collector: Grafana Alloy runs as a DaemonSet, scraping metrics (KSM, Node-Exporter) and forwarding them via OTLP.
- Storage: Grafana Mimir (Distributed) stores metrics with high efficiency.
- Dashboarding: Grafana Operator manages dashboards and alerts as code via Custom Resources (CRs).
- Auto-Reload: Stakater Reloader monitors Secrets and ConfigMaps to trigger zero-downtime rolling restarts on changes.
Promotions between stages (Dev -> Int -> Prod) are handled by Kargo.
- Freight is composed of Git commits and Container images.
- Promotions update stage-specific
values.yamlfiles via automated commits. - Managed via
apps/kargo-projects/.
Detailed information about the architecture and usage of this stack:
- Observability Guide: Central source of truth for the observability stack, including architecture, onboarding, and operational know-how.
- Data Pipeline Concept (Alloy): Deep dive into the Grafana Alloy pipeline (Receive -> Process -> Export) and label enrichment strategy.
- Instrumentation & Pod Association: How applications are identified and enriched with Kubernetes metadata.
Maintained by Saad Masood. Managed via Argo CD.