Prometheus Monitoring Mixin for ArgoCD

A set of Grafana dashboards and Prometheus alerts for ArgoCD.

How to use

This mixin is designed to be vendored into the repo with your infrastructure config. To do this, use jsonnet-bundler:

You then have three options for deploying your dashboards

1. Generate the config files and deploy them yourself
2. Use jsonnet to deploy this mixin along with Prometheus and Grafana
3. Use prometheus-operator to deploy this mixin

Or import the dashboard using json in ./dashboards_out, alternatively import them from the Grafana.com dashboard page.

Generate config files

You can manually generate the alerts, dashboards and rules files, but first you must install some tools:

go get github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb
brew install jsonnet

Then, grab the mixin and its dependencies:

git clone https://github.com/adinhodovic/argo-cd-mixin
cd argo-cd-mixin
jb install

Finally, build the mixin:

make prometheus_alerts.yaml
make dashboards_out

The prometheus_alerts.yaml file then need to passed to your Prometheus server, and the files in dashboards_out need to be imported into you Grafana server. The exact details will depending on how you deploy your monitoring stack.

Configuration

ArgoCD Badges

You can add ArgoCD application badges to the dashboards by configuring the applications field in config.libsonnet. This allows you to display links to your ArgoCD applications directly in the Grafana dashboards.

Example configuration:

local argoCdMixin = import 'argo-cd-mixin/mixin.libsonnet';

argoCdMixin {
  _config+:: {
    applications: [
      {
        name: 'my-app',
        namespace: 'argocd',
        applicationName: 'my-app',  // or self.name
        environment: 'Production',
        argoCdUrl: 'https://argo-cd.example.com',  // or use $._config.argoCdUrl
      },
      {
        name: 'another-app',
        namespace: 'argocd',
        environment: 'Staging',
        argoCdUrl: 'https://argo-cd.example.com',
      },
    ],
  },
}

Alerts

The mixin follows the monitoring-mixins guidelines for alerts.

Latency alert thresholds

Argo CD's reconciliation histogram ends at 16 seconds, and its Git request histogram ends at 20 seconds. The default P95 alerts use >= 16 and >= 20 to detect saturation of these last finite buckets. They cannot distinguish a reconciliation taking 60 seconds from one taking 120 seconds, or a Git request taking 30 seconds from one taking 45 seconds.

To measure higher latency thresholds, add wider buckets in Argo CD itself and override alerts.highReconciliationDuration.threshold and alerts.highGitRequestDuration.threshold. Changing the mixin threshold alone cannot extend the histogram's range.

Preview

Operational Dashboard

Application Dashboard

Notification Dashboard