Alerting

Alerting

CompareQueries supports Grafana Alerting via backend query execution.

This page explains how to configure alerts with CompareQueries, common pitfalls, and troubleshooting steps.

---

Scope and Compatibility

• Supported in CompareQueries backend-enabled versions (2.1.0+).
• Works on Grafana 11/12/13+ (with recommended Target Datasource flow).
• Alerting does not use panel-level -- Mixed -- query composition.

---

How Alerting Works in CompareQueries

In dashboards, CompareQueries is often used in a -- Mixed -- panel. In alert rules, execution is backend-driven, so you should configure the CompareQueries query directly with:

• Target Datasource
• Embedded target query
• One or more time shifts

CompareQueries will execute shifted queries in backend and return merged series for alert expressions.

---

Minimal Setup

1. Create or open a CompareQueries datasource in Connections -> Data sources.
2. Keep Authentication as No Authentication by default.
3. If backend/alert requests fail auth, switch to Basic authentication and set:
   • Service Account Token (Viewer or above)
   • optional Grafana URL (only if auto-detection is wrong)
4. Go to Alerting -> Alert rules -> New alert rule.
5. Add a query using CompareQueries datasource.
6. In CompareQueries query editor:
   • Select Target Datasource
   • Build target query inline
   • Add time shift rows (for example 1d, 1w)
7. Add alert expressions, typically:
   • Reduce
   • Threshold
8. Use Preview alert rule condition to validate query and expression output.

---

Recommended Query Pattern

• Keep one base row with empty Amount (current window).
• Add shifted rows (1d, 1w, etc.) for comparisons.
• Use clear aliases so downstream expressions are readable.
• If needed, enable Process TimeShift to align shifted timestamps to current window for direct comparison.

---

Authentication Guidance

Default recommendation:

• Start with No Authentication.
• Move to Basic authentication only when backend/alerting calls fail auth.

When using Basic authentication:

• Use a Grafana Service Account token.
• Keep token in secure configuration.
• Set Grafana URL only when environment auto-detection fails (reverse proxy, nonstandard routing, etc.).

---

Troubleshooting

1. Alert preview returns empty/no data

Check:

• Target datasource query itself returns data for the alert evaluation window.
• Time shift values are valid (1d, 1w, 30m, etc.).
• Alias/series selection in expressions matches returned series.

2. Authentication/permission errors

Check:

• CompareQueries datasource auth mode.
• Service account token validity and role.
• Grafana URL setting (if required in your deployment).

3. Works in panel but fails in alert rule

Likely cause:

• Panel behavior relied on -- Mixed -- composition.
• Alerting requires backend execution path with direct CompareQueries query definition.

Rebuild the query in the alert rule using Target Datasource flow.

4. Series alignment issues in expressions

Try:

• Enabling/disabling Process TimeShift depending on whether you need aligned timestamps.
• Reducing expression complexity first (single series) before adding multi-series comparisons.

---

Operational Notes

• Prefer simple, deterministic query+expression chains in alerts.
• Validate alert logic with Preview before enabling notifications.
• For production, avoid relying on anonymous access behavior from local/dev environments.

---

Example Alert Rule Flow

1. Query A (CompareQueries):
   • Base: current period
   • Shifted: 1d
2. Expression B (Reduce): aggregate target series
3. Expression C (Threshold): trigger when delta exceeds limit

---

Related Pages

• Quick Start
• Troubleshooting
• FAQ
• Developer Guide (advanced backend logs and diagnostics)