Add Helm-native provisioning for correlation rules (parity with deduplication/mapping rules)

[abusutil]

Summary

The startup-time resource provisioner (keep/api/config.py:provision_resources) configures providers, workflows, dashboards, deduplication rules, and mapping rules from Helm values, but does NOT cover correlation rules. The only ways to create one today are the UI, the Python SDK, or POST /rules against the backend.

This issue requests symmetric, GitOps-friendly provisioning of correlation rules — provision_correlation_rules_from_env — so they can be declared in values.yaml next to the workflows that the same rules group.

Why this matters

GitOps deployments (Helm + ArgoCD) want the entire alert→incident→ticket pipeline defined in code:

keep:
  backend:
    provision:
      providers: { ... }      # ✅ provisioned
      workflows: [ ... ]      # ✅ provisioned
      dashboards: [ ... ]     # ✅ provisioned
      deduplicationRules:     # ✅ provisioned via env
      mappingRules:           # ✅ provisioned via env
      correlationRules: [ ... ]  # ❌ currently NOT provisioned

Today, declaring a rules: block under keep.backend.provision is silently dropped. Operators have to manually POST /rules after every cluster bootstrap — error-prone and breaks reproducibility.

Proposed implementation

Mirror the existing provision_deduplication_rules_from_env and provision_mapping_rules_from_env shapes:

• Add a new env var (e.g. KEEP_CORRELATION_RULES) that accepts a JSON array of rule definitions matching the POST /rules schema (ruleName, celQuery, sqlQuery, timeframeInSeconds, groupingCriteria, createOn, resolveOn, incidentNameTemplate, incidentPrefix, threshold, requireApprove).
• In provision_resources(), call a new provision_correlation_rules_from_env() after the existing provision_mapping_rules_from_env() call.
• The new function should be idempotent: on each startup, upsert by ruleName (or by a stable ID exposed in the env spec) so rolling restarts don't create duplicates.

In the Helm chart, expose the same correlationRules key shape as deduplicationRules (already templated in templates/backend-deployment.yaml).

Footgun to surface in docs

cel-python (the evaluator used for correlation rule celQuery) does NOT enable the regex matches() extension. Posting a rule with name.matches('^Foo') returns HTTP 200 but the next GET /rules surfaces Method 'matches' not implemented and the rule remains in a broken state. The same applies to workflow trigger CEL. Documenting this in the correlation-rules page (and ideally validating CEL at provision time) would prevent silent ingestion of broken rules.

Use case backing this request

We deploy Keep via Helm + ArgoCD on EKS. We want a single values.yaml to define:

• The workflow that opens Jira tickets on critical alerts (severity == "critical" && !name.startsWith("Data")).
• The correlation rule that groups the same set of critical alerts into incidents, using the same CEL filter (for parity).

Today the workflow lives in Git; the rule lives only in the DB and we have to maintain a runbook (POST /rules curl) so the CEL parity isn't broken by a Git-only change. Native provisioning closes this gap.

Related code

• keep/api/config.py:provision_resources — entry point that already covers the four other resource types.
• keep/rulesengine/rulesengine.py — RulesEngine.create_or_update_rule would be the natural upsert target.
• examples/workflows/incident_example.yml — workflow side, useful as a doc anchor for the parity story.