Skip to content

feat(proxy): query cache + Grafana API parity#2

Merged
digiserg merged 2 commits into
mainfrom
feat/query-cache-and-grafana-parity
Jul 9, 2026
Merged

feat(proxy): query cache + Grafana API parity#2
digiserg merged 2 commits into
mainfrom
feat/query-cache-and-grafana-parity

Conversation

@digiserg

@digiserg digiserg commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

Makes promclick-proxy a seamless drop-in for Prometheus behind Grafana, and adds an optional in-memory query-result cache. Scoped to the Grafana → Prometheus API → ClickHouse query path.

Query cache (new proxy/cache package)

  • LRU + per-entry TTL storing pre-rendered response bytes, keyed by (query, start, end, step).
  • singleflight collapses concurrent identical queries into a single evaluation (the big win under dashboard load — many panels/viewers issuing the same query).
  • Windows whose end is within cache.max_freshness of now are served but not stored (data still arriving).
  • Wires up the previously-dead cache.{enabled,max_size,ttl,max_freshness} config. Default disabled — behaviour unchanged unless enabled.
  • No new module deps: xxhash and golang.org/x/sync were already present (promoted indirect → direct).

Grafana-critical API parity

  • /api/v1/labels, /api/v1/label/{name}/values: honour match[] (plus best-effort start/end), so label_values(up, instance) scopes to the metric instead of returning every value in the TSDB. POST now accepted on label values, matching Prometheus.
  • /api/v1/series: parses each match[] with the Prometheus parser, applies all label matchers, supports multiple selectors, dedupes by fingerprint, uses the label cache when a concrete metric name is present. Its ClickHouse fallback now decodes the JSON-string labels column correctly (the old hand-rolled parser silently returned empty label sets for String label columns).
  • /api/v1/metadata: populated (one unknown-typed entry per metric), honours metric/limit.

Security

All matcher values rendered into ClickHouse SQL go through chEscape (equality, inequality, regex), keeping the match[]-driven paths free of SQL injection.

Tests

  • New proxy/cache/cache_test.go: LRU eviction, TTL expiry, singleflight de-dup, key stability, no-store path.
  • New proxy/server/handlers/parity_test.go: first HTTP handler-level tests — series matcher enforcement, scoped label_values, scoped labels, populated metadata, WHERE-builder + escaping unit tests.
  • go build/go vet/go test ./... green in both modules; helm lint passes.

Docs

CHANGELOG, README (cache: block + TODO), deploy/proxy.yaml, and the Helm _helpers.tpl default config updated. Documented cache defaults match proxy/config/proxy.go.

Out of scope / notes

  • start/end on labels/series are accepted but not used to prune series (the label cache has no time dimension); match[] scoping is the correctness fix Grafana needs.
  • Remote read, exemplars, rules, subqueries, @ modifier, scalar/string result types are intentionally out of scope (per plan).
  • Pre-existing lint findings left untouched (scope discipline): two staticcheck U1000 dead symbols (query.go:604, :1059, from upstream 6fc3ca1) and errcheck on bufio intermediate writes in encodeMatrix (relocated verbatim from the original writeMatrixResponse; errors surface at the checked Flush()).

Verify live

docker compose up -d, then compare PromClick vs the reference Prometheus datasource in Grafana (:3000):

  • Cache: same query_range twice → path=cache-hit, byte-identical; 20 concurrent identical → one cache-miss.
  • Parity: label/instance/values?match[]=up returns only up's instances.

Assisted-by: Claude Code

digiserg added 2 commits July 8, 2026 22:24
Add an optional in-memory query-result cache to promclick-proxy and close
the Grafana-critical Prometheus API compatibility gaps so PromClick can act
as a seamless drop-in replacement for Prometheus behind Grafana.

Query cache (new proxy/cache package): an LRU with per-entry TTL storing
pre-rendered response bytes, keyed by (query, start, end, step), with a
singleflight group collapsing concurrent identical queries into one
evaluation. Windows whose end is within cache.max_freshness of now are served
but not stored, since that data is still being written. Wires up the existing
but previously dead cache.{enabled,max_size,ttl,max_freshness} config; default
disabled, so behaviour is unchanged unless explicitly enabled.

API parity:
- /api/v1/labels and /api/v1/label/{name}/values now honour match[] (plus
  best-effort start/end), so template variables like label_values(up, instance)
  are scoped to the selected metric instead of returning every value in the
  TSDB. POST is now accepted on label values, matching Prometheus.
- /api/v1/series parses each match[] with the Prometheus parser, applies all
  label matchers, supports multiple selectors, dedupes by fingerprint, and uses
  the label cache when a concrete metric name is present. Its ClickHouse
  fallback now decodes the JSON-string labels column correctly.
- /api/v1/metadata is populated (one unknown-typed entry per metric) and
  honours metric/limit.

All matcher values rendered into ClickHouse SQL go through chEscape, keeping
the match[]-driven paths free of SQL injection. Adds cache unit tests and the
first HTTP handler-level tests.

Signed-off-by: Sergio Rua <sergio.rua@digitalis.io>
Extend the Helm chart release workflow to trigger on v* tags in
addition to main pushes. On a tag it packages every chart, versions
them to the tag, pushes to GHCR, and cuts a GitHub Release with the
chart tarballs attached via the gh CLI. Branch pushes keep the prior
changed-charts-only behaviour.

Signed-off-by: Sergio Rua <sergio.rua@digitalis.io>

@rgooding rgooding left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved but I'm skeptical about the benefits of an in-memory cache of an exact PromQL query, especially in a distributed system.

@digiserg digiserg merged commit fe69824 into main Jul 9, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants