Skip to content

v0.51.0

Choose a tag to compare

@KIvanow KIvanow released this 16 Oct 16:15
· 110 commits to master since this release
0f15b8d

Until now, key analytics only ever looked at db0. This release scans every non-empty logical database, honors the configured dbIndex, and carries the database through the UI, API, Prometheus metrics, and dashboards, so a key's database is always visible. Alongside it, key-size data can now feed your own Prometheus/Grafana stack, and the monitor reports when a server starts refusing writes (MISCONF, OOM, READONLY, NOREPLICAS) and when it recovers.

Upgrades are drop-in. The new db column on the key analytics tables is lazy-migrated with a 0 backfill, so existing data stays valid. One behavior change to know about: a stored connection with a dbIndex is now actually connected to that database. If the server refuses the SELECT (index out of range, restricted ACL, pre-Valkey-9 cluster), the connection falls back to db0 with a prominent warning instead of failing. Saving a new connection or running Test Connection is strict and rejects a dbIndex the server won't accept.

Highlights

Key analytics across every logical database (#544)

If your keys lived outside db0, key analytics reported "No keys found in database" on every scan, and the dbIndex you configured was accepted but never applied. Both are fixed. The collector discovers every non-empty logical database via INFO keyspace and scans each one, through short-lived clients so the shared connection never switches databases; a failing database doesn't abort its siblings. Sampled scans share one sample budget across databases, split proportionally to each database's key count, so 16 populated databases don't cost 16× the SCAN work.

The database is now part of a key's identity everywhere. Pattern snapshots merge per (db, pattern); hot, largest, and composite keys carry their database, ranked per database and re-ranked globally for the all-databases view, so a quiet database isn't crowded out by a busy sibling and no database can hide another's patterns; rank deltas compare within the same database. The Key Analytics page gets a database selector (shown only when more than one database has data) and per-row db badges. Every endpoint accepts a db filter, and a new GET /key-analytics/databases lists the databases with stored analytics. Prometheus key-analytics gauges gain a db label so same-named keys in different databases no longer overwrite each other, and the Keyspace dashboard legends distinguish the series. Agent connections tag their results with the agent's --valkey-db.

Key size metrics in Prometheus and OTLP (#540)

Three new gauge families let you alert on keys ballooning in size from your own stack:

  • Per pattern - betterdb_key_pattern_{memory,avg_memory,max_memory}_bytes and betterdb_key_pattern_keys from the latest key analytics run, bounded to the top patterns by memory via METRICS_KEY_PATTERNS_TOP_N (default 100).
  • Per key - betterdb_key_memory_bytes and betterdb_key_cardinality for the largest tracked keys, bounded via METRICS_KEY_DETAILS_TOP_N (default 10, max 50; 0 disables, since raw key names become label values).
  • Keysize distribution - betterdb_keysize_bucket_keys from INFO keysizes (Redis 8.0+), per database × type × log2 bucket.

Series for connections without data are removed rather than zero-filled, analytics older than 3× the collector interval are dropped rather than served as current, and the OTLP mirror picks everything up. The Keyspace & Eviction dashboard gains three panels (memory by pattern, largest keys, keysize distribution), and the metrics reference documents the cardinality contract plus a worked key-memory-growth alert rule.

Two fixes rode along with this work:

  • Licensed instances could boot with key analytics silently disabled - the colt exactly once, inside the window where license validation was still resolving,and gave up permanently. It now waits for the verdict and checks per tick, so r (boot-time network failure, heartbeat re-validation) enables collectionwithout a restart.- Pre-multi-connection SQLite databases crashed at boot on upgrade - the conneschema creation referenced the column. It now runs first, and legacy tables arerebuilt to drop their stale connection-unscoped UNIQUE constraints, which werelog, and command-log rows across connections.Know when writes are being rejected (#527)The monitor now watches the persistence and errorstats INFO sections it alreadejected webhook event when the server starts refusing writes, naming the cause:failed BGSAVE or AOF write (MISCONF), maxmemory with no eviction (OOM), writes, or min-replicas-to-write unsatisfied (NOREPLICAS). A persistence failureopens the edge at warning before any client has been refused; once clients recritical with per-code counts. writes.recovered fires after a 60-second quietwindow, with memory genuinely below maxmemory for OOM edges (using Redis' own thmetic) and persistence reported healthy for BGSAVE/AOF edges. READONLY on anode reporting role:slave is expected and never alerts; counter drops from resfiring. The alertmanager examples gain a matching BetterDBPersistenceFailingrule.Reliability & correctness- Migration no longer loses stream consumer groups in command mode (#539) - grand the stream's ID counters are recreated on the target before the rename,empty streams that still hold groups are migrated, and validation compares groviously consumers hit NOGROUP after cutover while validation reported a match.Sorted sets now fail the key when a pipelined ZADD is rejected (e.g. OOM) inst into place.- Threshold webhook alerts are scoped per connection and per webhook (#526) - eshold no longer suppresses or triggers alerts for another, each webhook's ownthreshold configuration is honored, spike alerts are per connection too, and tnded LRU (10,000 entries by default, tunable via WEBHOOK_ALERT_STATE_MAX).
  • Scaling readiness reads are cheaper and more honest (#515) - the 7-day trendinutes per connection instead of on every compute, dimensions whose newest
    sample is over 10 minutes old are excluded instead of scoring stale data, and e just before leaving the page or switching connections is saved to the
    connection it was made for instead of being dropped or merged into the next co
  • Key analytics summary totals no longer undercount after a partial collectionhe latest snapshot per (pattern, db), so one database's failed scan can't
    silently drop a sibling's last row.

Security

  • Health endpoints no longer leak topology to anonymous callers - when workspaauthenticated /api/health learns only that the server is up; database host,version, config hazards, and license tier require a signed-in session, and the all-connections listing refuses anonymous callers outright.
  • Entitlement user and invitation operations are tenant-scoped (GHSA-2rwf-9ffpnd mutations 404 on a tenant mismatch exactly like a missing id, and an empty
    tenant fails closed instead of granting unscoped access.
  • Slack and Discord webhook payloads escape untrusted text (#531) - key names strings can no longer inject markup into notification messages.

Docker & deployment- A loopback DB_HOST set by the operator is honored (#518) - the image no longhost, so a loopback value you set survives instead of being rewritten tohost.docker.internal, which broke --network host setups. With no DB_HOST the are. Fixes #395.

  • Dockerfile.aiven is generated from the main Dockerfile (#517) - with a CI ch.

What's Changed

  • fix(webhooks): escape untrusted text in Slack and Discord payloads by @amitkojha05 in #531
  • fix(semantic-cache-py): default tracer scope to @betterdb/semantic-cache by @amitkojha05 in #524
  • chore(deps): bump @modelcontextprotocol/sdk from 1.27.1 to 1.31.0 by @dependabot[bot] in #521
  • chore(deps-dev): bump stream-json from 3.5.0 to 3.6.0 by @dependabot[bot] in #511
  • chore(deps): bump multidict from 6.7.1 to 6.9.1 in /packages/cache-benchmark by @dependabot[bot] in #510
  • Add stratified key-size sampler with validation harness by @jamby77 in #535
  • Tighten scaling readiness reads, staleness and settings saves by @jamby77 in #515
  • Generate Dockerfile.aiven from Dockerfile and guard drift in CI by @jamby77 in #517
  • Honor a loopback DB_HOST set by the operator by @jamby77 in #518
  • feat(prometheus): export key size metrics to Prometheus and OTLP by @KIvanow in #540
  • fix(webhooks): scope threshold-alert state per connection and honor per-webhook thresholds by @amitkojha05 in #526
  • chore(api): dedupe withTimeout into common/utils with direct spec by @Kathircpe in #513
  • feat(monitor): detect write rejections (MISCONF / OOM / READONLY / NOREPLICAS) with writes.rejected / writes.recovered events by @amitkojha05 in #527
  • fix(migration): keep stream consumer groups in command mode and fail partial sorted sets by @amitkojha05 in #539
  • feat(key-analytics): scan every non-empty logical database and honor dbIndex by @KIvanow in #544

Full Changelog: v0.50.0...v0.51.0