Skip to content

Releases: BetterDB-inc/monitor

v0.38.0

Choose a tag to compare

@KIvanow KIvanow released this 14 Aug 18:20
e080c38

Focus of this release: smarter anomaly detection. Four new detectors/advisories drawn from real Valkey failure modes, one detection accuracy fix, and a build-system cleanup.

✨ New — Anomaly detection & advisories

  • Replica full-resync failure loop (#373, valkey#1836) — Flags a replica that keeps failing full sync and reconnecting in a loop, instead of quietly churning. Surfaces the stuck replica before it burns bandwidth and delays replication indefinitely.
  • Orphaned keys in unowned cluster slots (#376, valkey#539) — Detects keys living in hash slots the node no longer owns (e.g. after a resharding that didn't fully migrate), which are otherwise invisible and unreachable through normal cluster routing.
  • appendfsync=always blocking advisory (#375, valkey#3515) — Raises a config hazard when appendfsync=always is set, since it fsyncs on the main thread for every write and can severely cap throughput. Called out so operators can make the durability-vs-latency tradeoff deliberately.
  • Config-drift now covers encoding thresholds (#374, valkey#3479) — Extends config-drift detection to encoding-threshold settings (e.g. *-max-listpack-* / *-max-intset-*), catching silent memory/CPU regressions from drifted encoding limits.

🐛 Fixed

  • Memory-overhead detector no longer false-alarms on underflowed samples (#372) — Underflowed memory readings are now skipped rather than treated as real overhead, removing spurious anomalies.

🔧 Changed / Internal

  • Single multi-target Dockerfile (#379) — Consolidated Dockerfile.prod into one multi-target Dockerfile, simplifying the build matrix and the publish workflow. No change to published image behavior.

Full changelog: v0.37.0...v0.38.0

What's Changed

  • fix(anomaly): skip underflowed memory samples in memory-overhead detector by @jamby77 in #372
  • feat(anomaly): detect replica full-resync failure loop (valkey#1836) by @jamby77 in #373
  • feat(anomaly): extend config-drift to encoding-threshold configs (valkey#3479) by @jamby77 in #374
  • feat(monitor): appendfsync=always main-thread blocking advisory (valkey#3515) by @jamby77 in #375
  • feat(anomaly): detect orphaned keys in unowned cluster slots (valkey#539) by @jamby77 in #376
  • refactor(docker): merge into a single multi-target Dockerfile by @KIvanow in #379

Full Changelog: v0.37.0...v0.38.0

v0.37.0

Choose a tag to compare

@KIvanow KIvanow released this 12 Aug 14:46
bef08ef

v0.37.0

Container hardening release: the Docker image is rebuilt clean and roughly half the size. The migration binary no longer drags an EOL Go runtime's CVEs into your cluster, the base image and bundled tooling are patched or removed, and the image scanner goes from 138 findings (4 critical, 52 high) to 11 (0 critical, 0 high).

No breaking changes; no config or migration steps required.

🔒 Container security

  • RedisShake migration binary rebuilt from source (Go 1.26). The upstream prebuilt release is compiled with Go 1.21.13 — a standard library that went end-of-life two years ago and carries dozens of CVEs, including criticals (CVE-2025-64121, CVE-2025-22871). These are welded into a statically-linked binary, so apk upgrade can't touch them. We now build RedisShake from source (pinned to the v4.6.1 release commit) against a current toolchain, and bump its embedded golang.org/x/text 0.14.0 → 0.39.0 (CVE-2026-56852).
  • Base image node:25-alpinenode:26-alpine. Node 25 is a non-LTS line with no patch for CVE-2026-56848 / CVE-2026-58043; node 26 (LTS) carries the fixes.
  • Removed the npm CLI from the runtime image. The image builds with pnpm; the bundled npm was unused at runtime and only contributed its own vendored-dependency CVEs (tar, brace-expansion, picomatch, sigstore, ip-address).
  • Removed the GNU tar package. It shipped currently-unfixable Alpine CVEs and was redundant — busybox already provides tar.
  • Net result: image scan goes from 138 findings → 11, and 4 critical + 52 high → 0 + 0. The 11 that remain are Medium/Low items with no upstream fix yet (busybox/ssl_client and a couple of suite-locked transitive deps) — left in the open rather than hidden.

📦 Image size — roughly halved

  • A single chown -R /app was duplicating the entire ~600 MB node_modules into a second overlay layer just to change ownership bits. Switched to COPY --chown (ownership set as files are written — one layer, not two), plus COPY --chmod for the migration binary.
    • Default (no-AI) image: 1.34 GB → 744 MB.
    • AI image: 2.22 GB → 1.17 GB.
  • Every pull moves less data, every deploy is faster, every node caches more.

🛠️ Fixes & build

  • Health check corrected. It now probes /api/health (the previous path hit the SPA fallback and reported healthy regardless of API state) using the base image's busybox wget — no extra package, and no interpreter cold-start that could exceed the 3 s timeout under the CPU pressure of a running migration.
  • Restored /app write access for the non-root runtime user. The layer-dedup change had left /app root-owned, which broke features that write there at runtime — notably RedisShake's working directory during a migration, and SQLite/license paths. Fixed non-recursively (no size regression).
  • arm64 image now cross-compiles natively instead of running the Go toolchain under QEMU emulation (--platform=$BUILDPLATFORM) — faster, more reliable multi-arch builds.
  • The shipped redis-shake binary now reports its real version (v4.6.1 + commit) instead of unknown in migration logs.

Thanks to @jamby77 for the thorough review that tightened the build (native cross-compile, busybox health check, version stamping, single-layer binary).

v0.36.0

Choose a tag to compare

@KIvanow KIvanow released this 07 Aug 12:23
5cb2625

Security release: every open critical, high, and medium Dependabot alert resolved - 62 alerts closed (60 fixed, 2 dismissed with documented justification), spanning 53 distinct security advisories across 19 dependencies, verified end-to-end on both the Docker image and the npx package.

BetterDB Monitor now ships with a fully clean security scorecard: zero open critical, high, or medium vulnerability alerts. Overall - 62 alerts fixed covering 53 distinct security advisories, tested every fix individually.

This update also makes the Docker container health check report correctly and comes with a refreshed README. No breaking changes - updating is safe and recommended for all users. Full technical details below.

🔒 Security

Critical

  • seroval ≤ 1.5.2 → 1.5.6 (CVE-2026-59940) - type confusion in fromJSON() let attacker-controlled JSON invoke unintended methods during Promise-resolver deserialization - potential RCE when untrusted Seroval JSON is deserialized with plugins enabled. (#364)

High

  • form-data → 4.0.6 (CVE-2026-12143) - CRLF injection via unescaped multipart field names/filenames.
  • brace-expansion → 1.1.18 / 2.1.4 / 5.0.9 (CVE-2026-14257, CVE-2026-69152, CVE-2026-13149) - three DoS vectors: unbounded expansion (OOM), unbounded intermediate arrays (bypass of the first fix), and exponential-time {} group expansion.
  • ip-address → 10.4.0 (CVE-2026-69192) - leading-zero octets decoded as decimal while resolvers decode octal, letting SSRF filters approve internal targets.
  • fast-uri → 3.1.5 (CVE-2026-18446, CVE-2026-16221, CVE-2026-13676) - three host-confusion parses (backslash authority introducer/delimiter, failed IDN canonicalization) enabling allowlist bypass.
  • postcss → 8.5.26 (CVE-2026-45623, GHSA-r28c-9q8g-f849) - arbitrary .map file disclosure via attacker-controlled sourceMappingURL, including the incomplete-fix follow-up.
  • shell-quote → 1.10.0 (CVE-2026-13311) - quadratic-complexity DoS in parse().
  • js-yaml → 3.15.1 / 4.3.1 (CVE-2026-59869) - YAML merge-key alias chains forcing quadratic CPU consumption.
  • axios → 1.18.0 (GHSA-gcfj-64vw-6mp9) - Node HTTP adapter could route requests through an inherited (prototype-polluted) proxy after interceptor config cloning; also picks up the formToJSON/maxBodyLength/NO_PROXY hardening set.
  • adm-zip → 0.6.0 (CVE-2026-39244) - crafted ZIP header triggers a 4 GB allocation (DoS).
  • find-my-way → 9.7.0 (CVE-2026-47219) - DDoS via HTTP/2 against the fastify router.
  • @fastify/static → 10.1.2 everywhere (CVE-2026-15074, CVE-2026-7120) - route-guard bypass via path traversal; the vulnerable 9.3.0 copy pulled by @fastify/swagger-ui is deduped away.
  • sharp → 0.35.3 (GHSA-f88m-g3jw-g9cj) - inherited libvips CVE-2026-33327/33328/35590/35591, exploitable via malicious image input.
  • react-router → 7.18.2 (CVE-2026-55685, plus mediums CVE-2026-53666/53667/53669) - unauthenticated DoS via inefficient route matching, open redirect, RSC error-handler XSS, and constructor injection.

Medium

  • hono → 4.13.1 (CVE-2026-69207, CVE-2026-59895, CVE-2026-59896, CVE-2026-59897) - CORS-middleware ReDoS, JSX cx() XSS escape bypass, cross-request jsx context disclosure, and repeated-header loss in the API Gateway adapter. (#366)
  • dompurify → 3.4.13 (CVE-2026-49978, CVE-2026-49458, CVE-2026-49459, CVE-2026-65902, CVE-2026-65898) - five sanitization-bypass / allowlist-pollution flaws; also closes three low-severity dompurify alerts.
  • valibot → 1.4.2 (CVE-2026-59952) - record() issue paths could make flatten() throw on inherited Object property names (DoS on untrusted input).
  • @hono/node-server → 2.1.0 (GHSA-frvp-7c67-39w9) - path traversal in serve-static on Windows via encoded backslash; 1.x → 2.x major keeps the hono ^4 peer range.
  • @opentelemetry/core → 2.9.0 (CVE-2026-54285) - unbounded memory allocation in W3C Baggage propagation; all 1.30.1/2.2.0/2.6.1 copies deduped to a single patched resolution.

Dismissed with justification (not exploitable here)

  • react-router RSC-mode CSRF (GHSA-qwww-vcr4-c8h2) - patched only in react-router 8.3.0, which requires React ≥ 19.2.7 / Node ≥ 22.22.0 (outside the supported matrix) and has no matching react-router-dom release. The vulnerable RSC server-action path is unreachable in the client-side Vite SPA. Revisit when react-router-dom ships an 8.x line.
  • file-type ASF-parser infinite loop (CVE-2026-31808) - the vulnerable 16.5.4 exists only inside ibm-cloud-sdk-core (unused watsonx path). The patched 21.x line is ESM-only and verifiably breaks the SDK's CJS FileType.fromBuffer call. Revisit if ibm-cloud-sdk-core updates its file-type dependency.

🛠️ Internal

  • packages/agent-memory - span tests moved from @opentelemetry/sdk-trace-base 1.x to ^2.9.0 to match the core 2.x security override (the 1.x SDK imported the removed getEnv()); removes the last OpenTelemetry 1.x entries from the lockfile. (#366)

🐛 Bug fixes

  • Docker HEALTHCHECK reported unhealthy on healthy containers - the Dockerfile healthcheck probed /health, which production builds only serve as the SPA fallback for GET; wget's spider request got a non-2xx and the container sat permanently unhealthy. Both Dockerfile and Dockerfile.prod now probe the real /api/health endpoint. (#365)

📚 Docs

  • README refreshed (product-first restructure, badges, new hero screenshot); production examples now use the /api-prefixed endpoint paths that production builds actually serve; the intentional Docker tag scheme is documented - latest deliberately tracks the no-ai build, while the experimental AI Helper ships only in the versioned full image. (#365)

⬆️ Upgrade notes

  • Fully backwards compatible - all security fixes are dependency version bumps via pnpm.overrides; no API, schema, or config changes. Both distribution channels were verified end-to-end against a live Valkey 9.1.0 instance (health, metrics/INFO parsing, web UI) on this exact dependency set.
  • Worth a glance if you embed the web app: react-router moves 7.17.0 → 7.18.2 (same major) and @hono/node-server moves 1.x → 2.x inside the MCP server stack (peer range unchanged, suites pass).
  • Docker: betterdb/monitor:0.36.0

What's Changed

  • fix(deps): resolve all critical and high Dependabot security alerts in #364
  • fix(deps): resolve all medium-severity Dependabot security alerts in #366
  • docs: refresh README and fix production endpoint paths in #365

Full Changelog: v0.35.0...v0.36.0

What's Changed

  • fix(deps): resolve all critical and high Dependabot security alerts by @KIvanow in #364
  • fix(deps): resolve all medium-severity Dependabot security alerts by @KIvanow in #366
  • docs: refresh README and fix production endpoint paths by @KIvanow in #365

Full Changelog: v0.35.0...v0.36.0

v0.35.0

Choose a tag to compare

@KIvanow KIvanow released this 06 Aug 13:15
bc65c11

Five new cluster/topology anomaly detectors for engine hazards that can't be fixed upstream, plus a real bug fix for a health metric that has been silently wrong since day one.

This release adds five advisory anomaly detectors targeting known-but-unfixable Valkey engine hazards (config drift across a replication group, stale hostname gossip, uncoordinated lagging promotion, ghost cluster membership, and large-reply COMMANDLOG throughput regression), fixes keyspaceSize always reporting 0 in the health summary and MCP get_health tool, and improves the update-check UX with install-aware upgrade commands.

🐛 Bug fixes

  • Health summary keyspaceSize always 0 - MetricsParser.parseInfoToTyped was an identity cast, so INFO keyspace/commandstats/errorstats sections stayed raw "k=v,k=v" strings at runtime even though their types promised parsed objects. getHealthSummary only counted a db entry when it was a typed object, so keyspaceSize structurally could never be anything but 0 - on every instance, in every deployment. This is the first field the MCP get_health tool description recommends checking, so it read as "empty database" when it never was. Now genuinely parsed; keyspaceSize is null when the keyspace section is absent so "no data" stays distinguishable from "empty database" (0). (#360)
  • Create-instance form 500s on names with spaces - the Add Connection → BetterDB Valkey instance form crashed provisioning with an opaque 500 if the name contained a space. The Name field now pre-fills with my-cache, sanitizes spaces to hyphens live as you type, and normalizes to a DNS-style label on submit; whitespace-only names show an inline error instead of firing a request.
  • Valkey provisioning failed for 1GB/2GB tiers - the tenant ResourceQuota hardcoded limits.memory: 2Gi, but the chart renders pods at 2× maxmemory, so the pod + sidecar exceeded quota and provisioning timed out with FailedCreate: exceeded quota. Since 1GB is the pre-selected first-run tier, this broke first-instance creation for new users. Quota now sizes off the same valkeyMemoryLimitMi() helper the chart uses.

✨ New anomaly detectors

  • Cross-node config drift (CONFIG_DRIFT) - CONFIG SET only ever applies to the node it's sent to, so nodes in the same replication group (cluster shard, or primary+replicas) can silently drift on critical settings (eviction policy, persistence, etc.) - a real source of incidents. Records a curated config subset per replication-group key and alerts on divergence. (upstream valkey#1193)
  • Hostname staleness / endpoint inconsistency (HOSTNAME_STALENESS) - in cluster mode, hostname gossip converges eventually rather than atomically, so CLUSTER NODES and CLUSTER SHARDS can disagree about a node's hostname, or a node can advertise a bare IP while peers already see a hostname. Either state can break TLS-SNI verification. Upstream has indicated this is likely won't-fix, so this is now a durable, engine-independent advisory. (upstream valkey#304)
  • Lagging uncoordinated promotion (LAGGING_PROMOTION) - in standalone (cluster-mode-disabled) setups, REPLICAOF NO ONE promotes a replica to primary with zero coordination. If the promoted replica was lagging, its unreplicated writes are lost and a more up-to-date sibling is forced into a full resync. Detects the transition and compares replication offsets against co-replica siblings. (upstream valkey#2587)
  • Ghost cluster membership (GHOST_MEMBERSHIP) - CLUSTER RESET or a restart makes a node forget its peers, but peers never forget it: the old node-id lingers as a ghost (fail/fail?/noaddr) next to the live id on the same ip:port. Surfaces the ghost and advises CLUSTER FORGET <ghost-id>. (upstream valkey#1757)
  • Large-reply COMMANDLOG throughput regression (LARGE_REPLY_PRESSURE) - Valkey's COMMANDLOG LARGE-REPLY facility, once a hot command's reply crosses commandlog-reply-larger-than, repeatedly takes a large-reply logging/copy path observed to cost up to ~25% GET throughput. The upstream fix is still open (a prior PR only raised the default threshold, masking rather than fixing it), so this adds a threshold-crossing advisory in the meantime. (upstream valkey#2926)

🖥️ Web / UX

  • Install-aware update banner - the update banner now detects how the instance was launched (Docker/Podman/npm/pnpm/yarn/npx) server-side and shows the matching one-click copy upgrade command, plus a link to a new step-by-step docs/updating.md upgrade guide.

📡 Telemetry / Licensing

  • telemetryEnabled sent on live license checks - license entitlement checks now report whether telemetry is enabled, alongside the existing check payload.

📚 Docs

  • Documented 5 previously-missing betterdb_* Prometheus metrics.

⬆️ Upgrade notes

  • Fully backwards compatible for the detectors and web changes - the five new detectors are additive and emit over the existing webhook, OTLP, and dashboard channels automatically; no config or schema migrations required.
  • API shape change (visible to external API consumers only): GET /metrics/info and GET /mcp/instance/:id/info now serialize keyspace, commandstats, and errorstats as parsed objects instead of raw "k=v,k=v" strings; keyspaceSize in the health payload is now number | null instead of always number. No in-repo consumer (web app, MCP tools, Prometheus exporter) relied on the old string shape.
  • Docker: betterdb/monitor:0.35.0

What's Changed

  • fix(metrics): parse keyspace/commandstats/errorstats in parseInfoToTyped in #362
  • feat(anomaly): ghost-membership detector for CLUSTER RESET stale ids (#1757) in #352
  • feat(anomaly): detect lagging uncoordinated promotion (REPLICAOF NO ONE) (#2587) in #356
  • feat(anomaly): hostname-staleness / endpoint-inconsistency detector (#304) in #357
  • feat(anomaly): cross-node config-drift detector (#1193) in #358
  • feat(anomaly): large-reply commandlog pressure advisory (#2926) in #359
  • feat(web): install-aware update banner with copy-paste upgrade command in #361
  • fix(web): pre-fill Valkey instance name and stop spaces from 500ing create in #355
  • fix(entitlement): size tenant ResourceQuota to the Valkey pod in #353
  • fix: Include telemetryEnabled on license checks in #363
  • docs(prometheus): document 5 missing betterdb_ metrics

Full Changelog: v0.34.0...v0.35.0

What's Changed

  • fix(entitlement): size tenant ResourceQuota to the Valkey pod by @KIvanow in #353
  • chore(deps): bump aiohttp from 3.14.1 to 3.14.3 in /packages/cache-benchmark by @dependabot[bot] in #354
  • feat(anomaly): ghost-membership detector for CLUSTER RESET stale ids (#1757) by @KIvanow in #352
  • fix(web): pre-fill Valkey instance name and stop spaces from 500ing create by @KIvanow in #355
  • feat(anomaly): detect lagging uncoordinated promotion (REPLICAOF NO ONE) (#2587) by @KIvanow in #356
  • feat(anomaly): hostname-staleness / endpoint-inconsistency detector (#304) by @KIvanow in #357
  • feat(anomaly): cross-node config-drift detector (#1193) by @KIvanow in #358
  • feat(anomaly): large-reply commandlog pressure advisory (#2926) by @KIvanow in #359
  • feat(web): install-aware update banner with copy-paste upgrade command by @KIvanow in #361
  • Include telemetryEnabled on license checks by @KIvanow in #363
  • fix(metrics): parse keyspace/commandstats/errorstats in parseInfoToTyped by @KIvanow in #362

Full Changelog: 0.34.0...v0.35.0

v0.34.0

Choose a tag to compare

@KIvanow KIvanow released this 31 Jul 07:48
5a94587

Three new memory & load anomaly detectors, plus stable telemetry instance identity.

This release adds three detectors targeting silent memory-budget and single-thread-saturation conditions - non-dataset overhead squeezing the maxmemory budget, event-loop saturation that raw CPU% hides, and copy-on-write OOM risk during forked saves - and fixes anonymous-telemetry instance counting so ephemeral containers stop looking like new installs.

✨ New anomaly detectors

  • Non-dataset memory overhead (MEMORY_OVERHEAD) - a maxmemory budget is meant to bound the dataset, but operational overhead (client output buffers, the replication backlog and per-replica buffers, the AOF rewrite buffer, scripts/functions, cluster-bus links) is charged against the same budget. When it grows it silently shrinks the room for user data and, under an eviction policy, drives eviction of keys that would otherwise fit. WARNING at ≥30% of maxmemory, CRITICAL at ≥50% - or a CRITICAL when eviction is active and overhead has grown larger than the dataset itself, so a normal capacity-bound cache evicting by design is not mistaken for a problem. Names the dominant overhead component and gives a targeted remedy (e.g. tune the relevant client-output-buffer-limit, lower repl-backlog-size); escalation-only hysteresis. (upstream valkey#1792)

  • Event-loop load saturation (LOAD_SATURATION) - on a largely single-threaded server, raw CPU% is a poor busyness indicator. This measures the fraction of wall-clock time the event loop actually spends working (instantaneous_eventloop_duration_usec × instantaneous_eventloop_cycles_per_sec). WARNING at ≥80% busy, CRITICAL at ≥95%, each requiring the band to hold for 3 consecutive polls so a momentary batch/pipeline burst doesn't alert. When CPU% reads misleadingly low next to the busy fraction, the alert calls that out explicitly and points at slow commands / big O(N) operations / a single-thread bottleneck rather than "add more CPU". Graceful no-op on servers that don't expose the event-loop metrics. (upstream valkey#2055)

  • Fork / copy-on-write OOM risk (FORK_MEMORY_RISK) - BGSAVE and AOF rewrite fork the server; writes arriving while the child runs dirty copy-on-write pages, so RSS climbs toward used_memory + bytes-written-during-save and can cross available RAM, letting the OOM killer terminate the server mid-save. Two paths: live (a save in progress, projecting used_memory_rss + current COW against total system memory) and projected (no save running, but a sustained write rate makes the next fork risky - gated on the per-poll write rate so it clears when writes stop, and estimated from the larger of the last RDB or AOF COW so AOF-only deployments are covered). WARNING at ≥80% of system memory, CRITICAL at ≥90% (live), with a slow-fork note from latest_fork_usec and advice to verify vm.overcommit_memory and RAM headroom. (upstream valkey#3609)

📡 Telemetry

  • Stable anonymous instance identity + ephemerality signals - the anonymous instance id was re-derived every boot from a hash including HOSTNAME, so ephemeral containers (random per-run hostname) each looked like a brand-new install and inflated distinct-instance counts. The id is now persisted under data/instance-id and reused across restarts, with a fallback to per-boot derivation when the data dir isn't writable, and a random UUID (instead of a shared constant hash) when no infrastructure identifiers are set. The ping also carries ci, container, and hostnameIsContainerId so throwaway boots (CI, e2e spin-ups, restart loops) can be segmented from real installs. (#350)

⬆️ Upgrade notes

  • Fully backwards compatible - no config or schema migrations. The three detectors are additive and emit over the existing webhook, OTLP, and dashboard channels automatically; they add three metric-type labels (memory_overhead, load_saturation, fork_memory_risk).
  • FORK_MEMORY_RISK compares projected RSS against total_system_memory, so it is most meaningful on memory-limited instances (a container/cgroup memory limit, where Valkey reports that limit). On an unconstrained host that reports full system RAM it will rarely fire.
  • Telemetry now writes an anonymous id to data/instance-id when the data directory is writable (still governed by BETTERDB_TELEMETRY; no new data is collected beyond the segmentation flags above).
  • Docker: betterdb/monitor:0.34.0

What's Changed

  • feat(telemetry): persist instance id and add ephemerality signals by @KIvanow in #350
  • feat(anomaly): memory-overhead, load-saturation and fork-OOM detectors by @KIvanow in #351

Full Changelog: v0.33.0...v0.34.0

v0.33.0

Choose a tag to compare

@KIvanow KIvanow released this 30 Jul 07:37
a5468a0

Five new anomaly detectors, composite hot/big-key ranking, richer OTLP telemetry, and an ACL false-negative fix.

This release is heavy on detection. It adds five anomaly detectors targeting silent, non-self-healing cluster and client-buffer conditions, a new diagnostic that surfaces "hot and big" keys, a SCAN hash-skew advisory, and completes the OTLP mirroring work so OTel-only deployments get full telemetry.

✨ New anomaly detectors

  • Client-eviction storms (EVICTED_CLIENTS) - warns when Valkey/Redis is evicting clients to reclaim client-buffer memory (maxmemory-clients). These disconnects were previously silent. Feeds the per-poll delta of the lifetime evicted_clients counter to a spike detector, absorbs counter resets on restart, and skips entirely when eviction is disabled (maxmemory-clients=0). The alert surfaces the limit and post-eviction client-buffer memory, with an explicit caveat that a single snapshot can't separate a justified eviction from an over-aggressive one - pointing the operator to the recurrence pattern instead. (upstream valkey#4151)

  • Replicas stuck in slot migrating/importing state - after a reshard, some replicas silently keep reporting slots as migrating/importing (or wrongly owning slots) in CLUSTER NODES; the only fix is an operator running CLUSTER SETSLOT <slot> STABLE. Two-layer detection (node-local markers, refined by a CLUSTER SHARDS role-authority cross-view to suppress mid-promotion false positives), fanned out per-node so a replica polled only through the primary is still seen. Auto-resolves on recovery. (upstream valkey#1664)

  • Gossip-mode failover churn - detects a single shard re-electing repeatedly within a 60s window (3+ epoch bumps or owner flips), the signature of competing FAILOVER coordinators. WARNING, escalating to CRITICAL across a second window, with a deterministic equal-epoch tiebreak and resharding suppression. (upstream valkey#3996)

  • Replication output-buffer (COB) pressure - per-replica client-output-buffer ratio alerts with escalation-only hysteresis (60% warn / 90% critical of the slave hard limit), soft-limit sustained early warning, and a mem_clients_slaves aggregate fallback. Includes resync-loop detection and per-replica Prometheus buffer-pressure gauges. (upstream valkey#3963)

  • Control-plane saturation (correlated) - flags a sustained ≥90% CPU streak paired with control-plane impact evidence: probe-RTT spike vs rolling baseline, graded replica drops, or recent control-plane anomalies (including failover churn). Emits one synthetic CRITICAL CPU event per episode, with restart-safe streak/baseline resets. (upstream valkey#3927)

📊 New diagnostics

  • Composite multi-dimensional hot/big-key detection - the existing lists rank each dimension alone; this surfaces the key that is extreme on more than one at once - the "hot big key" (a large collection that's also hammered, or a hot key whose value quietly grew huge). Pure post-processing over the data already gathered in one scan (no extra round-trips), ranking on the two globally-retained signals (hotness + cardinality), with idle-recency fallback for hotness on non-LFU policies. New GET /key-analytics/composite-keys endpoint. (upstream valkey#4189)

  • SCAN large-reply hash-skew advisory - a pure analyzer over stored large-reply entries that flags SCAN-family commands returning disproportionate bytes-per-requested-element (a sign of hash skew / degenerate chains), with worst-first ranking and per-offender remediation copy that distinguishes keyed-scan vs keyspace guidance. Surfaced on the SlowLog large-reply tab; hidden when there are no offenders. New GET /commandlog-analytics/scan-skew endpoint. (upstream valkey#3955)

📡 OpenTelemetry

  • Unit metadata + webhook-independent OTLP mirroring - the OTLP exporter now derives UCUM units from metric-name suffixes (_bytes → By, _seconds → s, _ratio → 1, _percent → %, …) and carries them as OTLP metadata without rewriting metric names. Anomaly dispatch, cluster.failover, and instance.down/instance.up availability edges are decoupled from the Pro webhook gate, so an OTLP-only deployment (collector configured, no webhook) now receives complete telemetry. (upstream valkey#4078)

🐛 Fixes

  • ACL: explicit command denials no longer read as an unrestricted grant. A default user with +@all followed by command denials (-@transaction, -exec, -multi) can still lose writes on AOF reload. The evaluator no longer treats such grants as the safe workaround - any deny token in the effective command rules now defeats the unrestricted claim (deny-wins, matching the ACL checker). (related to upstream valkey#3983)

🔧 Maintenance

  • Bumped esbuild (0.27.4 → 0.28.1) and tsx (4.21.0 → 4.23.1) in the agent-cache Anthropic example.

⬆ Upgrade notes

  • Fully backwards compatible - no config or schema migrations. New detectors and endpoints are additive; the composite key-analytics signal reuses existing hot_key_stats columns (no migration).
  • New anomaly detectors emit over the existing webhook + OTLP channels automatically. OTLP remains opt-in via OTEL_* and is now independent of any webhook configuration.
  • Docker: betterdb/monitor:0.33.0

What's Changed

  • feat(anomaly): detect client-eviction storms (maxmemory-clients) by @KIvanow in #334
  • feat(otel): unit metadata + decouple OTLP event mirroring from the webhook by @KIvanow in #335
  • feat(key-analytics): composite multi-dimensional hot/big-key detection by @KIvanow in #336
  • feat(anomaly): detect replicas stuck in slot migrating/importing state (valkey#1664) by @KIvanow in #344
  • chore(deps): bump esbuild and tsx in /packages/agent-cache/examples/anthropic by @dependabot[bot] in #343
  • fix(monitor): explicit ACL denials defeat the +@ALL workaround grant by @jamby77 in #345
  • feat(anomaly): gossip-mode failover-churn detector (valkey#3996) by @jamby77 in #346
  • feat(commandlog): SCAN large-reply hash-skew advisory (valkey#3955) by @jamby77 in #338
  • feat(anomaly): replication output-buffer pressure detector (valkey#3963) by @jamby77 in #347
  • feat(anomaly): control-plane saturation correlated detector (valkey#3927) by @jamby77 in #348

Full Changelog: v0.32.0...v0.33.0

v0.32.0

Choose a tag to compare

@KIvanow KIvanow released this 28 Jul 09:20
892f05b

Highlights

🛡️ Config-hazard advisory (#337)

Monitor now proactively detects dangerous server configurations and surfaces them before they cause silent data loss.

  • New Dashboard banner (ConfigHazardBanner) warns when a connected instance is in a hazardous config state, including the default-user-enabled + AOF silent data-loss hazard (valkey#3983).
  • Distinct "could not verify" state when Monitor lacks the permissions (NOPERM) or version support to evaluate the hazard, so an unknown isn't shown as a clean bill of health.
  • Hazards are exposed on /health/detailed (configHazards) and on MCP instance health, and are TTL-cached per connection server-side so the check is cheap and polled slowly.

📊 New observability API surface

Backend endpoints added across the API and proprietary modules, powering both the dashboards and the new MCP 1.4.0 tools:

  • AI observability (#330) - AI component instance discovery + stored-metrics history, and OpenTelemetry trace endpoints (list, span waterfall, and correlation of trace spans with live Valkey state).
  • Analytics (#331) - metric capacity forecast endpoint (ops/sec, used memory, CPU, fragmentation to time-to-ceiling), latency-regression queries over the anomaly event store, and a license-gated largest-keys endpoint (ranked by measured memory).
  • Vector search (#332) - per-index vector index health (doc count, memory, indexing failures, % indexed) and FT.SEARCH inference latency (p50/p95/p99 per index) with optional SLA breach status.

Web

  • Dashboard: config-hazard banner wired to detailed health.
  • Key Analytics: largest-keys tab now backed by memory-ranked snapshots; minor table/rank-delta cleanup.

Notes

  • Ships alongside @betterdb/mcp 1.4.0, which exposes the above as 10 new MCP tools.

What's Changed

  • feat(mcp): AI observability MCP tools by @jamby77 in #330
  • feat(mcp): analytics MCP tools by @jamby77 in #331
  • feat(mcp): vector search MCP tools by @jamby77 in #332
  • feat(monitor): config-hazard advisory — default user disabled + AOF silent data loss (valkey#3983) by @jamby77 in #337

Full Changelog: v0.31.0...v0.32.0

MCP v1.4.0

Choose a tag to compare

@github-actions github-actions released this 28 Jul 09:21
892f05b

Added

  • 5 AI observability tools for AI components running on the connected
    instance and traces ingested via OpenTelemetry (#330):
    • ai_list_instances — superset discovery view of all AI component
      instances (semantic caches, agent caches, agent-memory stores, retrieval
      pipelines) with liveness and the latest stored metrics sample.
    • ai_instance_history — stored metrics time-series for one AI component
      instance (hits, misses, hit rate, cost saved, evictions, item count, index
      size, threshold) for spotting hit-rate degradation, growth, or drift.
    • list_ai_traces — list recent AI application traces (LLM calls, cache
      lookups, memory recalls, retrieval spans).
    • get_ai_trace — full span waterfall for one trace with timing, parent
      relationships, and attributes (model, cache hit/miss, similarity scores).
    • correlate_ai_trace — join a trace's cache/memory spans with live Valkey
      state (key existence, TTL, active threshold, index state) to explain
      unexpected cache misses or stale memory recalls.
  • 3 analytics tools over the persisted metrics and anomaly stores (#331):
    • get_forecast — capacity forecast for a metric (opsPerSec,
      usedMemory, cpuTotal, memFragmentation) with projected time to
      ceiling.
    • get_latency_regressions — sustained p99 latency regressions vs baseline
      from the anomaly event store; companion to get_anomalies.
    • get_largest_keys — largest keys by measured memory usage from key
      analytics snapshots; companion to get_hot_keys. Requires BetterDB Pro
      (keyAnalytics).
  • 2 vector-search tools for the Search module (valkey-search / RediSearch)
    (#332):
    • get_vector_indexes — per-index health: document count, memory usage,
      indexing failures, and percent indexed.
    • get_inference_latency — FT.SEARCH p50/p95/p99 per vector index plus SLA
      breach status. SLA fields require BetterDB Pro inference SLA monitoring.

Changed

  • get_health / instance health now surfaces config-hazard advisories,
    including the default-user-enabled + AOF silent data-loss hazard
    (valkey#3983), with a distinct "could not verify" state (#337).
  • README "Available Tools" updated with the new AI Observability, Analytics,
    and Vector Search sections.
  • server.json registry version synced to 1.4.0 (it had drifted at 1.2.0
    through the 1.3.x line).

Pro tier

get_largest_keys requires BetterDB Pro (keyAnalytics) and the SLA fields of
get_inference_latency require Pro inference SLA monitoring. On community-tier
deployments the underlying endpoints degrade gracefully — sla is reported as
null and Pro-only endpoints surface the tier error to the agent.

v0.31.0

Choose a tag to compare

@KIvanow KIvanow released this 25 Jul 11:50
51a17f0

Frontend telemetry is now attributable to a release, plus security dependency updates.

✨ Changes

  • Frontend events tagged with the monitor version (#339). Events the web app sends to PostHog now carry the release version as a super property — matching the server-side events — so frontend usage and funnels can be attributed to a specific build. Baked in at build time from the existing APP_VERSION build-arg; no config change, falls back to unknown in local dev.
  • Security dependency updates. @fastify/static 9.1.1 → 10.1.2 (#340) and ws 8.20.1 → 8.21.0 (#342) in the API, plus a transitive protobufjs 7.6.3 → 7.6.5 (#325).

⬆ Upgrade notes

  • Fully backwards compatible; no config or schema changes.
  • Docker: betterdb/monitor:0.31.0

Full Changelog: v0.30.0...v0.31.0

What's Changed

  • chore(telemetry): restore delivery visibility, guard key injection, fix serverless by @jamby77 in #320
  • chore(deps): bump setuptools from 81.0.0 to 83.0.0 in /packages/cache-benchmark by @dependabot[bot] in #328
  • chore(deps): bump protobufjs from 7.6.3 to 7.6.5 by @dependabot[bot] in #325
  • chore(deps): bump torch from 2.12.0 to 2.13.0 in /packages/cache-benchmark by @dependabot[bot] in #323
  • feat(web): tag frontend PostHog events with the monitor version by @KIvanow in #339
  • chore(deps): bump @fastify/static from 9.1.1 to 10.1.2 by @dependabot[bot] in #340
  • chore(deps): bump ws from 8.20.1 to 8.21.0 by @dependabot[bot] in #342
  • chore(deps): bump aiohttp from 3.14.0 to 3.14.1 in /packages/cache-benchmark by @dependabot[bot] in #341

Full Changelog: v0.30.0...v0.31.0

Semantic Cache v0.12.0

Choose a tag to compare

@github-actions github-actions released this 23 Jul 14:15
9831d90

What's Changed

  • chore(telemetry): restore delivery visibility, guard key injection, fix serverless by @jamby77 in #320

Full Changelog: v0.30.0...semantic-cache-v0.12.0