[Design] Opt-in usage analytics to measure adoption

[ravjotbrar]

Summary

This proposes adding opt-in, anonymous usage analytics to Valkey Admin with a single, narrow initial goal: understand how many people use the app and through which deployment mode (Web, Electron, Kubernetes, Docker). Nothing is collected unless a user explicitly opts in. All collection code is open source, and the resulting aggregate data will be published on a public, read-only dashboard so the community sees exactly what we see.

Tracking issue: #477

Motivation

We currently have no signal for how widely Valkey Admin is used or which deployment modes matter most to users. Docker pull counts are a coarse proxy, but they can't distinguish an Electron desktop user from a Kubernetes operator, and they don't tell us whether the app is actually being run. This makes it hard to prioritize (e.g., how much to invest in Electron vs. Web/K8s hardening).

A minimal, transparent, opt-in telemetry signal lets us make those decisions with data while respecting that this is an open-source project where user trust is paramount.

Goals

• Measure count of active installs, segmented by deployment mode
• Be opt-in only: zero data leaves the app without explicit consent
• Be fully transparent: open-source collection code + public dashboard + documented scope
• Be minimal: collect the least data needed to answer the question

Non-Goals (for this initial version)

• No feature-usage tracking, no per-view analytics, no funnels
• No PII, no cluster addresses / hostnames / IPs, no commands, no key names, no credentials
• No behavioral or session-level tracking
• No third-party ad/marketing trackers

What we collect (initial scope)

Exactly one event, app_started, with a minimal payload:

Field	Example	Why
Hashed install ID	SHA-256 hash of a locally-stored random UUID v4	De-duplicate unique installs so counts aren't inflated by restarts
Deployment mode	Web / Electron / Kubernetes / Docker	The core dimension we want to measure
App version	1.2.0	Distinguish active version adoption (proposed — see Open Questions)

The install ID is a random UUID with no linkage to a user account, IP, hostname, or machine identifier. Only a SHA-256 hash of it is transmitted (following the Spinnaker approach LF referenced in the MLflow review), so the collected value can't be reversed to the local ID. It exists only so 100 restarts by one user don't look like 100 users, and has no other use within the app.

What we might collect in the future (NOT in this version)

Listed here for full transparency. Each of these would require its own disclosure and remain under the same opt-in before ever shipping:

• Feature usage frequency (which views are used: Big Keys, Hot Keys, Command Logs, etc.)
• Aggregate cluster size buckets (e.g., "1–5 nodes")
• Error rates by feature
• Session duration

Opt-in UX

1. On first app startup, show a one-time consent dialog that:
   • Explains exactly what's collected (the table above), in plain language
   • Links to TELEMETRY.md and the public dashboard
   • Defaults to off
2. If the user accepts: persist consent + the anonymous ID locally, and send the app_started event immediately (initial version).
3. If the user declines: persist the decline, never send anything, and don't nag.
4. A toggle in Settings lets users change their choice at any time. Opting out stops all further collection.

Future iteration: move sending to app close (and/or periodic batching) rather than immediately on accept, to better reflect a "session" and reduce chattiness. Called out now so the roadmap is transparent.

Storage & platform

We propose storing telemetry in Valkey itself, hosted on LF-approved infrastructure under project/LF custody (see Ownership).

• Counting: unique installs by deployment mode map naturally onto HyperLogLog (PFADD / PFCOUNT) — e.g. PFADD installs:web:2026-08 <hashed-id> gives approximate unique counts in tiny memory, per deployment-mode and time bucket.
• Public dashboard: we commit to a public, read-only dashboard of aggregates so the community sees exactly the same numbers we do.
• What we build: a small opt-in ingestion endpoint, the aggregation/retention logic, and a lightweight dashboard/visualization layer. This is more custom work than an off-the-shelf analytics platform, but keeps the data fully under project control.

Data flow: app → single opt-in event (app_started) → Valkey (HyperLogLog counters) → public dashboard. No inbound listener is added to the app.

The exact hosting (which LF-approved infrastructure) is deferred to the LF telemetry review and the ownership question below.

LF telemetry policy & review

Per the LF Projects Telemetry Data Collection and Usage Policy, telemetry is off by default and requires LFP legal-team review and approval before any collection begins, coordinated through the project manager. No data will be collected until that approval is granted. A filled-out review questionnaire (mirroring the approved MLflow review) accompanies this proposal, and the design already aligns with the policy's criteria: opt-in consent, full anonymization, public availability of collected data, and a documented, auditable collection mechanism.

Transparency & trust commitments

• Opt-in, off by default — always
• Collection code lives in the open-source repo and is reviewable
• No PII or infrastructure identifiers, ever; only a hashed install ID is sent
• Public read-only dashboard so the community sees the same data we do
• Any expansion of the collected fields will be proposed publicly (a discussion like this one) before it ships, and remains under the same opt-in
• A TELEMETRY.md in the repo documents current + planned collection, and how to opt out

Ownership & operational responsibility

Whatever platform we choose, the account/instance must be owned at the project level, not by any individual contributor. Tying it to a personal account creates a bus-factor and trust problem: if that person steps away, the data, billing, and admin access are orphaned.

The intent is for analytics infrastructure to follow the same custody model as the project's other shared assets (the domain, the GitHub org, the Docker Hub valkey org). Concretely:

• Custody: the store/account (self-hosted instance or managed account) is held under the Valkey project / Linux Foundation, alongside existing org-level assets — not a maintainer's personal account.
• Admin access: credentials stored wherever the org's other shared secrets live, with more than one maintainer able to administer it.
• Operation: the Valkey-backed store is self-hosted, so it needs a named owner for patching/uptime even though custody is institutional.
• Cost: hosting/subscription funded the same way other project infrastructure is, rather than by an individual.

This is an open question for the maintainers/governing body and is called out below.

Data retention

Proposed: aggregate event data retained for 12 months, with only derived aggregates surfaced publicly. (Open to community input.)

Open questions for the community

1. Ownership/custody: how should the store/account be owned so it sits under project/LF custody rather than an individual, and is there existing infra/budget for it?
2. Hosting: which LF-approved infrastructure should the Valkey store + ingestion endpoint + dashboard run on, and is there existing infra/budget for it?
3. App version in the initial payload: acceptable, or strip to deployment mode only for v1?
4. Retention window: is 12 months reasonable?
5. Decline behavior: never prompt again, or re-ask after a major version upgrade?
6. Anything the community would want to see on the public dashboard?

[ArgusLi]

I'm wondering why PostHog was chosen as the tool and what other alternatives are possible.

A quick look at PostHog shows that:

• Self-hosted PostHog is officially unsupported and has multiple features that do not exist when compared to PostHog cloud.
• PostHog uses Redis.
• PostHog is quite overkill for this application, as it defaults to autocapture, pageviews, page-leave events etc.

[ravjotbrar]

Good points. This was based off of quick research for how we can share a dashboard of analytics quickly without having to build the UX ourselves. That being said, that part of the discussion is still open and has to be an LF approved data store.