Skip to content

feat(web-analytics): emit cache_key_hash on query and lazy precompute logs#60378

Merged
lricoy merged 16 commits into
masterfrom
lricoy/web-analytics-cache-key-hash
May 28, 2026
Merged

feat(web-analytics): emit cache_key_hash on query and lazy precompute logs#60378
lricoy merged 16 commits into
masterfrom
lricoy/web-analytics-cache-key-hash

Conversation

@lricoy
Copy link
Copy Markdown
Member

@lricoy lricoy commented May 28, 2026

Problem

Our q/key (queries-per-distinct-cache-key) analysis for the lazy precompute rollout currently relies on reconstructing the cache key from system.query_log properties JSON. Loki/Posthog logs have the structured web_analytics_query log line on a 14-day retention horizon — but it only carries filter_count, not the dimensions that actually fragment the cache key (filter values, dateRange, breakdownBy).

We also can't currently join "this query went to precompute" (logged by the analytics_platform framework) against "this query came from web analytics" (logged by the runner) — they share no stable identifier across log streams.

Changes

Adds a stable cache_key_hash field to two structured log lines:

  1. web_analytics_query (in WebAnalyticsQueryRunner.calculate's finally block). Computed by compute_query_cache_key_hash(self.query, self.team.timezone) in web_lazy_precompute_common.py. SHA-256 of canonical JSON over the user-facing query schema (kind + properties with values + dateRange + breakdownBy + conversionGoal + sampling + interval + compareFilter + filterTestAccounts + timezone), stripping fields that don't influence the logical cache key (useWebAnalyticsPrecompute, modifiers, version, tags, response, limit/offset, limitBy).

  2. lazy_computation.executed (in LazyComputationExecutor.execute). Accepts an optional cache_key_hash parameter on execute() and ensure_precomputed(). All six web analytics precompute callers (overview, stats, paths, frustration, goals, vitals_paths) compute and pass it.

Same hashing mechanic (SHA-256 / canonical JSON) as the existing compute_query_hash in the lazy framework. Distinct from the framework's internal query_hash because that one hashes the post-build INSERT AST: this one hashes the input schema, so it stays stable across runner-code changes that don't alter the logical cache key, and can be computed regardless of whether the query was eligible for the precompute path.

Hash computation is wrapped in try/except at every call site so a hash-time failure cannot affect the request.

How did you test this code?

I'm an agent. Automated tests I ran locally:

  • New: products/web_analytics/backend/hogql_queries/test/test_web_lazy_precompute_common.py — 12 cases covering stability across calls, fragmentation by query kind / date range / breakdown / filter value / filter operator / timezone / compareFilter, and exclusion of the per-query opt-in toggle. Also asserts hash format (64-char hex).
  • Existing: test_web_analytics_query_runner.py (18 passed), test_lazy_computation_executor.py (111 passed), full lazy-precompute integration suites for overview / goals / stats / paths / frustration / vitals_paths (145 passed, 17 skipped).
  • ruff check + ruff format clean.

No manual prod traffic verification has been done — the log line lands in Loki once deployed.

Publish to changelog?

no — observability/internal-tooling change only.

Docs update

Not needed.

🤖 Agent context

Authored end-to-end by Claude Code (Opus 4.7) in an ad-hoc session focused on improving observability of the web analytics lazy precompute rollout.

Decisions worth noting for reviewers:

  • Why a new input-side hash rather than reusing the framework's query_hash. The framework's hash is computed over the post-build precompute AST. It is the right identity for the precompute job table but unsuitable for cross-path attribution — it doesn't exist when a query isn't eligible for precompute, and it changes when the INSERT AST construction changes even if the user query is logically identical. The new field is documented as a logical (not numerical) sibling of the framework hash.
  • Field exclusion list (useWebAnalyticsPrecompute, modifiers, version, tags, response, limit, offset, limitBy). Same query with the precompute opt-in toggled should share a cache key; same query with different pagination should share a cache key (limit/offset are applied at read time). Other dropped fields are non-cache-key metadata.
  • Property ordering is currently treated as cache-key-relevant. The test documents this as a known behavior rather than a desired one — if two clients send the same filter set in different orders they will hash to different keys. Worth normalizing upstream later if needed.
  • Hash threading vs framework recomputation. The lazy framework receives cache_key_hash as a caller-provided opaque string rather than recomputing it from QueryInfo. The framework is generic over query types (not web-analytics-specific), so the input-schema hash conceptually belongs to the caller.

@github-actions
Copy link
Copy Markdown
Contributor

🎭 Playwright report · View test results →

⚠️ 1 flaky test:

  • Combine two series with a formula and verify computed total (chromium)

These issues are not necessarily caused by your changes.
Annoyed by this comment? Help fix flakies and failures and it'll disappear!

@lricoy lricoy marked this pull request as ready for review May 28, 2026 15:45
@lricoy lricoy requested a review from a team May 28, 2026 15:45
@lricoy lricoy added the stamphog Request AI review from stamphog label May 28, 2026
Copy link
Copy Markdown

@stamphog stamphog Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review agent failed after 3 attempts — needs human review.

@stamphog stamphog Bot removed the stamphog Request AI review from stamphog label May 28, 2026
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 28, 2026

Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
products/web_analytics/backend/hogql_queries/test/test_web_lazy_precompute_common.py:204-216
**Test name directly contradicts its assertion**

`test_property_order_does_not_fragment_key` asserts `!=`, meaning property order *does* fragment the key. The name implies the opposite. A future developer looking for the test that enforces "order is irrelevant" will find this one, see `!=`, and be confused — or worse, flip the assertion back to `==` while renaming without realising the intent was to document current behaviour, not desired behaviour. The comment inside the test explains the distinction well, but the method name overrides that at first glance.

The name should match what the assertion actually proves, e.g. `test_property_order_currently_fragments_key` or `test_property_order_is_not_canonicalized`.

### Issue 2 of 2
products/web_analytics/skills/evaluating-precompute-eligibility/SKILL.md:339-344
The last two bullets reference personal local paths (`~/notes/work/posthog/...`) that only exist on the author's machine. Other engineers using this SKILL will hit dead links.

```suggestion
- The rollout plan of record (cohort definitions, Cache-A / B / C / D tier
  framework) — ask the web-analytics team for the current location.
- The `q/key` methodology note (cost model and break-even-at-1.0 derivation) —
  ask the web-analytics team for the current location.
```

Reviews (1): Last reviewed commit: "Merge branch 'master' into lricoy/web-an..." | Re-trigger Greptile

Comment thread products/web_analytics/skills/evaluating-precompute-eligibility/SKILL.md Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7eeb33a50f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread products/web_analytics/skills/evaluating-precompute-eligibility/SKILL.md Outdated
@lricoy lricoy added the stamphog Request AI review from stamphog label May 28, 2026
Copy link
Copy Markdown

@stamphog stamphog Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The automated review agent failed after 3 attempts and flagged this for human review. The PR exceeds the size threshold and touches core web analytics query runner logic. Request a human review from the web-analytics team before re-requesting.

@stamphog stamphog Bot removed the stamphog Request AI review from stamphog label May 28, 2026
@lricoy lricoy enabled auto-merge (squash) May 28, 2026 18:20
@lricoy lricoy merged commit 30312c9 into master May 28, 2026
270 of 272 checks passed
@lricoy lricoy deleted the lricoy/web-analytics-cache-key-hash branch May 28, 2026 18:24
@deployment-status-posthog
Copy link
Copy Markdown

deployment-status-posthog Bot commented May 28, 2026

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-05-28 19:38 UTC Run
prod-us ✅ Deployed 2026-05-28 19:52 UTC Run
prod-eu ✅ Deployed 2026-05-28 19:56 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants