Skip to content

docs(handoff): close the app-analytics stream and correct a dedup claim that was measured wrong - #204

Merged
ZacxDev merged 1 commit into
mainfrom
zach/handoff-analytics-close
Aug 5, 2026
Merged

docs(handoff): close the app-analytics stream and correct a dedup claim that was measured wrong#204
ZacxDev merged 1 commit into
mainfrom
zach/handoff-analytics-close

Conversation

@ZacxDev

@ZacxDev ZacxDev commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Docs-only. The canonical handoff doc on main had gone stale in the way that costs the next session real time — it listed shipped work as pending, and carried one confident claim that measurement disproved.

The correction that matters

The doc said follow-up #6 was "per-mount, so unique views need query-side dedup."

blockInstanceId is not per-mount. It is page_apb_<ULID>, roughly one per placement. Measured on prod: 124 rows carried 28 distinct blockInstanceId across 27 distinct appBlockId — ~1:1 with the app. Deduping on it would have reported ~1 unique viewer per app.

The correct identity is userId, and anonymous rows all carry userId = 0, so they need ip instead — hence uniqExactIf(userId, isAnon=0) + uniqExactIf(ip, isAnon=1) as shipped in civitai#3613. Querying prod before building is what caught it.

I corrected this in place rather than just striking it, because the wrong version was specific and plausible enough to be followed.

Brought current

  • Follow-ups feat(app): add 'app create' command (page-money default) #6 and fix(oauth): form-urlencode the token refresh (was JSON → /token 400s) #5 marked DONE (civitai#3613 + cli#195; civitai#3626), plus cli#193/fix(scaffold): bump page-money pins to published (app-sdk 0.30.0, blocks-react 0.38.0) #194 and civitai#3627. DONE table goes 8 → 15 PRs.
  • preview / component-tests is no longer an open question. It reported success on #3606 and every subsequent PR in the stream (1259/1259). Verified by reading the locator at each open PR's head SHA — PRs with the bare getByText('Generations') failed, PRs with { exact: true } passed — rather than inferring from timing.
  • Deploy/verify status now says verified in production, not merely deployed. civitai app metrics gen-matrix printed real non-zero App loads, cross-checked against a hand-written ClickHouse query and against the reader's own entry in system.query_log. Kept the caveat: 20ms against ~125 rows in a single partition proves the query is correct today, not that partition pruning holds at scale.
  • How to actually read the preview component log — it lives in the Tekton taskrun, not behind the GitHub status URL, and the taskrun reports Succeeded even when the suite fails because the step is report-only. That cost time twice; now it's written down.
  • The pins rot recurred within a day, exactly as the doc predicted — cli#203 re-bumped both pins on 2026-08-05. Now documented as a standing property of this repo rather than an incident to diagnose a third time.
  • Added a latent flake for whoever owns it: the marketplace search-debounce test races two await fill() calls against a 300ms debounce and measured 338ms locally. It failed once on civitai#3627's preview run and passed on a sibling PR ten minutes earlier.

Security

A finding from this stream was reported privately to security@civitai.com per SECURITY.md, which forbids public issues and PRs that demonstrate the issue. The mechanism is deliberately not in this doc — only the operational consequence (treat App loads / unique viewers as untrusted input; don't build payouts, ranking, discovery or leaderboards on them) and a pointer to ask security before changing the ingest path. Verified: the diff contains none of the mechanism.

make ci green; gofmt -s -l . prints nothing.

🤖 Generated with Claude Code

https://claude.ai/code/session_017vvMdxcMKJP9ripQLEiPm9

…im that was measured wrong

The canonical handoff doc on `main` had gone stale in the way that costs the
next session real time: it still listed shipped work as pending, and carried one
confident claim that measurement disproved.

🔴 THE DEDUP ADVICE WAS WRONG AND WOULD HAVE SHIPPED A BROKEN METRIC. The doc
said \"per-mount, so unique views need query-side dedup\". `blockInstanceId` is
NOT per-mount — it is `page_apb_<ULID>`, roughly one per PLACEMENT. Measured on
prod: 124 rows carried 28 distinct `blockInstanceId` across 27 distinct
`appBlockId`, ~1:1 with the app, so deduping on it reports ~1 unique viewer per
app. Corrected in place, with the measurement, because the wrong version was
specific and plausible enough to be followed.

Brought current:

- Follow-ups #6 and #5 marked DONE (civitai#3613 + cli#195, civitai#3626), plus
  cli#193/#194 and civitai#3627. The DONE table goes 8 -> 15 PRs.
- `preview / component-tests` is no longer an open question: it reported success
  on #3606 and every subsequent PR (1259/1259), verified by reading the locator
  at each PR's head SHA rather than inferring from timing.
- Deploy/verify status now records that the feature was verified IN PRODUCTION,
  not merely deployed — with the caveat that 20ms against ~125 rows in one
  partition proves the query is correct today, not that partition pruning holds
  at scale.
- Records how to actually READ the preview component log (it lives in the Tekton
  taskrun, not behind the GitHub status URL), and that the taskrun reports
  Succeeded even when the suite fails because the step is report-only.
- The pins rot recurred within a day exactly as predicted — cli#203 re-bumped
  both pins on 2026-08-05 — so that is now documented as a standing property of
  the repo, not an incident to diagnose a third time.
- Adds the marketplace search-debounce flake (338ms measured against its own
  300ms budget) as a latent issue for whoever owns that test.

A security finding from this stream was reported privately per SECURITY.md. The
mechanism is deliberately NOT in this doc — only the operational consequence,
and a pointer to ask security before changing the ingest path.

Docs-only; `make ci` green and `gofmt -s -l .` prints nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017vvMdxcMKJP9ripQLEiPm9
@ZacxDev
ZacxDev merged commit 1a90671 into main Aug 5, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant