feat(replay-vision): add version column to scanner observations table#61031
Merged
Conversation
Contributor
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
assign-reviewers-posthog
Bot
requested review from
a team,
TueHaulund,
arnohillen and
fasyy612
and removed request for
a team
Contributor
Prompt To Fix All With AIFix the following 1 code review issue. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 1
products/replay_vision/frontend/replay_scanners/components/ScannerObservationsTable.tsx:155
The version sorter uses `?? 0` to handle observations with no `scanner_version`, treating them as version 0. This means null-version rows (rendered as "—") float to the top in ascending sort order, mixed in with any genuine version-0 data, which is inconsistent with how `compareByScore` handles nulls (always last, regardless of direction). A user sorting "oldest first" would see the "—" rows appear first rather than at the end.
```suggestion
sorter: (a, b) => {
const av = a.scanner_snapshot?.scanner_version ?? null
const bv = b.scanner_snapshot?.scanner_version ?? null
if (av === null || bv === null) {
return av === bv ? 0 : av === null ? 1 : -1
}
return av - bv
},
```
Reviews (1): Last reviewed commit: "feat(replay-vision): add version column ..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![stamphog[bot]](https://avatars.githubusercontent.com/in/3000332?s=60&v=4){kind=small}
There was a problem hiding this comment.
A bot review comment on the current head raises a valid unaddressed concern: the version column's sorter treats null scanner_version as 0 (via ?? 0), which inconsistently mixes null rows with genuine v0 rows, unlike the other column sorters that always place nulls last.
Contributor
|
Size Change: 0 B Total Size: 80.9 MB ℹ️ View Unchanged
|
ksvat
added
the
update-snapshots
Graphite App
## Problem Editing a scanner's prompt or config bumps its `scanner_version`, and each observation keeps a frozen snapshot of the version that produced it. The observations table didn't surface this, so a scanner whose prompt changed over time shows very different results across rows (e.g. a scorer returning 0/10 under an old prompt and 8/10 under the current one) with no indication they came from different versions. ## Changes Add a sortable **Version** column to the scanner observations table. Each row shows the `scanner_version` that produced it (from `scanner_snapshot`), color-coded by how far behind the live scanner it ran: - latest / current → green - one version back → yellow - two back → orange - three back → red - four or more → purple A tooltip states how many versions behind each observation is; rows without a snapshot render a muted dash. Pure frontend presentation change — the version data was already persisted per observation and on the scanner. ## How did you test this code? I'm an agent (PostHog Code). Automated checks only: - `pnpm --filter=@posthog/frontend format` — clean - `pnpm --filter=@posthog/frontend typescript:check` — no errors in replay_vision Not rendered in a browser yet. ## 🤖 Agent context Authored with PostHog Code (Claude) during a Replay Vision dogfooding session. The age→color logic maps to existing LemonTag types (success/warning/caution/danger/completion) via a clamped array lookup. No API or model changes. Requires human review. Generated-By: PostHog Code Task-Id: fb62321a-5c49-4102-b367-6af893652809
ksvat
force-pushed
the
posthog-code/replay-vision-version-column
branch
from
c9a6772 to
b006dcb
Compare
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
The bot reviews raised a valid concern about null handling in the version sorter, but those reviews are on an older commit — the current implementation already uses ?? null with explicit null-last logic, matching the behavior of the other column sorters. No showstoppers; purely additive frontend change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Editing a scanner's prompt or config bumps its
scanner_version, and each observation keeps a frozen snapshot of the version that produced it. The observations table didn't surface this, so a scanner whose prompt changed over time shows very different results across rows — e.g. a scorer returning 0/10 under an old prompt and 8/10 under the current one — with no indication they came from different versions.Changes
Adds a sortable Version column to the scanner observations table. Each row shows the
scanner_versionthat produced it (read fromscanner_snapshot), color-coded by how far behind the live scanner it ran:How did you test this code?
visual:
I'm an agent (PostHog Code), so automated checks only:
pnpm --filter=@posthog/frontend format— cleanpnpm --filter=@posthog/frontend typescript:check— no errors inreplay_visionNot yet rendered in a browser; the column should be verified visually before merge.
🤖 Agent context
Authored with PostHog Code (Claude) during a Replay Vision dogfooding session, where comparing a scorer's results across prompt edits was confusing because old and current versions weren't distinguishable in the table. The age→color logic maps to existing
LemonTagtypes via a small clamped array lookup (VERSION_TAG_TYPES); an earlier draft used a nested ternary chain and was simplified for readability. No API or model changes were needed. Requires human review.Created with PostHog Code