feat(platform): governance feedback analytics page#1662
Merged
Conversation
…tion Surface user-submitted feedback (chat thumbs up/down + arena verdicts) on a new admin page under Settings → Governance, with first-class agent and model breakdowns. Backend: - messageFeedback gains agentSlug/model/provider (optional) + a by_org_createdAt index for window-bounded scans - messageMetadata gains agentSlug; saveMessageMetadata + onAgentComplete thread it through so feedback rows can attribute by agent - submitFeedback now (a) validates threadId↔organizationId to close a cross-org write gap and (b) reads attribution from messageMetadata rather than trusting client-supplied values; deleteFeedback gets a matching defense-in-depth check - feedback/stats.ts extracts a pure computeFeedbackStats reducer (byRating / byVerdict / topAgents / topModels) — no server-side sentimentPct, client derives ratios - getFeedbackStats: admin guard + period filter (1/7/30/90/all-time, day-aligned UTC) + agent/model/provider filters + hasAnyFeedback flag - listRecentFeedback: paginationOptsValidator + by_org_createdAt desc; projects user names, thread-deleted state, comment truncated at 4 KB Frontend: - New /settings/governance/feedback route with zod validateSearch URL state (period / kind / comments / agent / model / provider), mirroring the automations/metrics pattern - feedback-metrics-page orchestrator with loading / org-empty / period-empty / filter-empty / capped states - Hero sentiment cell + 2-color stacked bar (CSS only); arena summary reuses chat.arena.* i18n keys - Top Assistants + Top Models feedback tables (clickable rows set URL filters); filter chips strip - Recent feedback uses DataTable + enableExpanding to show full comment and arena pairing inline; renders "—" when source thread is deleted i18n: 70+ analytics.feedback.* keys + governance.groups.feedback added to en / de / fr (parity test passes). Tests: 12-case unit suite for the pure reducer.
Today's governance feedback commit (14bbf13) added agentSlug to the messageMetadata schema but not to messageMetadataValidator, so the getMessageMetadata query rejected every row written since with ReturnsValidationError, breaking the chat UI's per-message metadata fetch (cost, tokens, model).
The Open-thread link in the governance feedback page is only navigable when the admin happens to be the thread owner — `canAccessThread` blocks all non-owner reads, so for every other user's feedback the link lands on a "thread not found" page. Remove the column until a proper governance-reviewer role with audit logging lands. Side benefit: drops the per-page N+1 `threadMetadata` lookup from `listRecentFeedback` and the now-unused `feedback.recent.openThread` / `columns.thread` i18n keys (en/de/fr).
i18next is configured with `interpolation.prefix:'{'`/`suffix:'}'`
(see lib/i18n/i18n.ts), so `{{value}}` doesn't interpolate — it
renders verbatim. The new feedback page leaked five keys with
double-brace placeholders, causing literal `{{helpful}} / {{total}}`
under the sentiment hero and `Assistant: {{value}}` in filter chips.
Fix in en/de/fr.
Aggregating "A is better" / "B is better" across rows mixes different model pairs (A/B is a user-picked position, not a random assignment), so the count carries no actionable signal. Replace the four-cell arena summary with a position-agnostic triple — Decisive / Tie / Both bad — and surface a new "Top Model Matchups" table that buckets verdicts by canonical (lexicographic) model pair, projecting wins onto a stable left/right orientation so (X,Y) and (Y,X) merge. The reducer skips matchup aggregation on self-pairs and rows missing either model name. New tests cover orientation flipping and the self-pair / missing-side guards.
The matchup, top-models, and recent-feedback tables truncated model strings (e.g. `openrouter:deepseek/deepseek-v4-flash-...`) to a fixed max-width with ellipsis, which hides the actual identity that admins need to read. Replace `truncate`/`max-w-*` on every model and matchup cell with `break-all` so long IDs wrap to multiple lines instead. Comment previews keep their truncation since the expanded row shows the full text.
The page-level toolbar mixed three filters of different reach: period
flowed through every query, but kind ("All types"/"Message thumbs"/
"Arena verdicts") and comments-only fed only `listRecentFeedback`.
Placing them at the top suggested they affected the sentiment cards,
arena summary, and top tables too. Move both controls into the Recent
feedback section header so the affordance matches the effect; only
period stays in the global toolbar.
|
Caution Review failedPull request was closed or merged during review 📝 WalkthroughWalkthroughThis PR introduces a comprehensive feedback analytics dashboard for the platform. It adds new React components for displaying feedback metrics, including a metrics page with filtering and pagination, summary cards, arena verdict summaries, and data tables (recent feedback, top agents, top models, top matchups). Backend support includes new Convex queries for feedback statistics and recent feedback listings, a pure stats computation function, and mutations updated to capture server-side message attribution. Schema changes add optional agent slug, model, and provider fields to feedback records. The implementation also introduces a new route under governance settings and includes i18n translations in English, French, and German. Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Review rate limit: 4/5 reviews remaining, refill in 12 minutes. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
submitFeedback/deleteFeedbackwith thread↔org checks and reads attribution frommessageMetadataserver-side instead of trusting the client.messageFeedback.{agentSlug,model,provider}+by_org_createdAtindex;messageMetadata.agentSlug(additive only).paginationOptsValidator+DataTablewith expandable rows; kind / comments-only filters are scoped to the section to match their effect, period stays global.analytics.feedback.*keys +governance.groups.feedbackadded to en / de / fr (single-brace placeholders, parity).Pre-PR checklist
bun run check— please run before merge; not executed in this session.services/platform/messages/{en,de,fr}.json.docs/{,de/,fr/}— not yet; admin governance docs still describe pre-feedback state. Recommend follow-up doc PR or extending this one before merge.bun run --filter @tale/docs lint— N/A pending docs update.README*updates — N/A (no top-level surface change).Test plan
/dashboard/<org>/settings/governance/feedbackas an org admin; verify period, kind, comments-only, agent, model, provider URL state round-trips.submitFeedbackwith athreadIdfrom another org — should be rejected.messageMetadataValidatorfix in 564958d.Summary by CodeRabbit
Release Notes