Skip to content

feat(setbuilder): TrackVibe LLM enrichment + community vibe display (read) (#391)#422

Merged
thewrz merged 17 commits into
mainfrom
feat/issue-391
Jun 11, 2026
Merged

feat(setbuilder): TrackVibe LLM enrichment + community vibe display (read) (#391)#422
thewrz merged 17 commits into
mainfrom
feat/issue-391

Conversation

@thewrz

@thewrz thewrz commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

Closes #391

Summary

  • Batch LLM vibe enrichment (services/setbuilder/vibe_enrichment.py): enriches uncached pool tracks via the LLM gateway with forced tool_use (20 tracks/call, purpose="vibe_enrichment"), writing to the global TrackVibe cache (table pre-existed from the Phase-0 scaffold). 100 tracks cost exactly 5 gateway calls; a second DJ with the same tracks pays zero (test-pinned).
  • Community consensus (services/setbuilder/community_vibe.py): aggregates TrackVibeOverride votes (latest row per track+user); energy consensus gated on sample_size >= 3 AND pstdev < 1.5, mood on strict majority — both thresholds admin-tunable via new SystemSettings columns (migration 057, down_revision="056").
  • Three-tier precedence resolver (services/setbuilder/vibe_resolver.py): per-field cascade own override → community consensus → LLM cache, with low_confidence flagging (confidence None or < 0.5).
  • API: GET /api/setbuilder/sets/{id}/pool/vibes (60/min) and POST .../pool/vibes/enrich (5/min, NoLlmConfigured → friendly 400). Owner-private 404 semantics. OpenAPI + generated dashboard types regenerated.
  • Pool panel UI: Vibes toggle + Analyze button; read-only VibeTiers chips (You / Crowd / AI) per track, sample-size on the community chip, per-field winner highlight, low-confidence AI guesses flagged with dashed-amber ⚠ styling.
  • Gateway (additive only): ChatResponse.provider populated from the resolved connector's connector_type so the cache can store llm_provider/llm_model provenance.

Design decisions

  1. model_hint="fast" deviation: ChatRequest has no speed-tier concept — dispatch uses model=None, so the DJ's connector-configured model governs. A cross-provider fast/smart mapping would mean refactoring the gateway (out of scope); documented in-code.
  2. Provider provenance: ChatResponse didn't surface which connector served a call. Minimal additive provider field, populated in the gateway's _attempt success path via model_copy — no other gateway behavior changed (streaming path intentionally untouched).
  3. Vibe cache key: SetPoolTrack.track_id is nullable, so the fallback key is sig:{dedupe_sig} (normalized artist+title hash) — globally stable, consistent with the namespaced free-form track_id convention, and lets manual tracks share the global cache.
  4. Cache-hit semantics: any TrackVibe row matching (track_id, prompt_version, schema_version) counts regardless of provider/model — that's what makes "second DJ pays zero" true across different connectors. Newest row wins at read time; bumping PROMPT_VERSION in code lazily invalidates old rows.
  5. Batch failure policy: the first transient LlmError marks the current batch + all remaining tracks failed and stops (no provider hammering); NoLlmConfigured maps to a friendly 400. A mid-batch insert race keeps the paid LLM rows (re-query winners, re-insert only missing — test-pinned).
  6. Community semantics: the viewing DJ's own vote is excluded from their community tier (it's already tier 1); override rows are full snapshots (latest per track+user wins) — the v1.1 write endpoint must carry forward unchanged fields, documented in the module docstring.
  7. TrackVibeOverride existed from the Phase-0 scaffold (migration 046), so no new table was needed; per the issue note, no write endpoints were added (write UX is v1.1) — this PR is the read path only.
  8. Low-confidence flag computed backend-side (LlmVibeOut.low_confidence) so the frontend stays dumb; threshold 0.5.
  9. Enrichment runs synchronously in-request behind a 5/min rate limit — acceptable at current pool sizes (100 tracks = 5 sequential calls). A background job is a fair v1.1 follow-up for very large pools.
  10. Thresholds tunable via PATCH /api/admin/settings (bounds 1–100 / 0.1–5.0, 422-tested); admin-UI form controls deferred.

Test plan

  • Backend: ruff check, ruff format, bandit, alembic upgrade head && alembic check (single head 057), pytest — 2749 passed, coverage 88.38% (≥85%)
  • Frontend: ESLint (0 errors), tsc --noEmit, vitest — 1118 passed, coverage thresholds hold
  • Acceptance: test_100_tracks_costs_5_calls, test_second_run_fully_cached_even_for_other_dj, resolver tier tests (test_own_override_wins / test_community_beats_llm / test_llm_fallback / test_no_tiers_resolves_none)
  • openapi.json + api-types.generated.ts verified byte-identical to regeneration

Summary by CodeRabbit

  • New Features

    • Per-track three-tier "Vibe" display (You / Crowd / AI) with show/hide toggle and an Analyze action to enrich uncached tracks; visual chips, low-confidence warnings, and toasts for results/errors.
    • Admin settings expose configurable community-vibe thresholds.
  • Chores

    • Database migration and settings plumbing for vibe consensus parameters.
  • Tests

    • Extensive frontend and backend tests covering UI, enrichment, consensus rules, and API endpoints.
  • Docs

    • Implementation plan for the vibe/enrichment system.

thewrz added 15 commits June 10, 2026 22:44
… note

- community_vibe.py: module docstring notes that override rows are FULL
  SNAPSHOTS and the v1.1 write endpoint must carry forward unchanged fields;
  CommunityVibe dataclass docstring defines sample_size.
- vibe_resolver.py: dedupe keys via dict.fromkeys before IN-clause queries in
  _own_overrides and _llm_vibes; community_consensus already deduped in-place.
- community_vibe.py: community_consensus dedupes track_keys via dict.fromkeys
  so tracks sharing a vibe_key don't generate duplicate IN values.
- test_setbuilder_vibes.py: three edge-case tests added to TestCommunityConsensus:
  stddev exactly at threshold is rejected, single-vote with min_sample=1 passes,
  all-null retraction row supersedes the earlier vote (retraction test).
GET /api/setbuilder/sets/{id}/pool/vibes returns the three-tier vibe
state (own / community / LLM, plus per-field resolution with source
provenance) for every pool track. POST .../pool/vibes/enrich runs the
batch LLM enrichment through the gateway (5/min — each call fans out
to BATCH_SIZE-track dispatches) and returns run stats plus the
refreshed state. NoLlmConfigured maps to a friendly 400. Both
endpoints keep owner-private 404 semantics.
Migration 057 (1899c49) added vibe_consensus_min_sample /
vibe_consensus_max_stddev to SystemSettings but the exact-key contract
test was not updated, failing on branch since that commit.
@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c5d39c77-35c2-400c-a396-e21d167fd0dd

📥 Commits

Reviewing files that changed from the base of the PR and between 1c73797 and dc4b635.

📒 Files selected for processing (1)
  • dashboard/lib/api.ts

📝 Walkthrough

Walkthrough

Adds three‑tier TrackVibe support: DB migration and settings, backend consensus/enrichment/resolver services, new setbuilder GET/POST pool-vibes API and OpenAPI/TS surface, VibeTiers UI + CSS, PoolPanel toggle/analyze integration, and extensive tests and docs.

Changes

TrackVibe Enrichment Feature

Layer / File(s) Summary
System Settings & LLM Gateway Foundation
server/alembic/versions/057_add_vibe_consensus_settings.py, server/app/models/system_settings.py, server/app/schemas/system_settings.py, server/app/services/system_settings.py, server/app/api/admin.py, server/app/services/llm/base.py, server/app/services/llm/gateway.py, server/tests/test_api_contracts.py, server/tests/test_llm_gateway.py
Database migration adds vibe_consensus_min_sample and vibe_consensus_max_stddev. ORM model, service, and Pydantic schemas updated; admin PATCH wiring and API contract keys updated. ChatResponse.provider added and gateway populates it; gateway test verifies provider enrichment.
Backend Vibe Services
server/app/services/setbuilder/community_vibe.py, server/app/services/setbuilder/vibe_enrichment.py, server/app/services/setbuilder/vibe_resolver.py, server/tests/test_setbuilder_vibes.py
community_vibe.py computes per-track consensus with sample/stddev rules. vibe_enrichment.py batches LLM calls, parses tool outputs defensively, caches by key, and uses race-safe persistence. vibe_resolver.py composes per-track state and resolves per-field precedence with low-confidence detection. Tests cover batching, caching, parsing, race handling, consensus math, and resolver behavior.
Backend API Endpoints & Schemas
server/app/api/setbuilder.py, server/app/schemas/setbuilder.py
Adds GET /sets/{set_id}/pool/vibes returning PoolVibesState and POST /sets/{set_id}/pool/vibes/enrich returning VibeEnrichmentResult; _pool_vibes_state maps resolver output into response schemas and NoLlmConfigured maps to HTTP 400. New Pydantic schemas for tiers, resolved fields, per-track state, pool state, and enrichment results.
API Types & Generated Code Surface
dashboard/lib/api-types.generated.ts, dashboard/lib/api-types.ts, dashboard/lib/api.ts, server/openapi.json
OpenAPI regenerated with tier/track/pool/enrichment schemas and two new endpoints. TS aliases exported (PoolVibesState, TrackVibeState, VibeEnrichmentResult). ApiClient gains getPoolVibes and enrichPoolVibes.
VibeTiers React Component & Styling
dashboard/app/(dj)/setbuilder/components/VibeTiers.tsx, dashboard/app/(dj)/setbuilder/setbuilder.module.css, dashboard/app/(dj)/setbuilder/components/__tests__/VibeTiers.test.tsx
New VibeTiers component renders three tier chips (You/Crowd/AI) with values, aria labels, low-confidence warning, and per-chip classes for empty/low/winner states. CSS rules added for chip layout and variants. Tests validate rendering, placeholders, low-confidence indicator, sample-size suffix, and winner highlighting.
PoolPanel Integration & UI State
dashboard/app/(dj)/setbuilder/components/PoolPanel.tsx, dashboard/app/(dj)/setbuilder/components/__tests__/PoolPanel.test.tsx
PoolPanel adds vibes map and UI flags (showVibes, vibesLoaded, vibesBusy), toggleVibes to lazily GET pool vibes, and analyzeVibes to POST enrichment and refresh state with toast messages (special-cased HTTP 400). Header shows Analyze when vibes visible; track rows render VibeTiers when available. Frontend tests cover toggle caching, retry on failure, enrichment success, and error handling.
Comprehensive Testing & API Contracts
server/tests/test_setbuilder_vibes.py, server/tests/test_setbuilder_vibes_api.py, docs/superpowers/plans/2026-06-10-trackvibe-llm-enrichment.md, dashboard/app/admin/settings/__tests__/page.test.tsx
Extensive backend unit tests for enrichment batching/caching, consensus thresholds, resolver precedence, and race conditions. API tests for ownership/auth, GET resolution, enrichment behavior (including NoLlmConfigured → 400), and caching on reruns. Implementation plan document added; admin settings test fixture updated.

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant PoolPanel
  participant API_Get as GET /pool/vibes
  participant API_Enrich as POST /pool/vibes/enrich
  participant Resolver
  participant DB
  participant Gateway

  User->>PoolPanel: click "Vibes"
  PoolPanel->>PoolPanel: toggleVibes()
  alt vibes not loaded
    PoolPanel->>API_Get: request PoolVibesState
    API_Get->>Resolver: build_pool_vibe_states(set_id)
    Resolver->>DB: read overrides, community votes, LLM cache
    Resolver-->>API_Get: PoolVibesState
    API_Get-->>PoolPanel: PoolVibesState
    PoolPanel->>PoolPanel: buildVibeMap()
  end
  User->>PoolPanel: click "Analyze"
  PoolPanel->>API_Enrich: request enrichment
  API_Enrich->>DB: check cache, determine missing keys
  API_Enrich->>Gateway: dispatch batched tool calls for missing tracks
  Gateway-->>API_Enrich: tool-call responses
  API_Enrich->>DB: persist TrackVibe rows
  API_Enrich-->>PoolPanel: VibeEnrichmentResult + refreshed PoolVibesState
  PoolPanel->>PoolPanel: update vibes map, show toast
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

"A rabbit hops with a tiny pen,
I fetched the vibes for every den,
Crowd, You, and AI in three neat chips,
Low-confidence flagged with tiny blips,
Toasts and tests keep cadence well and zen."

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 29.55% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main feature: TrackVibe LLM enrichment and community vibe display for the setbuilder, with clear reference to issue #391.
Linked Issues check ✅ Passed All coding requirements from issue #391 are met: global TrackVibe cache with required fields, batch LLM enrichment achieving ≤5 calls for 100 tracks, community consensus with tunable thresholds, three-tier precedence resolver, and UI display of all tiers with low-confidence flagging.
Out of Scope Changes check ✅ Passed All changes are directly scoped to issue #391: TrackVibe infrastructure (services, schemas, API), frontend display (VibeTiers, PoolPanel), database migrations, system settings, and comprehensive tests. No unrelated modifications detected.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/issue-391

Comment @coderabbitai help to get the list of available commands and usage tips.

@thewrz

thewrz commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@thewrz

thewrz commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown
✅ Action performed

Full review finished.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (2)
server/tests/test_setbuilder_vibes_api.py (1)

59-59: 💤 Low value

Consider adding return type annotation for consistency.

The helper _llm_vibe_row is missing a return type annotation, while _fake_response on line 72 includes one. Adding -> TrackVibe would improve consistency and readability.

♻️ Suggested fix
-def _llm_vibe_row(track_id: str = "tidal:0", *, energy: int = 5, confidence: float = 0.8):
+def _llm_vibe_row(track_id: str = "tidal:0", *, energy: int = 5, confidence: float = 0.8) -> TrackVibe:
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/tests/test_setbuilder_vibes_api.py` at line 59, The helper function
_llm_vibe_row lacks a return type annotation—add "-> TrackVibe" to its signature
to match the style of _fake_response and improve consistency; update the
function definition for _llm_vibe_row(track_id: str = "tidal:0", *, energy: int
= 5, confidence: float = 0.8) to include the return type TrackVibe and ensure
any imports or type references needed for TrackVibe are available in the test
file.
dashboard/app/(dj)/setbuilder/setbuilder.module.css (1)

619-619: ⚡ Quick win

Use rem units for font-size to match file conventions and improve accessibility.

Line 619 declares font-size: 10px, but every other font-size in this file uses rem units (e.g., line 359 uses 0.625rem for the same 10px equivalent). Using rem respects user browser font-size preferences and maintains consistency with the codebase.

♻️ Proposed fix
 .vibeChip {
   display: inline-flex;
   align-items: center;
   gap: 4px;
-  font-size: 10px;
+  font-size: 0.625rem;
   padding: 1px 6px;
   border-radius: 4px;
   border: 1px solid var(--border, `#3a3a3a`);
   color: var(--text-secondary);
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@dashboard/app/`(dj)/setbuilder/setbuilder.module.css at line 619, Replace the
hard-coded "font-size: 10px" in the affected rule in setbuilder.module.css with
the equivalent rem unit used throughout the file (0.625rem) so it matches the
project's rem-based convention and respects user/browser font-size settings.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@dashboard/app/`(dj)/setbuilder/components/PoolPanel.tsx:
- Around line 163-176: The toggleVibes handler sets vibesLoaded before the async
getPoolVibes completes, so a failed fetch leaves vibesLoaded true and prevents
retries; change the flow in toggleVibes (and related state calls) so you do not
set vibesLoaded(true) up-front — instead call setVibesBusy(true) then kick off
api.getPoolVibes(setId) and on success call setVibes(buildVibeMap(...)) and
setVibesLoaded(true); on failure clear vibesBusy and also ensure vibesLoaded is
false (or not set) and propagate the toast; keep the final finally to clear
vibesBusy. Reference: toggleVibes, api.getPoolVibes, setVibesLoaded,
setVibesBusy, setVibes, setToast.

In `@dashboard/lib/api-types.generated.ts`:
- Around line 11217-11247: The generated operation
enrich_pool_vibes_api_setbuilder_sets__set_id__pool_vibes_enrich_post is missing
the documented 400 response for NoLlmConfigured; update that operation's
responses to include a 400 entry (headers and content "application/json")
pointing to the appropriate error schema (e.g.,
components["schemas"]["HTTPError"] or the specific NoLlmConfigured schema) so
the frontend can type-check and handle the user-facing 400 error.

In `@server/openapi.json`:
- Around line 16492-16512: Add the missing 400 response for POST
/pool/vibes/enrich by updating the route definition to include a 400 response
that references the friendly error schema (e.g., "NoLlmConfigured") and then
regenerate the OpenAPI spec so server/openapi.json includes that response;
specifically, modify the POST /pool/vibes/enrich route source to add a
responses["400"] entry with description "No LLM configured" and content
application/json -> schema -> $ref to the NoLlmConfigured (or equivalent
friendly error) component, then run the OpenAPI generation step to produce an
updated openapi.json containing the 400 entry.

---

Nitpick comments:
In `@dashboard/app/`(dj)/setbuilder/setbuilder.module.css:
- Line 619: Replace the hard-coded "font-size: 10px" in the affected rule in
setbuilder.module.css with the equivalent rem unit used throughout the file
(0.625rem) so it matches the project's rem-based convention and respects
user/browser font-size settings.

In `@server/tests/test_setbuilder_vibes_api.py`:
- Line 59: The helper function _llm_vibe_row lacks a return type annotation—add
"-> TrackVibe" to its signature to match the style of _fake_response and improve
consistency; update the function definition for _llm_vibe_row(track_id: str =
"tidal:0", *, energy: int = 5, confidence: float = 0.8) to include the return
type TrackVibe and ensure any imports or type references needed for TrackVibe
are available in the test file.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e53711ae-f2b8-49b1-bf86-9afd0d87139e

📥 Commits

Reviewing files that changed from the base of the PR and between 3b0fee4 and 6dd66bf.

📒 Files selected for processing (27)
  • dashboard/app/(dj)/setbuilder/components/PoolPanel.tsx
  • dashboard/app/(dj)/setbuilder/components/VibeTiers.tsx
  • dashboard/app/(dj)/setbuilder/components/__tests__/PoolPanel.test.tsx
  • dashboard/app/(dj)/setbuilder/components/__tests__/VibeTiers.test.tsx
  • dashboard/app/(dj)/setbuilder/setbuilder.module.css
  • dashboard/app/admin/settings/__tests__/page.test.tsx
  • dashboard/lib/api-types.generated.ts
  • dashboard/lib/api-types.ts
  • dashboard/lib/api.ts
  • docs/superpowers/plans/2026-06-10-trackvibe-llm-enrichment.md
  • server/alembic/versions/057_add_vibe_consensus_settings.py
  • server/app/api/admin.py
  • server/app/api/setbuilder.py
  • server/app/models/system_settings.py
  • server/app/schemas/setbuilder.py
  • server/app/schemas/system_settings.py
  • server/app/services/llm/base.py
  • server/app/services/llm/gateway.py
  • server/app/services/setbuilder/community_vibe.py
  • server/app/services/setbuilder/vibe_enrichment.py
  • server/app/services/setbuilder/vibe_resolver.py
  • server/app/services/system_settings.py
  • server/openapi.json
  • server/tests/test_api_contracts.py
  • server/tests/test_llm_gateway.py
  • server/tests/test_setbuilder_vibes.py
  • server/tests/test_setbuilder_vibes_api.py

Comment thread dashboard/app/(dj)/setbuilder/components/PoolPanel.tsx Outdated
Comment thread dashboard/lib/api-types.generated.ts
Comment thread server/openapi.json
thewrz and others added 2 commits June 11, 2026 02:46
…h endpoint

- PoolPanel: only set vibesLoaded after a successful getPoolVibes, so a
  failed initial fetch can be retried on the next toggle-on; guard with
  vibesBusy to prevent duplicate in-flight GETs. Regression test added.
- Declare the 400 (NoLlmConfigured) response on POST /pool/vibes/enrich
  and regenerate openapi.json + api-types.generated.ts so the typed
  contract includes the error case the frontend special-cases.

Addresses CodeRabbit review on PR #422.
@thewrz thewrz merged commit 5f2a411 into main Jun 11, 2026
9 of 10 checks passed
@thewrz thewrz deleted the feat/issue-391 branch June 11, 2026 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

WrzDJSet: TrackVibe LLM enrichment + community vibe display (read)

1 participant