Skip to content

feat(setbuilder): pool import — 4 sources with source tagging + dedupe (#388)#414

Merged
thewrz merged 8 commits into
mainfrom
feat/issue-388
Jun 9, 2026
Merged

feat(setbuilder): pool import — 4 sources with source tagging + dedupe (#388)#414
thewrz merged 8 commits into
mainfrom
feat/issue-388

Conversation

@thewrz

@thewrz thewrz commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Closes #388

Summary

  • Pool tables (set_pool_sources, set_pool_tracks) — migration 053, cascade under sets; every track tagged with its import source (importedVia chip), track_id follows the TrackVibe namespaced-string convention (tidal:123, request:7, …)
  • Pool service (server/app/services/setbuilder/pool.py): dedupe-on-import (exact ISRC first, then normalized artist+title signature via track_normalizer; first import wins, original source tag preserved), per-set source reuse, removal flows (by ids / by source), candidate builders for all five flows
  • Public-URL validator (playlist_url.py): https-only, exact-host allowlist, strict ID charsets, no userinfo/port tricks — the user URL is never fetched (SSRF defense); IDs are extracted and pulled via official APIs (spotipy client-credentials for Spotify, DJ's connected Tidal session for Tidal)
  • API (additive on /api/setbuilder): GET …/pool, GET /playlists (picker), POST …/pool/import/{event,tidal,beatport,url,manual}, POST …/pool/url-preview, POST …/pool/tracks/remove, DELETE …/pool/sources/{id} — all get_current_active_user, owner-or-404, rate-limited, Pydantic-constrained inputs
  • Frontend (dashboard/app/(dj)/setbuilder/components/): PoolPanel (sources accordion w/ click-to-filter + hover-× remove, type tabs w/ live counts, search, multi-select footer, right-click context menu, toast "N new · M de-duped"), ImportModal (event picker / Tidal / Beatport / public-URL validate→preview→import / manual debounced search), PoolBadges (Camelot via lib/camelot-colors, BPM, energy bars, source icons)
  • Tests: 41 service-level + 23 API-boundary (pytest) + 7 component tests (vitest)

Design decisions

  • Public URL providers: end-to-end support for Spotify (client credentials — no user auth needed) and Tidal (requires the DJ's connected session). Apple Music / YouTube / SoundCloud URL shapes are recognized but return supported: false with a clear message — no public-API credentials exist in this codebase for them. Beatport URLs route DJs to the OAuth picker instead.
  • Re-import = refresh: importing the same playlist/event again reuses the existing source row (no duplicate accordion rows) and imports only new tracks; the dedupe toast reports it.
  • Dedupe constraint in DB: UNIQUE(set_id, dedupe_sig) keeps dedupe honest under concurrent imports; ISRC matching is handled at the service layer (normalized, dash-stripped, uppercased).
  • Event imports exclude REJECTED requests (DJ explicitly declined those).
  • energy is nullable — TrackVibe enrichment is WrzDJSet: TrackVibe LLM enrichment + community vibe display (read) #391; the badge renders empty bars until then.
  • Manual bucket is a per-set singleton source with no hover-× (matches the design prototype); manual tracks are removable via track/multi-select flows.
  • artwork_url must be https (Pydantic pattern) — defensive against scheme tricks in stored/rendered image URLs.
  • Remix titles: Beatport mix names other than "Original Mix" are folded into the title ("Song (Club Mix)") so named remixes don't fuzzy-collide with originals (the normalizer preserves named remixes).

Migration note

Migration 053 anchored on 052; sibling PR for #398 uses slot 054 — second to merge re-anchors.

Test plan

  • Backend: ruff check/format, bandit, pytest (2625 passed, 87.6% cov), alembic upgrade head && alembic check clean on a fresh DB
  • Frontend: eslint, tsc --noEmit, vitest (1012 passed)
  • Manual: import each source type on a live set, verify chips/counts/toast, remove-by-source

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Pool panel in Set Builder with full import flows (events, Tidal/Beatport, public URL preview/import, manual), source badges (BPM/Camelot/energy), search/filters, multi-select, bulk removal, context menu, and dedupe-toasts.
  • Documentation
    • Added rollout/implementation plan for pool import.
  • Tests
    • Comprehensive frontend and backend tests covering imports, filtering, dedupe, URL parsing, previews, and removals.

@coderabbitai

coderabbitai Bot commented Jun 9, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 3565464a-b03e-42cf-bf85-69eb91ddcead

📥 Commits

Reviewing files that changed from the base of the PR and between 4bb5c39 and 417d340.

📒 Files selected for processing (8)
  • dashboard/app/(dj)/setbuilder/[setId]/page.tsx
  • dashboard/lib/api-types.generated.ts
  • dashboard/lib/api-types.ts
  • dashboard/lib/api.ts
  • server/alembic/versions/053_add_setbuilder_pool_tables.py
  • server/app/models/set.py
  • server/app/schemas/setbuilder.py
  • server/openapi.json
✅ Files skipped from review due to trivial changes (2)
  • dashboard/app/(dj)/setbuilder/[setId]/page.tsx
  • dashboard/lib/api-types.generated.ts
🚧 Files skipped from review as they are similar to previous changes (5)
  • dashboard/lib/api-types.ts
  • server/app/models/set.py
  • server/alembic/versions/053_add_setbuilder_pool_tables.py
  • server/app/schemas/setbuilder.py
  • dashboard/lib/api.ts

📝 Walkthrough

Walkthrough

Adds a full pool candidate-track surface: DB migration and ORM models, pool service (import builders, dedupe, removal), API endpoints and types, dashboard ApiClient methods, UI components (PoolPanel, ImportModal, badges), CSS, and comprehensive backend + frontend tests.

Changes

WrzDJSet Pool Import Feature

Layer / File(s) Summary
Database schema and ORM models
server/alembic/versions/053_add_setbuilder_pool_tables.py, server/app/models/set_pool.py, server/app/models/__init__.py, server/app/models/set.py
New set_pool_sources and set_pool_tracks tables with CASCADE delete, unique constraint on (set_id, dedupe_sig), plus SQLAlchemy models with cascade-orphan relationships and exports.
Public playlist URL validation
server/app/services/setbuilder/playlist_url.py
Parse-only, SSRF-safe playlist URL parser enforcing https, rejecting creds/ports, extracting Spotify/Tidal IDs, and returning supported/unsupported typed results.
Pool service: candidates, dedup, import/removal
server/app/services/setbuilder/pool.py
Candidate builders for event/Tidal/Beatport/public URL/manual, ISRC-first dedupe with fuzzy artist+title fallback, source creation/reuse, single-commit import returning added/deduped counts, and scoped track/source removal.
API routes for pool operations
server/app/api/setbuilder.py
Pool snapshot, builder-playlist listing, import endpoints (event/tidal/beatport/url/manual), URL preview, and remove endpoints with ownership checks and HTTP error mapping.
Pydantic schemas and generated API types
server/app/schemas/setbuilder.py, dashboard/lib/api-types.generated.ts, dashboard/lib/api-types.ts
Request/response models for pool flows, pool state snapshot types, builder playlists output, and generated TypeScript operation types.
Frontend API client
dashboard/lib/api.ts
Typed ApiClient methods: getPool, getBuilderPlaylists, importPoolEvent/Tidal/Beatport/Url/Manual, previewPoolUrl, removePoolTracks, removePoolSource.
Pool badge primitives
dashboard/app/(dj)/setbuilder/components/PoolBadges.tsx
Components: source icon/color, Camelot badge, BPM pill, and compact energy visualization.
Import modal with 5 flows
dashboard/app/(dj)/setbuilder/components/ImportModal.tsx
Client-only modal implementing event picker, Tidal/Beatport playlist import, public-URL preview/import, and manual track search/import with loading/error handling and dedupe labeling.
Pool panel: filtering, removal, multi-select
dashboard/app/(dj)/setbuilder/components/PoolPanel.tsx
Dashboard UI showing sources accordion, type tabs, search, per-track badges, context menu, multi-select with batch removal, import popover, and toast feedback.
Pool and import modal styling
dashboard/app/(dj)/setbuilder/setbuilder.module.css
Comprehensive CSS for pool panel layout, source rows with hover remove affordance, track row selection/stripe, and full-screen import modal styles.
Setbuilder page integration
dashboard/app/(dj)/setbuilder/[setId]/page.tsx
Mounts PoolPanel with setId, replacing the prior static placeholder.
Backend tests
server/tests/test_setbuilder_pool_service.py, server/tests/test_setbuilder_pool_api.py
Unit and API tests covering dedupe rules, camelot parsing, source management, provider imports, URL parsing/preview, manual import, ownership, and removal semantics.
Frontend tests
dashboard/app/(dj)/setbuilder/components/__tests__/PoolPanel.test.tsx, dashboard/app/(dj)/setbuilder/components/__tests__/ImportModal.test.tsx
Vitest + RTL suites validating ImportModal flows and PoolPanel interactions: rendering, filtering, removal, multi-select, import dedupe toast, and context-menu actions.
Implementation plan
docs/superpowers/plans/2026-06-09-setbuilder-pool-import.md
Design and rollout plan documenting data model, service design, URL parser SSRF defenses, API matrix, frontend UX, and test coverage.

Sequence Diagram(s)

sequenceDiagram
  participant Browser
  participant PoolPanel
  participant ApiClient
  participant SetbuilderAPI
  participant PoolService
  participant Database
  Browser->>PoolPanel: open Import modal / enter playlist URL
  PoolPanel->>ApiClient: previewPoolUrl(setId, url)
  ApiClient->>SetbuilderAPI: POST /api/setbuilder/sets/{set_id}/pool/url-preview
  SetbuilderAPI->>PoolService: preview_public_playlist(provider, playlist_id)
  PoolService->>SetbuilderAPI: preview metadata
  SetbuilderAPI-->>ApiClient: PoolUrlPreview
  ApiClient-->>PoolPanel: preview data
  Browser->>PoolPanel: confirm import
  PoolPanel->>ApiClient: importPoolUrl(setId, url)
  ApiClient->>SetbuilderAPI: POST /api/setbuilder/sets/{set_id}/pool/import/url
  SetbuilderAPI->>PoolService: candidates_from_public_url(provider, playlist_id)
  PoolService->>Database: get_or_create_source + import_candidates
  Database-->>PoolService: inserted rows + dedupe counts
  PoolService-->>SetbuilderAPI: PoolImportResult
  SetbuilderAPI-->>ApiClient: PoolImportResult with pool snapshot
  ApiClient-->>PoolPanel: import result -> update UI / show toast
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • wrzonance/WrzDJ#410: Introduces the setbuilder page shell that this PR extends with the new Pool panel component.

"🐰 Five sources flowing in, dedupe keeps them clean,
Badges glow with camelot, the best pool I've seen!
Remove by source or track, filter by the kind,
Pool import magic, candidate tracks designed."

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 22.22% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title clearly summarizes the main feature: pool import with 4 sources, source tagging, and deduplication logic, matching the changeset's primary objective.
Linked Issues check ✅ Passed All primary coding objectives from issue #388 are implemented: four import sources (event, Tidal, Beatport, URL, manual), source tagging via importedVia, dedupe with ISRC/signature fallback, sources accordion with filtering/removal, removal flows, and pool badges.
Out of Scope Changes check ✅ Passed All changes are directly related to implementing issue #388's pool import feature—database schema, service logic, API endpoints, frontend components, and supporting tests are all in scope.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/issue-388

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
server/tests/test_setbuilder_pool_service.py (1)

108-109: ⚡ Quick win

Consider using position-independent assertions.

Lines 108-109 (and similar patterns at lines 125) assume tracks[0] is the first inserted track. If pool.get_pool() doesn't include an explicit ORDER BY, this could be flaky across different database engines or states.

Recommend either:

  1. Verifying get_pool has explicit ordering (e.g., ORDER BY created_at, id), or
  2. Using a dictionary lookup pattern like the API tests (line 107-108 in test_setbuilder_pool_api.py):
    by_title = {t.title: t for t in tracks}
    assert by_title["Song A"].camelot == "8B"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/tests/test_setbuilder_pool_service.py` around lines 108 - 109, The
assertions in test_setbuilder_pool_service.py that use tracks[0] (e.g., checks
of tracks[0].camelot and tracks[0].source_id) are brittle because get_pool() has
no guaranteed ordering; replace position-dependent checks with a
position-independent lookup: build a mapping from a stable key (e.g., title or
id) to track objects (like by_title = {t.title: t for t in tracks}) and assert
against by_title["Song A"].camelot and by_title["Song A"].source_id; also update
the other similar assertion block (the one near the later check around line 125)
to use the same dictionary lookup pattern, or alternatively add an explicit
ORDER BY in the get_pool implementation if you prefer deterministic ordering.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@server/tests/test_setbuilder_pool_service.py`:
- Around line 108-109: The assertions in test_setbuilder_pool_service.py that
use tracks[0] (e.g., checks of tracks[0].camelot and tracks[0].source_id) are
brittle because get_pool() has no guaranteed ordering; replace
position-dependent checks with a position-independent lookup: build a mapping
from a stable key (e.g., title or id) to track objects (like by_title =
{t.title: t for t in tracks}) and assert against by_title["Song A"].camelot and
by_title["Song A"].source_id; also update the other similar assertion block (the
one near the later check around line 125) to use the same dictionary lookup
pattern, or alternatively add an explicit ORDER BY in the get_pool
implementation if you prefer deterministic ordering.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: bc4b32e8-97a1-45f3-b917-f634375d39a3

📥 Commits

Reviewing files that changed from the base of the PR and between c50ec3f and 9f31c24.

📒 Files selected for processing (21)
  • dashboard/app/(dj)/setbuilder/[setId]/page.tsx
  • dashboard/app/(dj)/setbuilder/components/ImportModal.tsx
  • dashboard/app/(dj)/setbuilder/components/PoolBadges.tsx
  • dashboard/app/(dj)/setbuilder/components/PoolPanel.tsx
  • dashboard/app/(dj)/setbuilder/components/__tests__/PoolPanel.test.tsx
  • dashboard/app/(dj)/setbuilder/setbuilder.module.css
  • dashboard/lib/api-types.generated.ts
  • dashboard/lib/api-types.ts
  • dashboard/lib/api.ts
  • docs/superpowers/plans/2026-06-09-setbuilder-pool-import.md
  • server/alembic/versions/053_add_setbuilder_pool_tables.py
  • server/app/api/setbuilder.py
  • server/app/models/__init__.py
  • server/app/models/set.py
  • server/app/models/set_pool.py
  • server/app/schemas/setbuilder.py
  • server/app/services/setbuilder/playlist_url.py
  • server/app/services/setbuilder/pool.py
  • server/openapi.json
  • server/tests/test_setbuilder_pool_api.py
  • server/tests/test_setbuilder_pool_service.py

Raises frontend branch coverage above the 68% CI threshold
(67.23% -> 69.64%) by covering event picker, Tidal/Beatport
playlist, public-URL validate+import, and manual search flows.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
dashboard/app/(dj)/setbuilder/components/__tests__/ImportModal.test.tsx (1)

144-144: ⚡ Quick win

Prefer data-testid over class-based querySelector for the backdrop.

The [class*="modalBackdrop"] selector couples the test to CSS module implementation details. If the class name changes during refactoring, this test will break.

♻️ Recommended fix using data-testid

Add data-testid="modal-backdrop" to the backdrop element in ImportModal.tsx, then update the test:

-    fireEvent.click(container.querySelector('[class*="modalBackdrop"]')!);
+    fireEvent.click(screen.getByTestId('modal-backdrop'));

Alternatively, if the backdrop has an accessible role, prefer getByRole.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@dashboard/app/`(dj)/setbuilder/components/__tests__/ImportModal.test.tsx at
line 144, Update the backdrop selection to avoid coupling to CSS class names:
add a stable attribute (e.g., data-testid="modal-backdrop") to the backdrop
element in the ImportModal component (ImportModal.tsx) and change the test in
ImportModal.test.tsx to use the testing-library query
(getByTestId('modal-backdrop') or screen.getByTestId) or, better, an accessible
query like getByRole if applicable instead of
container.querySelector('[class*="modalBackdrop"]'). This removes reliance on
CSS-module class names and makes the test more robust.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@dashboard/app/`(dj)/setbuilder/components/__tests__/ImportModal.test.tsx:
- Line 144: Update the backdrop selection to avoid coupling to CSS class names:
add a stable attribute (e.g., data-testid="modal-backdrop") to the backdrop
element in the ImportModal component (ImportModal.tsx) and change the test in
ImportModal.test.tsx to use the testing-library query
(getByTestId('modal-backdrop') or screen.getByTestId) or, better, an accessible
query like getByRole if applicable instead of
container.querySelector('[class*="modalBackdrop"]'). This removes reliance on
CSS-module class names and makes the test more robust.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: d9fd1d67-6204-4fbb-9775-f6680ce43175

📥 Commits

Reviewing files that changed from the base of the PR and between 9f31c24 and 4bb5c39.

📒 Files selected for processing (1)
  • dashboard/app/(dj)/setbuilder/components/__tests__/ImportModal.test.tsx

…links)

- keep both additive sections in schemas/setbuilder.py, api.ts, api-types.ts, builder page imports
- re-anchor migration 053 onto 054 (share-token migration merged via PR #413)
- regenerate openapi.json + api-types.generated.ts from merged backend
- verified: alembic upgrade head + check clean, backend 2638 passed (87.64%), frontend tsc + 1037 tests green
@thewrz thewrz merged commit 75050c0 into main Jun 9, 2026
11 checks passed
@thewrz thewrz deleted the feat/issue-388 branch June 9, 2026 23:01
thewrz added a commit that referenced this pull request Jun 9, 2026
#414 merged to main (migration 053, pool feature) after #413 (054), making
main's chain 052 -> 054 -> 053. Resolved additive conflicts (curve + pool +
share sections coexist) in setbuilder schemas/router, models __init__, api
client, api-types, and the builder page. Regenerated openapi.json and
api-types.generated.ts. Re-anchored 055 down_revision -> 053; verified
single head and alembic upgrade/check on a fresh database
(052 -> 054 -> 053 -> 055).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

WrzDJSet: Pool import — 4 sources with source tagging + dedupe

1 participant