Skip to content

Rework mood boards as a universe style-building tool: persisted universe link, gallery/upload pickers, video items, prompt-from-media analysis, and style synthesis #4188

Description

@atomantic

Problem / Goal

Mood boards today are a dead end: items can only be added by pasting a URL, typing a note, syncing Pinterest, or pinning from the media UI — there is no gallery browse, no upload, and no real video support. And the universe↔mood-board "reference" is a localStorage-only glance widget (MoodBoardReferenceStrip), so a board never actually contributes to anything: board contents never reach a prompt, and the association isn't persisted, synced, or per-universe.

Make the mood board a tool within universes for constructing the universe's visual style: collect images and videos onto a board (from the shared gallery or by upload), analyze each item into positive/negative prompt descriptions, then synthesize the board into a main style prompt + negative prompt and adopt it into the universe's style fields.

Context

Mood boards (server/services/moodBoard/, server/routes/moodBoard.js, client/src/pages/MoodBoardDetail.jsx):

  • Item shape (normalizeItem, server/services/moodBoard/logic.js:152): { id, type, mediaKey, imageUrl, text, caption, source, createdAt }; type is only 'image' | 'text' (server/lib/moodBoardValidation.js:11). mediaKey already accepts video:<ref> but such items are stills — the renderer is a hard <img> (MoodBoardDetail.jsx:436) and moodBoardItemSrc.js can't derive a video thumbnail; playback is impossible.
  • Add-item UI is a two-tab form (Image URL / Note) — no picker, no upload (MoodBoardDetail.jsx:335-421). There is no mood-board upload endpoint; the only local-byte path is the Pinterest downloader.
  • Fully federated: kind moodBoard, sync category moodBoards (server/lib/schemaVersions.js:382, version 1), whole-board LWW merge, asset manifest already ships image: and video: keyed files (server/services/sharing/peerSyncAssets.js:494-527).

Universe link: no moodBoardId exists anywhere. client/src/components/universeBuilder/UniverseBibleTab.jsx:169 renders <MoodBoardReferenceStrip storageKey="universe-builder" /> — per-browser localStorage, read-only, shared across all universes.

Universe style fields (server/services/universeBuilder/sanitize.js:757-800): influences.embrace[] (positive style tokens), influences.avoid[] (negative), styleNotes, styleReferences[], with AI-edit locks. Consumed at render time by compilePrompts (server/services/universeBuilder/compile.js:152-163). The analyze→diff→adopt pattern already exists in UniverseStyleReferences.jsx + addStyleReference (server/services/universeBuilder/crud.js:727-748), which deliberately does the read-modify-write server-side inside the record's write queue (#3109) — the synthesis "adopt" must follow the same pattern.

Building blocks ready to reuse:

  • GalleryImagePicker (client/src/components/imageGen/GalleryImagePicker.jsx:54) and GalleryVideoPicker (client/src/components/videoGen/GalleryVideoPicker.jsx:23) — identical { open, onClose, onSelect, allowUpload } contracts. Image upload goes to POST /api/image-gen/upload → peer-syncable /data/images/. Video upload goes to POST /api/uploadsdata/uploads/, which does NOT federate and 404s on a peer — a mood-board video upload needs a syncable destination.
  • PromptFromMedia / PromptFromMediaModal (client/src/components/media/PromptFromMedia.jsx) → POST /api/media-jobs/prompt-from-media (server/services/mediaPromptFromMedia.js:269): vision-provider-gated, handles gallery images, gallery videos (ffmpeg frame sampling, max 5), and uploads; returns { imagePrompt, imageNegativePrompt, videoPrompt, videoNegativePrompt, rationale }.
  • Related: Migrate music/Authors gallery-upload callers onto GalleryImagePicker allowUpload #4127 migrates other bespoke upload paths onto GalleryImagePicker allowUpload.

Proposed approach

Design decision — "subsidiary" means link + embed, not a data move. Mood boards stay standalone federated records (preserving Pinterest sync, PinToMoodBoardMenu, peer sync, and boards that predate/outlive any universe). The universe gains a persisted pointer and the board editor becomes a tool surfaced inside the universe builder. The standalone /mood-boards pages remain as the board manager.

Phase 1 — persisted universe↔board link.

  • Add optional moodBoardId (nullable string) to the universe record: sanitize.js (schema parity, kept on wire — the id is meaningful on peers since boards federate), accepted on create/PATCH, no schema-version bump (additive optional field).
  • Universe Bible tab: replace the localStorage strip with a persisted picker (choose existing board or create one named after the universe) + the read-only thumbnail strip driven by the record field. Pipeline.jsx's strip usage keeps localStorage behavior.
  • Deep link "Open board" → /mood-boards/:id.

Phase 2 — gallery picker + uploads + real video items.

  • Mood board detail add-item flow gains "Pick from gallery" (GalleryImagePicker allowUpload) and "Pick video" (GalleryVideoPicker allowUpload) alongside the existing URL/Note tabs; selections store mediaKey + imageUrl preview exactly as PinToMoodBoardMenu does.
  • Add 'video' to MOOD_BOARD_ITEM_TYPES, normalizeItem, the Zod schemas, and moodBoardItemSrc; render video items with a poster thumbnail + playback (lightbox or inline <video>).
  • Route mood-board video uploads to a peer-syncable destination: add a gallery-video upload path (parallel to saveUploadedGalleryImage) that lands the file under PATHS.videos with a history record, so the resulting video:<ref> key federates through the existing buildBoardAssetManifest — do not use /api/uploads for board-referenced media.
  • Bump moodBoards schema version to 2 with the sync bundled-doc checklist: a type:'video' item is an execution-semantics change for older receivers.

Phase 3 — per-item prompt analysis.

  • Reuse PromptFromMediaModal from a board item's card/lightbox; persist the result on the item as an optional analysis: { prompt, negativePrompt, rationale, providerId, model, analyzedAt } (item PATCH schema extension; additive on wire).
  • Show analyzed state on the card; analysis is per-item and user-triggered (vision provider picker, per the AI Provider Usage Policy — no batch auto-analysis).

Phase 4 — board → universe style synthesis.

  • New endpoint POST /api/mood-boards/:id/synthesize-style (user-triggered, provider/model selectable): feeds item analyses + captions + notes + board description to an LLM and returns { embrace[], avoid[], styleNotes? } shaped for influences (respecting the token caps at sanitize.js:97-99).
  • UI on the universe's mood-board tool: "Synthesize style" → show a StyleDiff-style preview against current influences → "Adopt" calls a server-side queued-write adopt (mirroring addStyleReference's { adopt } path — never a client wholesale influences PATCH), honoring the locked.influencesEmbrace/Avoid/styleNotes locks.

Phases are ordered so each lands independently — Phase 1–2 can ship and the rest spin off into follow-up issues if scope demands.

Acceptance criteria

  • Universe record persists moodBoardId; picking/creating a board in the universe builder survives reload and syncs to peers; the localStorage-only strip is gone from UniverseBibleTab.
  • Mood board detail can add items from the shared image gallery, the video gallery, and via upload of both images and videos; uploaded files land in peer-syncable locations (/data/images/, PATHS.videos) — never data/uploads/.
  • type:'video' items render with poster + playback; moodBoards schema version bumped and version-gated so older peers aren't corrupted.
  • A board item can be analyzed via the existing prompt-from-media pipeline and the resulting positive/negative prompts persist on the item and display on the card.
  • "Synthesize style" produces an embrace/avoid/styleNotes proposal with a diff preview; "Adopt" updates the universe via a server-side queued write and respects field locks.
  • All LLM/vision calls are user-triggered with a visible provider picker (no background batches).
  • Tests: validation for the new item type and analysis field, synthesis endpoint, universe moodBoardId parity, and peer fan-out coverage for the new record fields; changelog fragment added.

Out of scope

  • Removing the standalone /mood-boards pages or the Pinterest integration.
  • Multiple mood boards per universe (single moodBoardId first; revisit if needed).
  • Auto-analyzing Pinterest-synced or bulk-added items (violates the no-silent-batch AI policy).
  • Item reorder/drag/zoom on the board canvas.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

planTracked by /do:replan

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions