Skip to content

feat([issue-4188]): mood board items gain persisted prompt-from-media analysis - #4234

Merged
atomantic merged 4 commits into
mainfrom
next/issue-4188
Aug 15, 2026
Merged

feat([issue-4188]): mood board items gain persisted prompt-from-media analysis#4234
atomantic merged 4 commits into
mainfrom
next/issue-4188

Conversation

@atomantic

Copy link
Copy Markdown
Owner

Summary

Phase 3 of #4188 (mood boards as a universe style-building tool): board items backed by gallery media can now be analyzed with the Prompt-from-media flow, and the result persists on the item.

  • Analyze with AI on board items — gallery-backed items (an image:<file> media-key, a gallery-shaped imageUrl, or a video:<filename> media-key) gain a ScanEye action that opens PromptFromMediaModal seeded with the item's media. The analyzer is unchanged for its existing hosts; it gains an optional onResult callback (threaded through the modal along with kindDefault/children) so a host can persist a successful run.
  • Persisted analysis on the item — a run auto-saves { prompt, negativePrompt, rationale, providerId, model, analyzedAt } onto the item via the item PATCH (moodBoardItemAnalysisSchema, media items only, null clears; analyzedAt stamped server-side when absent). Additive on the wire — whole-record LWW spreads unknown keys and older peers preserve them (same precedent as the pinterest sub-object), so no moodBoards gate bump. The card shows an accent-highlighted analyzed state; the modal shows the saved analysis with copy/remove.
  • Gallery video resolves by filenamePOST /api/media-jobs/prompt-from-media accepts sourceKind:'video' with a filename in place of the history videoId (board video items store the on-disk filename, not an id). The extraction id is stable per clip (pfm-vf-<stem>), so re-analysis overwrites its sampled frames instead of accumulating sets under data/video-thumbnails. The gallery flow's id path is unchanged.
  • Analysis-source resolver (moodBoardItemAnalysisSource) mirrors the server's imageUrlToAppAsset contract: legacy bare gallery refs and query/hash-suffixed /data/images URLs normalize to the on-disk basename; external URLs, non-gallery paths, text items, and legacy video:<id> pins on image items are not analyzable (no button).

Analysis is per-item and user-triggered with an explicit vision provider/model picker — no batch auto-analysis (AI Provider Usage Policy).

A codex review pass surfaced two P2s, fixed in the third commit: the stable frame-extraction id (was a fresh UUID per run → unbounded thumbnail accumulation) and the legacy/query-string gallery-URL normalization (those items previously got no analyze action, or sent the query string as part of the filename).

Remaining phase of #4188 (board → universe style synthesis) is not in this PR — see the reconciliation comment on the issue.

Refs #4188

Test plan

  • server/lib/moodBoardValidation.test.js — analysis PATCH cases: full/minimal accepted, null clears, rejects missing prompt / unknown keys / non-ISO analyzedAt.
  • server/services/moodBoard/logic.test.js — normalized persistence (explicit nulls + stamped analyzedAt), client-provided analyzedAt kept, other patches leave it untouched, null clears, ignored on text items.
  • server/services/mediaPromptFromMedia.test.js — filename-based video resolution with the stable pfm-vf-<stem> extraction id and no history lookup; extension-less filename rejected before any ffmpeg work.
  • server/routes/mediaJobs.test.js — video source with neither videoId nor filename 400s.
  • client/src/lib/moodBoardItemSrc.test.js — analysis-source resolution: video-by-filename with poster, image by media-key/app path (decoded), legacy bare refs, query/hash stripping before basename, and the null cases.
  • client/src/components/media/PromptFromMedia.test.jsxonResult fires with the payload; a filename-only clip sends no videoId.
  • client/src/pages/MoodBoardDetail.test.jsx — analyze action only on gallery-backed items, persist PATCH payload and analyzed-state flip, saved-analysis view, remove via analysis: null.
  • Touched suites green locally; biome check clean. Full-suite runs were degraded by extreme machine load (load average ~280) — every failing file also fails identically on main and none are touched by this change; CI is the authoritative gate.

Board items backed by gallery media gain an Analyze with AI action that runs
the video page's Prompt-from-media flow and persists the result on the item
as an optional analysis field (prompt, negative, rationale, provider/model,
analyzedAt) via the item PATCH — additive on the wire, so no moodBoards gate
bump (whole-record LWW preserves unknown keys, same precedent as the
pinterest sub-object). The prompt-from-media endpoint now also resolves a
gallery video by on-disk filename, matching the video:<filename> refs board
items carry (the gallery flow's history-id path is unchanged).
… gallery-URL normalization

A filename-based video analysis now derives its extraction id from the clip's
filename stem, so re-analyzing the same board item overwrites its sampled
frames under data/video-thumbnails instead of accumulating a new five-frame
set per run (matching the history-id path's stable pfm-<id> key). The client
analysis-source resolver now mirrors the server's imageUrlToAppAsset
contract: legacy bare gallery refs and query/hash-suffixed /data/images URLs
reduce to their on-disk basename instead of being unanalyzable or sending
the query string as part of the filename.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant