feat: Unified Story Builder + universe style-probe + arc reader map#514
Merged
Conversation
Add docs/plans/ as a repo record of feature design plans (one file per plan, dated). Seed it with the Unified Story Builder design and a docs/plans/README.md. Add a CLAUDE.md Git Workflow convention so future approved plans get copied from ~/.claude/plans/ into docs/plans/ before implementation.
Add /story-builder — a guided, linear conductor over the existing Universe/Series/Issue records that walks a story from idea -> universe aesthetic -> plot arc -> reader map -> characters -> issues -> production. Each step is LLM-assisted with an AI-refinement affordance and ends with an explicit lock before the next unlocks; revising an earlier step soft-flags downstream locked steps as stale (integrity gate) via a sha256 hash of each step's upstream inputs, without destroying content. Heavy per-issue production hands off to the existing Pipeline issue page. New series.arc.readerMap field: the audience-experience roadmap (hooks, payoffs, emotional beats, cliffhangers) built on the Vonnegut shape, distinct from the protagonist arc. generateReaderMap/refineReaderMap in arcPlanner.js; arc-overview regeneration now preserves it. Lockable via the existing arcFields mechanism. pipelineSeries wire-schema bumped 1->2 (per-category gate) so an older peer can't round-trip and strip it. Server: services/storyBuilder.js (local-only session store + state machine + integrity-on-read + generate/refine delegation), routes, lib/storyBuilderSteps.js + lib/storyBuilderIntegrity.js, Zod schemas, 3 prompt stages + migration 043. Client: pages/StoryBuilder.jsx stepper, services/apiStoryBuilder.js, App/Layout/navManifest wiring. Seed-mode v1; import-mode intake, in-builder issue seeding, SSE streaming, arc refine-with-feedback, and inline ArcCanvas embedding tracked in PLAN.md.
Data-loss fixes (the reader map / arc could be silently destroyed):
- arcSchema (routes/pipeline.js) now lists readerMap; Zod was stripping it
on every arc PATCH, then updateSeries's wholesale arc replace wiped it.
Regression test added (PATCH /series/:id preserves arc.readerMap).
- generateReaderMap/refineReaderMap throw on an empty LLM payload instead of
returning null; refineReaderMap also falls back to the existing map — a
refine (or generate) that yielded nothing no longer nulls out the saved
reader map. Regression tests added.
- generateStep('plotArc') refuses a null arc rather than writing
updateSeries({ arc: null }) and wiping the whole arc.
- characters step no longer blanket-toggles locked on every universe
character (clobbered per-entry locks + mutated a shared universe); the
session step-lock alone gates the wizard, UI hides per-entry refine.
UX fixes:
- StoryBuilder reload() passes { silent: true } to the universe/series/issues
GETs (added an options param to those wrappers) so a failed fetch no longer
double-toasts over its own .catch fallback.
- <StepPanel> keyed by stepId so RefineBox feedback resets between steps.
- Step manifest now loads before the loading gate clears, so the detail view
never renders an empty step rail.
Deferred (non-blocking, in PLAN.md): createStorySession orphan-universe
rollback, goToStep gate-divergence revert.
The intro screen now has two tabs: 'Start from an idea' (seed) and 'Import a finished work'. Import reuses the existing Importer pipeline — analyzeImport (creates the universe+series, extracts canon/arc/seasons/issues) → preview summary → commitImport (all canon + arc + seasons + issues) → createStorySession in intake='import' mode → drops into the wizard with the stages pre-filled to review and lock. Handles the comic-script mechanical split, an empty-split 'Retry issue split' affordance, and existing-universe merge. Seeds the session seedIdea from the extracted arc summary so the aesthetic step's Expand has a real starter. Fixes a React anti-pattern found while testing: the tab buttons were a component defined inline in the index render (new type each render), so the mode toggle didn't take — inlined the buttons. Tests: import analyze→preview→commit→create flow + the no-issues retry gate. Import aesthetic/reader-map pre-fill gaps tracked in PLAN.md.
Add a provider/model picker (reuses getProviders + filterSelectableModels) at the top of the Story Builder — in the detail header and the import tab. The choice persists on session.llm, and the conductor's generate/refine now default provider/model from session.llm when no explicit per-call override is given, so one selection applies to idea expand, aesthetic, plot arc, reader map, character refine, AND the importer's analyze/retry. The import flow threads the picked provider into analyzeImport/retryImporterIssues and stores it on the created session. - server: generateStep/refineStep resolve reqProviderId/reqModel = options.* || session.llm.* ; storySessionCreateSchema accepts llm. - client: ProviderModelPicker component; import panel local state + detail header persists via updateStorySession; inline tab buttons. Tests: server session.llm-default + explicit-override-wins; client provider threading on import + detail-picker persistence. server 8052 / client 592.
On a successful LOCK, the lock-toggle's onSuccess now advances the current step to the next one (persists the pointer + navigates), so the user no longer has to click Lock and then Next. Unlocking stays put. The standalone Next button remains for moving between already-locked steps.
…exists Each step's generate button flips to 'Re-generate' (with a refresh icon) once that step has generated content — idea (universe logline set), universe aesthetic (premise/styleNotes set), plot arc (arc logline/summary), reader map (readerMap present) — so it's clear the action was already run.
The characters step now renders a preview image per character so the world style + character style can be eyeballed together. Reuses the Universe Builder's exact render path — composeStyledPrompt + universeStylePreset → generateImage → EntryThumbSlot (diffusion spinner via MediaJobThumb, then the image, walk-back on 404) — and persists the resulting filename onto the universe canon entry's imageRefs (section-local renders don't get the server-side append hook). Honors the per-builder provider/model picker and the user's image-gen settings (readPipelineImageSettings). - EntryThumbSlot: optional onComplete pass-through to MediaJobThumb.onFilename. - apiSystem.generateImage: accept an options arg (so the call can be silent alongside its own catch toast). Tests: characters step renders the per-character slot + the generated prompt fuses the descriptor with the universe style. client 595.
…Builder) Add a 'base style image' that renders from the universe's raw style guide alone — styleNotes (style guide) + influences embrace (positive) / avoid (negative), with NO subject — so the user can see the world's base visual emphasis. New StyleProbeImage component (reuses generateImage + EntryThumbSlot + universeStylePreset); buildStyleProbePrompt/hasStyleForProbe helpers. - server: universe gains a styleImageRefs[] field (sanitizeTemplate + createUniverse field list + updateUniverse PATCHABLE_SCALARS, sanitized via sanitizeEntryImageRefs); create + patch Zod schemas accept it. Additive, not wire-version-gated (low-stakes, one-click regenerate). - client: rendered under the style/influences editor in UniverseBuilder (merges only styleImageRefs into the draft) and in the Story Builder Universe Aesthetic step; generateImage gained an options arg already. Tests: prompt-composition helpers (client) + styleImageRefs round-trip, dedupe-on-create, default-[] (server). server 8054 / client 600.
- StepCharacters preview persist: guard against MediaJobThumb's onFilename firing more than once (processedRef per char+filename) and refetch the freshest universe before the imageRefs append, so a sibling character's just-persisted ref isn't clobbered by a stale full-array PATCH. Also dedupe the appended filename. - StyleProbeImage: same multi-fire guard, dedupe, and only reflect the new styleImageRefs once the save actually succeeds (was optimistically showing an image the server may not have persisted). - Lock & continue auto-advance: navigate only AFTER setStoryCurrentStep resolves (don't strand the URL ahead of the persisted pointer on a gate rejection) + defensive !isStale guard. - saveLlm: merge the picker choice into local session state instead of a full reload() (avoids refetching universe+series+issues and flicker for an llm-only change). Deferred to PLAN.md: import partial-commit recovery, styleImageRefs wire/ conflict-journal trade-off, and extracting the duplicated image-render hook / provider picker. client 600.
Local review surfaced six data-loss / silent-failure bugs the prior xhigh
passes missed plus three tightening items:
- generateStep('plotArc') was doing updateSeries({ arc, seasons }) which
bypassed mergeArcWithLocks / mergeSeasonsWithLocks / buildSeasonRemap —
silently dropping per-field arc locks, locked seasons, and orphaning
every child issue attached to a renamed/removed season. Route through
commitSeasonsWithRemap (the Arc Canvas's own helper).
- resolveVerifyIssues (Arc Canvas auto-resolve) wasn't forwarding
series.arc.readerMap into its sanitizeArc call. Because the field has
no per-field lock by default, mergeArcWithLocks didn't restore it on
commit, so a user who'd generated a reader map without locking it lost
it the first time they auto-resolved a finding. Mirrors the fix already
present in generateArcOverview. Regression test added.
- Reader-map prompt JSON example used "kind": "hook|reveal|payoff|..."
which LLMs reproduce literally; sanitizeReaderBeat then drops every
beat because the joined string isn't in READER_MAP_BEAT_KINDS, leaving
beats: [] and tripping the empty-payload check. Use a single valid
example ("hook"); the existing {{beatKindsCsv}} clause enumerates the
vocabulary.
- currentStep PATCH was z.string().max(40); sanitizer silently coerced
unknown values to STEP_IDS[0]. Now z.enum(STEP_IDS) — sanitizer's
coerce-on-LOAD stays for resilience against corrupted persisted state.
- styleImageRefs route caps (.max(50) on POST + PATCH) were 4× the
sanitizer cap (IMAGE_REFS_PER_ENTRY_MAX = 12), so a 40-entry POST
silently 200'd with 28 dropped. Align to entryImageRefsField.
- storyBuilderStore() registered in the boot-time verifyCollectionVersions
array.
- reachable() in StoryBuilder.jsx returns 'unlocked' | 'stale' | true so
the "earlier steps" toast can distinguish staleness from unlocked when
the disabled-button gate fires. Booleanized at the sidebar disabled
prop so the truthy string doesn't re-enable a blocked button.
Deferred to PLAN.md (non-blocking): step-content staleness (lock vs
hash semantics), refineReaderMap providerId/model return, refineReaderMap
fallback changes/rationale, hash-shape vs derivation test, migration-043
line-ending precedent, StyleProbeImage draft-vs-saved divergence, second
swallow-rejection path in lock.toggle onSuccess.
server 8055/8055 + 7 skipped, client 600/600.
…ale hash
Two P2 findings from codex's review of the PR:
- generateStep('idea') at storyBuilder.js:441-443 was writing both
starterPrompt AND logline via a plain updateUniverse(). After the user
locks the universeAesthetic step (which sets universe.locked.{logline,
premise,styleNotes,influencesEmbrace,influencesAvoid}=true), re-running
the idea step would silently overwrite the locked logline because
updateUniverse doesn't enforce per-field locks on scalar writes. Skip
any patched scalar that's locked on the current record.
- buildUpstreamInputs() at storyBuilder.js:270-272 was tracking only
session.seedIdea for the universeAesthetic / plotArc steps. But the
idea expand is non-deterministic — same seed can produce different
universe.starterPrompt + series.logline/premise, all of which feed
the aesthetic and plotArc generators. A locked aesthetic / plotArc
step would NOT flag stale after re-running idea with the same seed,
leaving the wizard advancing on locks derived from older expanded-idea
outputs. Fix: include the idea-step OUTPUTS (universe.starterPrompt,
series.logline, series.premise) in the upstream hash inputs for both
downstream steps.
Two regression tests added:
- storyBuilder — generate delegation: generateStep(idea) skips writing a
locked universe.logline (asserts the lock-honoring fix).
- storyBuilder — integrity / staleness: flags a locked universeAesthetic
stale when the idea step re-runs with a new starterPrompt (asserts the
stale-hash fix).
server 8057/8057 + 7 skipped, client 600/600.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new guided Story Builder that walks a story from idea → universe aesthetic → plot arc → reader map → characters → issues → production as one linear flow, plus the two model-side primitives it leans on. The builder is a thin conductor over existing Universe / Series / Issue records — no duplicated data, heavy per-issue production hands off to the existing Pipeline issue page.
Builder UX
/story-builderpage with two intake modes:createUniverse+createSeriesshells, LLM expand for aesthetic, arc, reader map)session.llmso one selection applies throughoutui_navigate)Model primitives (also independently useful)
series.arc.readerMap: a distinct audience-experience roadmap on the arc — hooks, payoffs, emotional beats, cliffhangers — built on the Vonnegut shape, separate from the protagonist arc.generateReaderMap/refineReaderMapinarcPlanner.js; preserved by arc-overview regen and (now) by auto-resolve.pipelineSerieswire-schema bumped 1→2 so an older peer can't round-trip and strip it.universe.styleImageRefs[](base style probe): render an image from the raw style guide alone — styleNotes + influences embrace/avoid as positive/negative prompt, no subject — to preview the world's base visual emphasis. Triggerable from both the Universe Builder (under the style/influences editor) and the Story Builder Universe Aesthetic step; persists on the universe so both surfaces share it.Local code-review fixes (data-loss class)
Two prior xhigh code-review passes already landed (commits 47a56c9 and e1c0fd0). A third local-review pass (orchestrated for this PR) surfaced six more data-loss / silent-failure bugs they missed, all fixed in c085a11:
generateStep('plotArc')was wholesale-replacing seasons via plainupdateSeries({ arc, seasons })— silently wiped per-field arc locks, locked seasons, and orphaned every child issue attached to a renamed/removed season. Now routes throughcommitSeasonsWithRemap(the same helper the Arc Canvas regen uses).resolveVerifyIssues(Arc Canvas auto-resolve) was missingreaderMapin its sanitizeArc forward — a user who'd generated a reader map without locking it lost the entire map on the first auto-resolve. Regression test added.kindwhich LLMs reproduce literally;sanitizeReaderBeatthen dropped every beat. Replaced with a single valid example.currentStepPATCH schema tightened toz.enum(STEP_IDS)so stale-client posts of removed step ids 400 instead of silently coercing to step 0.styleImageRefsroute caps aligned with the sanitizer cap (was 4× higher, so over-cap requests silently dropped entries).storyBuilderStore()added to the boot-timeverifyCollectionVersionsarray.reachable()in StoryBuilder.jsx now discriminates'unlocked' | 'stale' | trueso the toast says the right thing when staleness (not unlocked) is the gate.Items deliberately not in this PR are tracked in PLAN.md under the existing
story-builder-*keys (orphan-universe rollback on seed-mode create failure, two swallow-rejection paths insetStoryCurrentStep, refine-with-feedback for plotArc, ArcCanvas inline embed, SSE streaming, import partial-commit recovery, image-render hook extraction, optional cross-machine session sync, step-content staleness UX, refineReaderMap provider/model return, refineReaderMap fallback changes, hash-shape vs derivation test, migration-043 line-ending precedent, StyleProbeImage draft-vs-saved divergence).Notable
pipelineSeriesv1→v2 inRECORD_KIND_SCHEMA_CATEGORIES.series, so federations on mixed versions don't round-trip-strip the newreaderMapfieldstage-config.jsonentries on existing installs — boot's migration runner picks it up so plain pull+pm2-restart upgrades work withoutnpm run setup:data/story-builderso ⌘K + voice agent both find itserver/lib/file is registered inindex.js+README.md(server/lib/index.test.jsenforces this)storyBuilder.jsheader comment). Cross-machine resumable sessions tracked in PLAN.md as a v2 option.Test plan
cd server && npm test— 8055 passed / 7 skipped, 363 filescd client && npm test -- --run— 600 passed, 59 filesresolveVerifyIssuespreservesseries.arc.readerMapwhen the LLM omits itserver/services/storyBuilder.test.js,server/routes/storyBuilder.test.js,server/lib/storyBuilderIntegrity.test.js,client/src/pages/StoryBuilder.test.jsx,client/src/components/universe/StyleProbeImage.test.jsx)rm data/prompts/stages/story-builder-reader-map.md data/prompts/stages/story-builder-reader-map-refine.mdthen restart — migration 043 re-copies the fixed templates fromdata.reference/. Fresh installs are unaffected.