feat(universe-builder): Phase B — enrich expand contract with canon + plumb canon into arc prompts#267
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR enriches Universe Builder expansion output with first-class canon entities and makes linked-world canon available to downstream arc/volume prompt contexts.
Changes:
- Expands the server/client “Generate From Idea” contract to include characters, settings, objects, and category
kind. - Adds canon prompt rendering and wires
worldCanonTextinto arc planner context/templates. - Adds migration/setup-data hash handling and tests/documentation for the new contract.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| server/services/universeBuilderExpand.js | Adds canon arrays and category kind guidance to expansion output. |
| server/services/universeBuilderExpand.test.js | Adds tests for canon normalization and prompt contract text. |
| server/lib/universePromptRenderers.js | Adds canon-to-prompt rendering helper. |
| server/services/pipeline/arcPlanner.js | Adds worldCanonText to linked-world context. |
| server/services/pipeline/arcPlanner.test.js | Verifies canon context flows into arc planner calls. |
| server/routes/universeBuilder.js | Updates route docs for expand response shape. |
| client/src/pages/UniverseBuilder.jsx | Merges returned canon into the draft and auto-save payload. |
| data.sample/prompts/stages/pipeline-arc-resolve.md | Adds World canon block to resolve prompt. |
| data.sample/prompts/stages/pipeline-volume-verify.md | Adds World canon block to volume verification prompt. |
| scripts/migrations/019-arc-verify-resolve-canon-context.js | Adds hash-gated prompt template migration. |
| scripts/setup-data.js | Updates shipped prompt hashes for drift detection. |
| PLAN.md | Marks Phase B canon context work as folded in and records follow-ups. |
| .changelog/NEXT.md | Adds user-facing release notes for expansion canon and prompt canon context. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (2)
client/src/pages/UniverseBuilder.jsx:713
- This auto-save path has the same stale-merge problem as manual save: it refetches the server canon, then merges the draft's full canon arrays back in. Any concurrent delete or rename that happened while the LLM call was running can be undone because the old draft entry is treated as a new non-colliding addition. Merge only the entries introduced by this expand result, rather than the whole
expandedDraftcanon.
characters: mergeCanonByName(fresh.characters || [], expandedDraft.characters || [], 'character'),
settings: mergeCanonByName(fresh.settings || [], expandedDraft.settings || [], 'setting'),
objects: mergeCanonByName(fresh.objects || [], expandedDraft.objects || [], 'object'),
client/src/pages/UniverseBuilder.jsx:946
- While preserving pending expand additions, this also merges the draft's entire stale canon back over the server response. If the server response intentionally removed or renamed an entry (for example from another tab) while
canonDirtyis true, the old draft entry can be appended again because it no longer collides. Keep a list of the pending expand-created entries and merge only those intoupdated.
characters: mergeCanonByName(updated.characters || [], d.characters || [], 'character'),
settings: mergeCanonByName(updated.settings || [], d.settings || [], 'setting'),
objects: mergeCanonByName(updated.objects || [], d.objects || [], 'object'),
…rrays + plumb canon into arc prompts Universe Builder "Generate From Idea" now returns rich first-class canon alongside categories: - characters[]: name, physicalDescription, personality, background, prompt, tags - settings[]: name, slugline, description, palette, recurringDetails, ... - objects[]: name, description, significance, prompt, tags The expand LLM is also taught to tag each category with `kind` so the Phase A data model lands them under the right canon trunk in the upcoming UI. Client handleExpand merges canon into draft (existing entries always win on name/ slugline collision — server-side dedupe via sanitizeBibleList). arcPlanner now feeds named canon into LLM prompts via a new worldCanonText context field (sibling to worldCategoriesText). Migration 019 auto-updates the pipeline-arc-resolve and pipeline-volume-verify templates on launch for unmodified installs; customized prompts get a manual-merge warning.
Also forward LLM-returned category `kind` through normalizeCategories so custom buckets land under the right canon trunk; use normalizeSlugline for settings dedupe so dash/punct-variant sluglines collide; rename migration 019 to reflect the files it actually updates (pipeline-arc-resolve.md + pipeline-volume-verify.md, not arc-verify).
… in all arc/volume prompts
Five Copilot follow-ups:
- handleExpand: build mergedCategories with both 'kind' (existing draft wins,
LLM-returned kind as fallback) and 'variations', so the LLM/user-tagged
bucket trunk survives the round-trip through ensureDraftCategories.
- arc-overview.md + arc-verify.md: render {{worldCanonText}} so the canon
context fed by arcPlanner actually reaches the prompt (previously only
arc-resolve + volume-verify referenced it).
- Migration 019 now updates all four arc/volume templates, renamed to
019-arc-volume-prompts-canon-context.js; setup-data.js hash bumps mirror.
- normalizeCanonArray strips LLM-supplied id/createdAt/updatedAt before
sanitize so a hallucinated/example id can't introduce duplicate canon ids.
- mergeCanonByName is kind-aware — settings use normalizeSlugline for both
name and slugline (matches storyBible MERGE_CONFIG.setting.keyFields), so
dash/punct-variant identifiers collide instead of duplicating.
…oast + complete changelog Four Copilot follow-ups: - normalizeCanonArray strips locked/sourceSeriesId/imageRefs/primaryImageRef alongside the previously-stripped id/timestamps. Without this, a hallucinated 'locked: true' from the LLM would silently lock new canon entries (blocking user edits via the lock UI), and stale sourceSeriesId/ imageRefs would falsely attribute provenance + pin visuals. - expandToast now reports NEW canon entries added by this expand (post-merge minus pre-existing), not the draft's running total. Re-expanding on a populated universe no longer claims credit for entries the user authored. - mergeCanonByName is alias-aware for character/object kinds, matching the server's MERGE_CONFIG keyFields. Existing 'Ashley' with alias 'Ash' collides with an LLM-returned 'Ash' instead of duplicating. - Changelog NEXT.md names all four templates updated by migration 019 (arc-overview, arc-verify, arc-resolve, volume-verify), not just the two originally listed.
…reate on expand for unsaved drafts
… boundary; gate Linked World prompt block on hasLinkedWorld; add migration 019 test
…rompt rendering, drift-catch + OLD→NEW migration test via applyMigration helper
… enrich canon renderer with personality/slugline/recurringDetails, fix mergeCanonByName within-batch dedupe, correct PLAN doc
…e on setSaving to prevent double-submit, exclude *.test.js from migration runner, refresh stale comment
…e payload, drop premature cap in characters-bucket fold, capture canon-clobber risk in PLAN
…plus manual Save still persists merged canon
…ldRetiredCharactersBucket
…ory kind on updateCategory + handleGenerateInCategory
… edits (other tabs, NounsStage) aren't clobbered
…pand; defer canon-deletion-revert edge case to PLAN
atomantic
force-pushed
the
feat/ub-redesign-phase-b
branch
from
4eb7602 to
3003b88
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.
Comments suppressed due to low confidence (2)
client/src/pages/UniverseBuilder.jsx:717
- The refetch protects concurrent additions, but this merge still replays the entire stale draft canon onto the fresh server canon. If another tab deleted or renamed an existing canon entry while the expand request was running,
expandedDraft.characters/settings/objectsstill contains the old entry andmergeCanonByName(fresh, expandedDraft, ...)will re-add it in the auto-save payload. This should merge only the entries newly returned by this expand (or track an additions ledger), not the full draft arrays.
canonForPayload = {
characters: mergeCanonByName(fresh.characters || [], expandedDraft.characters || [], 'character'),
settings: mergeCanonByName(fresh.settings || [], expandedDraft.settings || [], 'setting'),
objects: mergeCanonByName(fresh.objects || [], expandedDraft.objects || [], 'object'),
};
client/src/pages/UniverseBuilder.jsx:950
- While
canonDirtyis set, every canon-section update is merged with the entire previous draft canon. A delete/rename performed in the canon UI returnsupdatedwithout the old entry, butd.characters/settings/objectsstill includes it, so this local merge can immediately put the deleted/renamed entry back into the draft and a later save can persist it again. This should preserve only pending expand additions rather than merging the full stale draft arrays.
characters: mergeCanonByName(updated.characters || [], d.characters || [], 'character'),
settings: mergeCanonByName(updated.settings || [], d.settings || [], 'setting'),
objects: mergeCanonByName(updated.objects || [], d.objects || [], 'object'),
… reverted; normalize keys when folding retired characters bucket; fileURLToPath for path safety; corrected vitest glob comment
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.
Comments suppressed due to low confidence (1)
client/src/pages/UniverseBuilder.jsx:764
- If this refetch fails, the code falls back to
expandedDraft's full canon arrays and still sends them in the PATCH. Because canon arrays are replaced wholesale server-side, a transient GET failure can turn the auto-save into a stale-canon write that clobbers concurrent canon edits/deletions. For existing worlds, either abort the auto-save (leaving the user to Save) or omit canon unless the current server canon was successfully fetched and merged with the pending additions.
if (selectedId) {
const fresh = await getUniverse(selectedId).catch(() => null);
if (fresh) {
canonForPayload = {
// Merge ONLY pending additions onto refetched server canon
// (not the full stale draft). Without this, concurrent canon
// deletions in other tabs/surfaces get resurrected because the
// deleted entry is still present in the stale draft.
characters: mergeCanonByName(fresh.characters || [], pendingCanonAdditionsRef.current.characters, 'character'),
settings: mergeCanonByName(fresh.settings || [], pendingCanonAdditionsRef.current.settings, 'setting'),
objects: mergeCanonByName(fresh.objects || [], pendingCanonAdditionsRef.current.objects, 'object'),
};
}
}
…e from other; abort save on refetch fail; import migration hash tables in test - universeBuilderExpand.js + universeBuilder.js: fix isCharactersBucket to match variant keys like "character variations" / "Character_Variations" via /^characters?(_|$)/i — exact equality only caught "characters", not multi-word normalized forms (e.g. "character_variations") - UniverseBuilder.jsx kind merge: allow a fresh LLM kind to supersede an existing 'other' default so pre-Phase-B buckets saved as 'other' can be promoted on re-expand; user-curated non-'other' kinds are still preserved - UniverseBuilder.jsx handleSave + handleExpand: abort the save when the server refetch fails for an existing universe instead of falling back to the stale draft canon, which would clobber concurrent canon deletions/edits - 019 migration: export ACCEPTED_OLD_MD5 + NEW_SHIPPED_MD5; test now imports them directly so hash-table drift fails loudly; idempotency test asserts alreadyCurrent count via applyMigration with real tables
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase B of the Universe Builder redesign (PLAN.md Next Up #1). Builds on the Phase A schema (#264) by enriching the LLM "Generate From Idea" contract to return first-class named canon and plumbing that canon into downstream prompts.
characters[]/settings[]/objects[]with rich metadata (physicalDescription,palette,slugline,recurringDetails, etc.) alongside the existing categories + composite sheets. Each category also gets akindtag so the Phase C UI knows which canon trunk to render it under.handleExpandmerges returned canon into the draft'suniverse.characters/.settings/.objectsarrays. Existing entries always win on name/slugline collision so a re-expand can't clobber hand-authored or series-extracted records.worldCanonTextcontext field viarenderCanonForPrompt(world)— the LLM now sees named cast/places/objects alongside the existingworldCategoriesTextexploratory variations. Folds in the "arcPlanner prompt context — include canon" backlog item.pipeline-arc-resolve.md+pipeline-volume-verify.mdtemplates to include the new{{worldCanonText}}block when unmodified; customized prompts get a manual-merge warning (mirrors the migration-003 pattern).Files changed
server/services/universeBuilderExpand.js—buildExpansionPromptasks for canon +kind;isExpansionShaperecognizes canon arrays;normalizeCanonArray(raw, kind)runs LLM output throughsanitizeBibleListwithBIBLE_SOURCE.UNIVERSE_EXPANDstamped pre-sanitize.server/lib/universePromptRenderers.js— newrenderCanonForPrompt(world), table-driven per-kind formatting.server/services/pipeline/arcPlanner.js— wiresworldCanonTextintoloadWorldContext+EMPTY_WORLD_CONTEXT.client/src/pages/UniverseBuilder.jsx—mergeCanonByNamehelper (dedupe by name + slugline, identity-preserving on empty input);handleExpandmerges canon viapickCanon; toast dedupe viaexpandToasthelper.data.sample/prompts/stages/pipeline-{arc-resolve,volume-verify}.md— new "World canon" block above the categories block.scripts/migrations/019-arc-verify-resolve-canon-context.js— hash-driven one-shot template upgrade.scripts/setup-data.js— bumped shipped-MD5 entries (OLD becomes array of two hashes; NEW gets the post-019 hash).server/routes/universeBuilder.js— JSDoc update to advertise the new expand return shape.Test plan
cd server && npm test— 5108 passing, 5 skipped (was 5102 in Phase A, +6 new)cd client && npm run build— cleanEXPANSION_PROMPT contains characters/settings/objects + physicalDescription + slugline + significance(prompt contract)EXPANSION_PROMPT teacheskindenum on categories(LLM hint matches Zod enum)normalizeCanonArrayx4 (non-array → []; character → bible shape; setting requires name OR slugline; object requires name)worldCanonTextcontainsMira Holt+field detective+The Tongue(canon context flows into prompt); placeholder branch assertsnone/simplifyreview applied: HIGH ×1 (BIBLE_SOURCEconstant); MED ×4 (toast dedupe, table-driven renderer,mergeCanonByNameshort-circuit,pickCanonhelper); 1 MED deferred → PLAN.md (mergeExpandIntoDraftextraction)data/runs/<id>/output.txt) includes the World canon blockMigration safety
Migration 019 is unmodified-only —
data/prompts/stages/pipeline-arc-resolve.mdandpipeline-volume-verify.mdare replaced ONLY when their on-disk hash matches one of the two prior shipped hashes (pre-005 or current). Customized prompts get a warning + manual-merge instructions and stay unchanged. Idempotent: re-runs against the new hash are no-ops.Known deferred
mergeExpandIntoDraft(draft, result)extraction from the now ~155-linehandleExpand. Pure merge logic can be lifted out + unit-tested. Captured in PLAN.md under "Code quality / dedup".world.categorieswithoutworld.canon(PLAN.md backlog item).🤖 Generated with Claude Code