diff --git a/.changelog/NEXT.md b/.changelog/NEXT.md index b284e7258..e69f983dd 100644 --- a/.changelog/NEXT.md +++ b/.changelog/NEXT.md @@ -4,6 +4,7 @@ - **Universe canon now lives inside Universe Builder.** Characters, places, and objects are managed inline on the universe page — no separate canon page to navigate to. The old canon URL still works as a redirect, and the Series Pipeline link lands you on the same combined view. Pending edits to other universe fields are no longer lost when canon changes are saved. - **Locking a canon entry now also blocks new reference renders.** Locked characters/places/objects already prevented AI rewrites; they now prevent new reference and clean-plate image renders too, so a locked entry's identity stays frozen across both text and visuals. Disabled buttons explain the lock in their tooltips. +- **Universe Builder categories now carry a canon trunk (`kind`)** so each bucket knows whether it belongs under Cast, Places, Objects, or Other. Built-in defaults are pre-tagged (landscapes/environments/structures → Places, vehicles → Objects); custom buckets default to Other until you sort them. The default `characters` category was retired — canon owns characters now, and any pre-existing character variations get folded into universe canon on upgrade. Foundation for the upcoming tabbed-trunk Universe Builder redesign. ## Fixed diff --git a/PLAN.md b/PLAN.md index 5a15f6ad2..242992d5e 100644 --- a/PLAN.md +++ b/PLAN.md @@ -6,7 +6,11 @@ For project goals, see [GOALS.md](./GOALS.md). For completed work, see [DONE.md] ## Next Up -1. **Universe-as-Canon Phase 2 UI — remaining schema retirement.** Canon is now folded into UniverseBuilder and the standalone Canon page is retired. Still to do: drop `universe.categories` from the schema (sanitizer, Zod, expand prompt template, `mergeCategoriesWithLocks`, `compilePrompts`'s `variations`/`all` branches) and enrich the expand LLM contract to ask for `characters[]`/`settings[]`/`objects[]` with rich metadata directly. +1. **Universe Builder redesign — trunks + sub-buckets layout.** Replace the vertical-stack layout with a tabbed-trunk layout (`Bible / Cast / Places / Objects / Composites / Render`). Unify custom categories as sub-buckets under one of the 3 canon trunks via a new `kind: 'characters'|'settings'|'objects'` field on each category. Card grids with thumbnails replace the per-category accordion. Multi-phase (each phase is its own PR): + - **Phase A — Data model + migration.** Add `kind: 'characters'|'settings'|'objects'|'other'` field to category sanitizer (`server/services/universeBuilder.js#sanitizeCategories`) and Zod (`server/routes/universeBuilder.js#categoryShape`); default to `'other'` when missing/invalid. Sub-bucket keys remain **globally unique** (no per-trunk namespacing). Migration `scripts/migrations/NNN-categorize-universe-buckets.js` assigns built-in defaults: `landscapes/environments/structures → settings`, `vehicles → objects`. The default `characters` category is retired; its variations backfill into `universe.characters[]` (reuse `backfillCanonFromCategories` logic). All other existing custom categories get `kind: 'other'`; Phase C surfaces them under an **Other** tab with an **Auto-sort** button that LLM-classifies each bucket into the right trunk + a meaningful name. + - **Phase B — Expand contract enrichment.** Teach `buildExpansionPrompt()` (`server/services/universeBuilderExpand.js`) to return rich canon arrays (`characters[]`/`settings[]`/`objects[]` with `physicalDescription`/`palette`/`recurringDetails`/`wardrobe`) alongside the existing `categories` (now kind-tagged) + `compositeSheets`. Update `isExpansionShape()` predicate. Client `handleExpand` in `UniverseBuilder.jsx` merges canon entries into the canon arrays parallel to the category merge (dedupe by name, respect per-entry locks). Categories the LLM emits include a `kind` so they land under the right trunk. + - **Phase C — Layout rewrite.** Tabbed top-level (Bible / Cast / Places / Objects / Other / Composites / Render) with URL state (`?tab=cast&bucket=heroes` per CLAUDE.md linkable-routes convention) — Bible is its own tab (not a sticky header). Per-trunk view = sub-bucket chip filter row + responsive card grid. Unify `CanonCard` + a new card for category variations into a single `EntryCard` component (renders thumbnail from `primaryImageRef` or most recent render). **Canon entries are first-class batch-render targets** alongside category variations and composite sheets: per-bucket actions are "Generate N more" + "Bulk-render this bucket"; per-trunk action is "Bulk-render all Cast/Places/Objects" (includes BOTH canon entries AND every variation in every sub-bucket under that trunk); the Batch Render tab itself offers per-trunk, per-bucket, and "All canon" selectors plus the existing composite-sheets mode. Composite reference images stay intact as their own render mode (no regression). Server-side: extend `compilePrompts` to accept canon entries as render sources (synthesize a prompt from `name + physicalDescription/palette/description`, layered with the universe style preset the same way variations are today). The **Other** tab only appears when un-kinded buckets exist, and shows the **Auto-sort with AI** action. Mobile: tabs collapse to a select dropdown. + - **Phase D — Polish & promotion.** "Promote variation to canon" action: take a `{label, prompt}` variation, expand it via LLM into a full canon entry (`name`, `physicalDescription`, etc.) and move it from the category bucket into the canon array. Tests for the new components. Extract design to `docs/features/universe-builder.md` if the doc warrants it. 2. **Step-by-step approval/lock UX across Universe → Series → Arc → Seasons → Episodes.** Iteration 1 shipped a single arc-level lock; extend to per-season + per-field locks, lock the bulk runners, surface stage-progress strip, enforce locks server-side before LLM invocations. 3. **Sharing v2 contracts** — per-peer subscription filenames (`sub---.json`), tombstone-based item removals, "🔄 live" badge on inbox subscription rows. 4. **Pipeline continuity gaps** — plumb character physicalDescription/personality/background into idea-stage prompt; plumb setting `palette`/`era`/`weather`/`recurringDetails` into visual stages; add `worldEntitiesSummary` to text stages; add a dedicated `voice` / speech-pattern field to the bible schema. @@ -36,8 +40,10 @@ For project goals, see [GOALS.md](./GOALS.md). For completed work, see [DONE.md] ### Universe-as-Canon — Phase 2 + extensions - [ ] **CanonCard "from series: " full provenance label.** Card currently shows a "from series" chip with the series id in the tooltip. Plumb a `seriesNameMap` (or `sourceSeriesName` per entry) so the chip can render the actual series name. Needs the parent (`UniverseCanonSection` / `NounsStage`) to pass a `{ [seriesId]: name }` lookup. -- [ ] **Retire `universe.categories` on the schema.** After UI no longer reads it, drop from `sanitizeTemplate`, route Zod schemas, expand prompt template, `mergeCategoriesWithLocks`, `compilePrompts`'s `'variations'`/`'all'` branches. -- [ ] **Universe expand LLM contract enrichment.** Ask the LLM directly for `characters[]` / `settings[]` / `objects[]` with rich narrative metadata alongside visual `prompt`. +- [x] ~~**Retire `universe.categories` on the schema.**~~ **Rejected 2026-05-17** — categories are an active user-facing exploration workflow (custom buckets like `factions`/`colonies`/`raider_clans`, bulk variation generation, batch render). Canon has no equivalent. See "Categories vs canon — decision" below. +- [→] **Drop the default `characters` category.** Folded into Next Up #1 Phase A. +- [→] **Universe expand LLM contract enrichment.** Folded into Next Up #1 Phase B. +- [ ] **arcPlanner prompt context — include canon characters/places/objects.** `server/services/pipeline/arcPlanner.js:96` only renders `world.categories` into `worldCategoriesText`. With the `characters` default category retired (schema v4), characters now live in `world.characters[]` (canon) and don't surface in arc-planning prompts. Add a sibling `renderCanonForPrompt(world)` helper and a `worldCanonText` context field; update the arc prompt template + tests. Same gap likely exists in other prompt builders that read `world.categories` — sweep with `grep -rn "world\.categories" server/services/pipeline server/services/universeBuilder*.js`. - [ ] **Settings → Places kind rename.** `BIBLE_KIND.SETTING → BIBLE_KIND.PLACE`, `BIBLE_FIELD[SETTING]: 'settings' → 'places'`. Touches ~20 files. Stick the rename to bible context — app settings stays as "settings". - [ ] **Use rendered reference images as i2i anchors in downstream comic-page renders for models that support it.** SDXL/Flux pipelines anchor every panel render on the per-character rendered ref. @@ -177,6 +183,24 @@ For project goals, see [GOALS.md](./GOALS.md). For completed work, see [DONE.md] --- +## Design decisions + +### Categories vs canon — decision (2026-05-17) + +**First framing (rejected same day):** retire `universe.categories` entirely, assuming canon subsumed it. This was wrong — canon and categories serve different workflows (consistency vs. exploration) and custom buckets like `factions`/`colonies` have no clean home in canon. + +**Second framing (rejected same day):** keep canon and categories as *complementary siblings* (two top-level sections of the Universe Builder page). Rejected because it preserves the bifurcated mental model — the user sees `Cast` and `Factions` as separate top-level concepts even though factions are characters. + +**Final framing (accepted 2026-05-17):** **unify under 3 canon trunks.** The Universe Builder has 3 first-class trunks — `Characters`, `Places`, `Objects` — and every entity in the universe (canon entries AND category variations) lives under exactly one trunk. Each category gets a new `kind` field tagging it to its trunk: + +- **Canon entries** = first-class entities with rich production metadata (`physicalDescription`, `palette`, `recurringDetails`, `wardrobe`, `imageRefs`). Named, consistent across episodes. +- **Sub-buckets** (formerly "categories") = organizational + bulk-generation surfaces *within* a trunk. `Cast > Heroes/Villains/Factions`, `Places > Colonies/Ruins`, `Objects > Vehicles/Weapons`. Each holds flat `{label, prompt}` variations for visual exploration. +- **Promotion**: a variation can be promoted to canon — the LLM expands it into a full canon entry and moves it from the bucket into the canon array. + +This collapses the page to 3 navigable trunks (plus Bible/Composites/Render), supports inline thumbnails per entry, and gives every entity one obvious home. See Next Up #1 for the multi-phase implementation. + +--- + ## Deferred Architecture (human-led planning) God-file decomposition candidates — none are bugs; pick up when touching the file for unrelated reasons. diff --git a/client/src/pages/UniverseBuilder.jsx b/client/src/pages/UniverseBuilder.jsx index 584d6ebd9..5d9cbb8d4 100644 --- a/client/src/pages/UniverseBuilder.jsx +++ b/client/src/pages/UniverseBuilder.jsx @@ -38,7 +38,6 @@ import { PIPELINE_IMAGE_DEFAULTS, readPipelineImageSettings } from '../lib/pipel const CATEGORY_LABELS = { landscapes: 'Landscapes', environments: 'Environments', - characters: 'Characters', structures: 'Structures', vehicles: 'Vehicles', }; diff --git a/scripts/migrations/018-categorize-universe-buckets.js b/scripts/migrations/018-categorize-universe-buckets.js new file mode 100644 index 000000000..5a95e16d8 --- /dev/null +++ b/scripts/migrations/018-categorize-universe-buckets.js @@ -0,0 +1,143 @@ +/** + * Universe Builder — assign `kind` to category buckets + retire default + * `characters` category (schema v3 → v4). + * + * Why: + * The Universe Builder redesign tags every category bucket with a `kind` + * (`characters` | `settings` | `objects` | `other`) so the Phase C UI can + * render it under the right canon trunk. The default `characters` category + * was retired because canon owns characters now — leaving the bucket in + * place creates two homes for the same data. + * + * What this does to each universe in data/universe-builder.json: + * - If schemaVersion >= 4, skip (already migrated). + * - Fold any `categories.characters.variations[]` into `universe.characters[]` + * (canon), dedupe by name (case-insensitive). Drop the `characters` bucket. + * - Assign `kind` to every remaining category: built-ins use + * WORLD_CATEGORY_DEFAULT_KINDS (landscapes/environments/structures → + * settings, vehicles → objects); custom buckets default to `'other'` + * (Phase C surfaces an Auto-sort action to LLM-classify them). + * - Stamp `schemaVersion: 4`. + * + * Idempotent: schemaVersion gate makes a re-run a no-op. The on-read + * sanitizer in universeBuilder.js#sanitizeTemplate applies the same + * transformations so installs that skip this script still converge — this + * script just makes the transition observable + atomic. + */ + +import { readFile, writeFile } from 'fs/promises'; +import { join } from 'path'; + +const TARGET_SCHEMA_VERSION = 4; + +// Kept inline so this one-shot migration's contract is frozen against +// future runtime renames or kind-set changes. +const DEFAULT_KINDS = { + landscapes: 'settings', + environments: 'settings', + structures: 'settings', + vehicles: 'objects', +}; +const VALID_KINDS = new Set(['characters', 'settings', 'objects', 'other']); +const FALLBACK_KIND = 'other'; + +const resolveKind = (key, rawKind) => { + if (VALID_KINDS.has(rawKind)) return rawKind; + return DEFAULT_KINDS[key] || FALLBACK_KIND; +}; + +// Lowercase-trim — same shape as normalizeBibleName for cross-process dedupe. +const nameKey = (name) => (typeof name === 'string' ? name.trim().toLowerCase() : ''); + +const readJson = async (path) => { + const raw = await readFile(path, 'utf-8').catch((err) => { + if (err.code === 'ENOENT') return null; + throw err; + }); + if (raw == null) return null; + return JSON.parse(raw); +}; + +const writeJson = (path, value) => + writeFile(path, JSON.stringify(value, null, 2) + '\n'); + +export default { + async up({ rootDir }) { + const filePath = join(rootDir, 'data', 'universe-builder.json'); + const doc = await readJson(filePath); + if (!doc || !Array.isArray(doc.universes)) { + console.log('🌐 migration 018: no universe-builder.json found, skipping'); + return; + } + + let touched = 0; + let charactersFolded = 0; + let kindsAssigned = 0; + let bucketsDropped = 0; + + for (const universe of doc.universes) { + if (!universe || typeof universe !== 'object') continue; + if ((universe.schemaVersion || 0) >= TARGET_SCHEMA_VERSION) continue; + + try { + const categories = universe.categories && typeof universe.categories === 'object' + ? universe.categories + : {}; + + // Fold legacy `characters` bucket into canon characters[] (dedupe by + // name). The on-read backfill already does this for v1/v2 universes; + // we re-run here to cover v3 universes that still carry the bucket. + const charactersBucket = categories.characters; + if (charactersBucket && Array.isArray(charactersBucket.variations)) { + if (!Array.isArray(universe.characters)) universe.characters = []; + const seen = new Set(universe.characters.map((e) => nameKey(e?.name))); + for (const variation of charactersBucket.variations) { + const name = (variation?.label || '').trim(); + if (!name) continue; + const key = nameKey(name); + if (seen.has(key)) continue; + seen.add(key); + const entry = { + name, + prompt: (variation?.prompt || '').trim(), + tags: [], + source: 'universe-expand', + }; + if (variation?.locked === true) entry.locked = true; + universe.characters.push(entry); + charactersFolded += 1; + } + delete categories.characters; + bucketsDropped += 1; + } + + // Assign kind to remaining buckets. + for (const [key, bucket] of Object.entries(categories)) { + if (!bucket || typeof bucket !== 'object') continue; + const kind = resolveKind(key, bucket.kind); + if (bucket.kind !== kind) { + bucket.kind = kind; + kindsAssigned += 1; + } + } + + universe.categories = categories; + universe.schemaVersion = TARGET_SCHEMA_VERSION; + touched += 1; + } catch (err) { + // Re-throw with the offending universe id so a partial-batch failure + // is debuggable. Without this, the bare stack points at the field + // access but doesn't say which universe. + throw new Error(`migration 018 failed on universe id=${universe.id || ''}: ${err.message}`); + } + } + + if (touched === 0) { + console.log('🌐 migration 018: all universes already at schema v4, skipping write'); + return; + } + + await writeJson(filePath, doc); + console.log(`🌐 migration 018: updated ${touched} universe(s) — folded ${charactersFolded} character variation(s) into canon, dropped ${bucketsDropped} 'characters' bucket(s), assigned kind to ${kindsAssigned} bucket(s)`); + }, +}; diff --git a/scripts/migrations/018-categorize-universe-buckets.test.js b/scripts/migrations/018-categorize-universe-buckets.test.js new file mode 100644 index 000000000..de9a2c888 --- /dev/null +++ b/scripts/migrations/018-categorize-universe-buckets.test.js @@ -0,0 +1,161 @@ +/** + * Test for migration 018 — categorize universe buckets + retire characters bucket. + * + * Vitest's `include` pattern (`**\/*.test.js`) only scans server/ and + * client/src/. To get this picked up, server/vitest.config.js was extended + * to also include `../scripts/migrations/**\/*.test.js`. + */ +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { mkdtempSync, rmSync, writeFileSync, readFileSync, mkdirSync, existsSync } from 'fs'; +import { tmpdir } from 'os'; +import { join } from 'path'; + +import migration from './018-categorize-universe-buckets.js'; + +describe('migration 018 — categorize universe buckets', () => { + let rootDir; + let dataDir; + let filePath; + + beforeEach(() => { + rootDir = mkdtempSync(join(tmpdir(), 'migration-018-')); + dataDir = join(rootDir, 'data'); + mkdirSync(dataDir, { recursive: true }); + filePath = join(dataDir, 'universe-builder.json'); + }); + + afterEach(() => { + rmSync(rootDir, { recursive: true, force: true }); + }); + + const writeUniverses = (universes) => { + writeFileSync(filePath, JSON.stringify({ universes, runs: [] }, null, 2)); + }; + const readUniverses = () => JSON.parse(readFileSync(filePath, 'utf-8')).universes; + + it('no-ops cleanly when universe-builder.json is missing', async () => { + expect(existsSync(filePath)).toBe(false); + await expect(migration.up({ rootDir })).resolves.not.toThrow(); + expect(existsSync(filePath)).toBe(false); + }); + + it('skips universes already at schema v4', async () => { + writeUniverses([{ + id: 'already-migrated', + name: 'Already', + schemaVersion: 4, + categories: { landscapes: { kind: 'settings', variations: [] } }, + characters: [], + }]); + const before = readFileSync(filePath, 'utf-8'); + await migration.up({ rootDir }); + const after = readFileSync(filePath, 'utf-8'); + expect(after).toBe(before); // bit-for-bit unchanged + }); + + it('folds categories.characters into universe.characters[] and drops the bucket', async () => { + writeUniverses([{ + id: 'legacy', + name: 'Legacy', + schemaVersion: 3, + categories: { + landscapes: { variations: [{ label: 'Crystal Canyon', prompt: 'canyon' }] }, + characters: { variations: [ + { label: 'Ash', prompt: 'young survivor' }, + { label: 'Roan', prompt: 'weathered scavenger', locked: true }, + ] }, + }, + characters: [], + }]); + await migration.up({ rootDir }); + const [u] = readUniverses(); + expect(u.schemaVersion).toBe(4); + expect(u.categories.characters).toBeUndefined(); + expect(u.categories.landscapes).toBeDefined(); + expect(u.characters.find((c) => c.name === 'Ash')).toBeDefined(); + const roan = u.characters.find((c) => c.name === 'Roan'); + expect(roan).toBeDefined(); + expect(roan.locked).toBe(true); + expect(roan.source).toBe('universe-expand'); + }); + + it('dedupes by canon name when folding the characters bucket', async () => { + writeUniverses([{ + id: 'mixed', + name: 'Mixed', + schemaVersion: 3, + categories: { + characters: { variations: [{ label: 'Ash', prompt: 'from variation' }] }, + }, + // Hand-authored canon entry with richer metadata — must not be clobbered. + characters: [{ + id: 'chr-existing', name: 'Ash', + physicalDescription: 'hand-authored description', + source: 'manual', + }], + }]); + await migration.up({ rootDir }); + const [u] = readUniverses(); + expect(u.characters.filter((c) => c.name === 'Ash')).toHaveLength(1); + expect(u.characters[0].source).toBe('manual'); // hand-authored preserved + }); + + it('assigns kind to built-in defaults and custom buckets', async () => { + writeUniverses([{ + id: 'kinds', + name: 'Kinds', + schemaVersion: 3, + categories: { + landscapes: { variations: [] }, + environments: { variations: [] }, + structures: { variations: [] }, + vehicles: { variations: [] }, + factions: { variations: [{ label: 'Iron Reach', prompt: 'x' }] }, + colonies: { variations: [{ label: 'Tycho', prompt: 'y' }] }, + }, + characters: [], + }]); + await migration.up({ rootDir }); + const [u] = readUniverses(); + expect(u.categories.landscapes.kind).toBe('settings'); + expect(u.categories.environments.kind).toBe('settings'); + expect(u.categories.structures.kind).toBe('settings'); + expect(u.categories.vehicles.kind).toBe('objects'); + expect(u.categories.factions.kind).toBe('other'); + expect(u.categories.colonies.kind).toBe('other'); + }); + + it('preserves an explicit valid `kind` over the built-in default', async () => { + writeUniverses([{ + id: 'explicit-kind', + name: 'Explicit', + schemaVersion: 3, + categories: { + // Hand-authored kind that disagrees with the built-in default. + landscapes: { kind: 'objects', variations: [] }, + }, + characters: [], + }]); + await migration.up({ rootDir }); + const [u] = readUniverses(); + expect(u.categories.landscapes.kind).toBe('objects'); + }); + + it('is idempotent — running twice produces no further changes', async () => { + writeUniverses([{ + id: 'idem', + name: 'Idempotent', + schemaVersion: 3, + categories: { + characters: { variations: [{ label: 'Ash', prompt: 'x' }] }, + landscapes: { variations: [] }, + }, + characters: [], + }]); + await migration.up({ rootDir }); + const afterFirst = readFileSync(filePath, 'utf-8'); + await migration.up({ rootDir }); + const afterSecond = readFileSync(filePath, 'utf-8'); + expect(afterSecond).toBe(afterFirst); + }); +}); diff --git a/server/routes/universeBuilder.js b/server/routes/universeBuilder.js index fc0308b4f..0c3b2008a 100644 --- a/server/routes/universeBuilder.js +++ b/server/routes/universeBuilder.js @@ -57,6 +57,11 @@ const compositeSheetSchema = z.object({ locked: z.boolean().optional(), }); const categoryShape = z.object({ + // Tags this bucket to one of the 3 canon trunks (or 'other' as the + // un-classified sink). Optional on input — sanitizeCategories resolves a + // sensible default from the built-in map (landscapes→settings etc.) or + // falls to 'other'. Added in schema v4. + kind: z.enum(svc.CATEGORY_KINDS).optional(), variations: z.array(variationSchema).max(svc.VARIATIONS_PER_CATEGORY_MAX), }); const categoriesSchema = z.record( diff --git a/server/routes/universeBuilder.test.js b/server/routes/universeBuilder.test.js index 18f59ae42..fe1d1c6ea 100644 --- a/server/routes/universeBuilder.test.js +++ b/server/routes/universeBuilder.test.js @@ -106,10 +106,58 @@ describe('universe-builder routes', () => { expect(res.status).toBe(201); expect(res.body.id).toBe('uuid-1'); expect(res.body.name).toBe('My Universe'); - // All five categories populated even when not supplied. + // Default categories populated, each tagged with its canon trunk via + // WORLD_CATEGORY_DEFAULT_KINDS. `characters` was retired in schema v4 + // (canon owns characters now). expect(Object.keys(res.body.categories).sort()).toEqual( - ['characters', 'environments', 'landscapes', 'structures', 'vehicles'], + ['environments', 'landscapes', 'structures', 'vehicles'], ); + expect(res.body.categories.landscapes.kind).toBe('settings'); + expect(res.body.categories.vehicles.kind).toBe('objects'); + }); + + it('POST / accepts and persists a `kind` on each category', async () => { + const res = await request(buildApp()) + .post('/api/universe-builder') + .send({ + name: 'Kinded', + categories: { + factions: { kind: 'characters', variations: [{ label: 'Iron Reach', prompt: 'x' }] }, + colonies: { kind: 'settings', variations: [{ label: 'Tycho', prompt: 'y' }] }, + }, + }); + expect(res.status).toBe(201); + expect(res.body.categories.factions.kind).toBe('characters'); + expect(res.body.categories.colonies.kind).toBe('settings'); + }); + + it('POST / rejects an invalid `kind` enum value via Zod', async () => { + const res = await request(buildApp()) + .post('/api/universe-builder') + .send({ + name: 'Bad Kind', + categories: { + colonies: { kind: 'not-a-kind', variations: [] }, + }, + }); + expect(res.status).toBe(400); + }); + + it('POST / folds a stale `characters` bucket into canon characters[] and drops the bucket', async () => { + const res = await request(buildApp()) + .post('/api/universe-builder') + .send({ + name: 'Stale Client', + categories: { + // Mimic an outdated client still sending the retired bucket. + characters: { variations: [{ label: 'Ash', prompt: 'young survivor' }] }, + }, + }); + expect(res.status).toBe(201); + expect(res.body.categories.characters).toBeUndefined(); + const ash = res.body.characters.find((c) => c.name === 'Ash'); + expect(ash).toBeDefined(); + expect(ash.prompt).toBe('young survivor'); }); it('POST / accepts dynamic universe-building categories', async () => { @@ -242,7 +290,10 @@ describe('universe-builder routes', () => { influences: { embrace: ['style'], avoid: ['neg'] }, categories: { landscapes: { variations: [{ label: 'A', prompt: 'a prompt' }, { label: 'B', prompt: 'b prompt' }] }, - characters: { variations: [{ label: 'C', prompt: 'c prompt' }] }, + // `characters` was retired as a default bucket in schema v4 (canon + // owns characters); use a custom bucket to keep the 3-variation + // render scenario intact. + outfits: { variations: [{ label: 'C', prompt: 'c prompt' }] }, }, }); const res = await request(app) diff --git a/server/services/pipeline/arcPlanner.test.js b/server/services/pipeline/arcPlanner.test.js index b242cc851..e78650689 100644 --- a/server/services/pipeline/arcPlanner.test.js +++ b/server/services/pipeline/arcPlanner.test.js @@ -169,9 +169,13 @@ describe('arcPlanner — generateArcOverview', () => { { label: 'The Lollipop Bureau', prompt: 'pastel public-facing agency' }, { label: 'The Velvet Null', prompt: 'minimalist rival' }, ] }, - characters: { variations: [ - { label: 'Mira Holt', prompt: 'field detective' }, - ] }, + // `characters` was retired as a default category in schema v4 — any + // variations under it now fold into universe.characters[] (canon). + // Use the cast leads as canon characters directly. Note: arcPlanner's + // current `worldCategoriesText` only reads from categories; canon + // characters aren't included yet (deferred to Phase B/C — see + // PLAN.md Backlog: "arcPlanner prompt context should include canon"). + // For now this test only asserts category data flows through. }, compositeSheets: [ { kind: 'reference_sheet', label: 'Rival agencies branding', prompt: 'comparison sheet' }, @@ -192,7 +196,7 @@ describe('arcPlanner — generateArcOverview', () => { expect(ctx.worldName).toBe('Clandestiny'); expect(ctx.worldCategoriesText).toContain('factions'); expect(ctx.worldCategoriesText).toContain('The Lollipop Bureau'); - expect(ctx.worldCategoriesText).toContain('Mira Holt'); + expect(ctx.worldCategoriesText).toContain('The Velvet Null'); expect(ctx.worldCompositesText).toContain('Rival agencies branding'); expect(ctx.worldInfluencesEmbrace).toContain('Moebius'); expect(ctx.worldInfluencesAvoid).toContain('gritty'); diff --git a/server/services/universeBuilder.js b/server/services/universeBuilder.js index c816b3530..ee388d00b 100644 --- a/server/services/universeBuilder.js +++ b/server/services/universeBuilder.js @@ -41,7 +41,11 @@ import { emitRecordUpdated, emitRecordDeleted } from './sharing/recordEvents.js' // v3 — drop prose stylePrompt/negativePrompt fields; legacy values are // split on commas and merged into influences.embrace / influences.avoid // so there is a single token-list editing surface. -export const CURRENT_SCHEMA_VERSION = 3; +// v4 — categories carry a `kind` field tagging them to one of the 3 canon +// trunks (characters/settings/objects/other); the default `characters` +// category is retired and any variations get folded into canon +// characters[]. See "Categories vs canon — decision" in PLAN.md. +export const CURRENT_SCHEMA_VERSION = 4; // Lazy state-path resolution so test harnesses that swap PATHS.data // per-test (mkdtempSync + Proxy mock) see the right temp root. Computing @@ -122,19 +126,52 @@ export const LOCKABLE_FIELD_LABELS = Object.freeze({ export const INFLUENCE_LOCK_FIELDS = Object.freeze(['influencesEmbrace', 'influencesAvoid']); export const isInfluenceLockField = (key) => INFLUENCE_LOCK_FIELDS.includes(key); -// Legacy buckets the v1 universe schema used. Retires after Phase 2 UI reads -// canon directly — until then it's still consumed by existing expand / refine -// / route code. +// Built-in default category buckets the Universe Builder seeds on every new +// universe. Each is tagged with a canon trunk (see WORLD_CATEGORY_DEFAULT_KINDS) +// so the Phase C UI renders it under the right tab without needing a per-bucket +// picker. The default `characters` bucket was retired in schema v4 — canon +// owns characters now; any pre-v4 variations are folded into universe.characters[]. export const WORLD_CATEGORIES = Object.freeze([ 'landscapes', 'environments', - 'characters', 'structures', 'vehicles', ]); +// Valid values for a category's `kind`. Tagged onto each category so the UI +// knows which canon trunk to render it under. `other` is the sink for +// un-classified custom buckets; an "Auto-sort" UI action LLM-classifies them +// into one of the 3 real kinds. +export const CATEGORY_KINDS = Object.freeze(['characters', 'settings', 'objects', 'other']); +export const DEFAULT_CATEGORY_KIND = 'other'; + +// Bucket keys retired from the schema — sanitizeCategories drops them from +// `categories` on read (variations get folded into canon by +// backfillCanonFromCategories). Single hook for future retirements. +const RETIRED_CATEGORY_KEYS = Object.freeze(new Set(['characters'])); + +// Built-in default categories carry a known kind so they land under the right +// trunk in the UI without user intervention. Custom keys not in this map fall +// to DEFAULT_CATEGORY_KIND ('other') unless the input carries an explicit +// valid `kind`. +export const WORLD_CATEGORY_DEFAULT_KINDS = Object.freeze({ + landscapes: 'settings', + environments: 'settings', + structures: 'settings', + vehicles: 'objects', +}); + +// Resolve a category's kind. Precedence: explicit valid kind on the input wins; +// otherwise the built-in default map; otherwise DEFAULT_CATEGORY_KIND. +const resolveCategoryKind = (key, rawKind) => { + if (CATEGORY_KINDS.includes(rawKind)) return rawKind; + return WORLD_CATEGORY_DEFAULT_KINDS[key] || DEFAULT_CATEGORY_KIND; +}; + // Maps v1 category buckets to canon kinds + tags. Unknown keys fall to -// object (catch-all kind) tagged with the bucket name. +// object (catch-all kind) tagged with the bucket name. Still used by the +// v3→v4 backfill that folds the retired `characters` bucket into canon, and +// by the optional pre-v4 backfill for legacy `landscapes/vehicles/etc` buckets. const CATEGORY_TO_CANON = Object.freeze({ characters: { kind: BIBLE_KIND.CHARACTER, tags: [] }, landscapes: { kind: BIBLE_KIND.SETTING, tags: ['landscape'] }, @@ -189,10 +226,15 @@ const sanitizeCompositeSheet = (raw) => { return out; }; -const sanitizeCategory = (raw) => { - // Per-category structure: { variations: [{ label, prompt }] }. Cap so a +const sanitizeCategory = (raw, key) => { + // Per-category structure: { kind, variations: [{ label, prompt }] }. Cap so a // runaway LLM can't blow up the universe template; matches the route schema. - if (!raw || typeof raw !== 'object') return { variations: [] }; + // `kind` tags the bucket to one of the 3 canon trunks (characters/settings/ + // objects) or 'other'; resolveCategoryKind picks the best value from + // (explicit input || built-in default || 'other'). + if (!raw || typeof raw !== 'object') { + return { kind: resolveCategoryKind(key), variations: [] }; + } const variations = []; if (Array.isArray(raw.variations)) { for (const v of raw.variations) { @@ -202,30 +244,43 @@ const sanitizeCategory = (raw) => { if (variations.length >= VARIATIONS_PER_CATEGORY_MAX) break; } } - return { variations }; + return { kind: resolveCategoryKind(key, raw.kind), variations }; }; +// Merges an `incoming` category into `base`, concatenating variations under +// the cap and trusting `incoming.kind`. The sole caller (`sanitizeCategories`) +// always passes a `sanitizeCategory`-produced `incoming`, so kind is +// guaranteed valid — no fallback needed. const mergeCategories = (base, next) => { const merged = { ...base }; for (const [key, category] of Object.entries(next)) { const current = merged[key]?.variations || []; - const incoming = category?.variations || []; - merged[key] = { variations: [...current, ...incoming].slice(0, VARIATIONS_PER_CATEGORY_MAX) }; + const incoming = category.variations; + merged[key] = { + kind: category.kind, + variations: [...current, ...incoming].slice(0, VARIATIONS_PER_CATEGORY_MAX), + }; } return merged; }; export const sanitizeCategories = (raw = {}) => { - const categories = Object.fromEntries(WORLD_CATEGORIES.map((key) => [key, { variations: [] }])); + const categories = Object.fromEntries( + WORLD_CATEGORIES.map((key) => [key, { kind: resolveCategoryKind(key), variations: [] }]) + ); if (!raw || typeof raw !== 'object') return categories; let customCount = WORLD_CATEGORIES.length; for (const [rawKey, rawCategory] of Object.entries(raw)) { const key = normalizeCategoryKey(rawKey); if (!key) continue; + // Retired buckets get dropped here; variations are folded into the + // matching canon array by backfillCanonFromCategories, which runs + // alongside this sanitizer in sanitizeTemplate. + if (RETIRED_CATEGORY_KEYS.has(key)) continue; if (!categories[key] && customCount >= WORLD_CATEGORY_COUNT_MAX) continue; if (!categories[key]) customCount += 1; - Object.assign(categories, mergeCategories(categories, { [key]: sanitizeCategory(rawCategory) })); + Object.assign(categories, mergeCategories(categories, { [key]: sanitizeCategory(rawCategory, key) })); } return categories; }; diff --git a/server/services/universeBuilder.test.js b/server/services/universeBuilder.test.js index 1f87cf275..54bc4ed6c 100644 --- a/server/services/universeBuilder.test.js +++ b/server/services/universeBuilder.test.js @@ -23,6 +23,11 @@ const svc = await import("./universeBuilder.js"); // Default universe with non-empty influences. Override `influences` for tests // that need isolation from the seed tokens. +// +// The seed uses `outfits` (a custom category) rather than the legacy default +// `characters` bucket — the latter was retired in schema v4 and any variations +// under it get folded into universe.characters[] (canon) on sanitize. Using a +// custom name keeps the 2-bucket / 3-variation scenario these tests rely on. const seedWorld = async (overrides = {}) => svc.createUniverse({ name: "Moebius SciFi", @@ -38,7 +43,7 @@ const seedWorld = async (overrides = {}) => { label: "Sand Sea", prompt: "endless sand sea, dunes" }, ], }, - characters: { + outfits: { variations: [ { label: "Scavenger", @@ -60,18 +65,24 @@ describe("universeBuilder service", () => { expect(await svc.listUniverses()).toEqual([]); }); - it("createUniverse persists with sanitized categories", async () => { + it("createUniverse persists with sanitized categories + kind tags", async () => { const w = await seedWorld(); expect(w.id).toBe("uuid-1"); expect(w.name).toBe("Moebius SciFi"); - // All five categories materialized even when only two were provided. + // All default categories materialized even when only one was provided, + // each tagged with its canon trunk via the WORLD_CATEGORY_DEFAULT_KINDS map. for (const c of svc.WORLD_CATEGORIES) { expect(w.categories[c]).toBeDefined(); expect(Array.isArray(w.categories[c].variations)).toBe(true); + expect(svc.CATEGORY_KINDS).toContain(w.categories[c].kind); } expect(w.categories.landscapes.variations).toHaveLength(2); - expect(w.categories.characters.variations).toHaveLength(1); + expect(w.categories.landscapes.kind).toBe("settings"); + expect(w.categories.vehicles.kind).toBe("objects"); expect(w.categories.environments.variations).toHaveLength(0); + // Custom (un-defaulted) bucket falls to 'other'. + expect(w.categories.outfits.kind).toBe("other"); + expect(w.categories.outfits.variations).toHaveLength(1); }); it("createUniverse preserves custom universe-building categories", async () => { @@ -244,7 +255,7 @@ describe("universeBuilder service", () => { it("returns one prompt per variation across selected categories with style prefix", async () => { const w = await seedWorld(); const compiled = svc.compilePrompts(w); - // 2 landscapes + 1 character = 3 (other categories empty) + // 2 landscapes + 1 outfit = 3 (other categories empty) expect(compiled).toHaveLength(3); // Style prefix uses `. ` separator (composeStyledPrompt convention, // shared with the scenePrompt composer in the client). @@ -290,9 +301,9 @@ describe("universeBuilder service", () => { it("selection: array filters by label (case-insensitive)", async () => { const w = await seedWorld(); const compiled = svc.compilePrompts(w, { - selection: { landscapes: ["crystal canyon"], characters: "all" }, + selection: { landscapes: ["crystal canyon"], outfits: "all" }, }); - // 1 landscape (filtered) + 1 character (all) = 2 + // 1 landscape (filtered) + 1 outfit (all) = 2 expect(compiled).toHaveLength(2); expect(compiled.map((c) => c.label).sort()).toEqual([ "Crystal Canyon", @@ -716,6 +727,90 @@ describe("universeBuilder service", () => { }); }); + describe("category kind (schema v4)", () => { + it("assigns built-in default kinds (landscapes/environments/structures→settings, vehicles→objects)", async () => { + const w = await seedWorld(); + expect(w.categories.landscapes.kind).toBe("settings"); + expect(w.categories.environments.kind).toBe("settings"); + expect(w.categories.structures.kind).toBe("settings"); + expect(w.categories.vehicles.kind).toBe("objects"); + }); + + it("defaults custom (non-built-in) buckets to 'other'", async () => { + const w = await seedWorld({ + categories: { + factions: { variations: [{ label: "Rebels", prompt: "x" }] }, + colonies: { variations: [{ label: "Tycho", prompt: "y" }] }, + }, + }); + expect(w.categories.factions.kind).toBe("other"); + expect(w.categories.colonies.kind).toBe("other"); + }); + + it("honors an explicit valid `kind` from the input over the built-in default", async () => { + const w = await seedWorld({ + categories: { + landscapes: { kind: "objects", variations: [] }, + factions: { kind: "characters", variations: [{ label: "Iron Reach", prompt: "z" }] }, + }, + }); + expect(w.categories.landscapes.kind).toBe("objects"); // override beats default + expect(w.categories.factions.kind).toBe("characters"); + }); + + it("falls back to default when explicit `kind` is invalid", async () => { + const w = await seedWorld({ + categories: { + landscapes: { kind: "not-a-kind", variations: [] }, + vehicles: { kind: 42, variations: [] }, + colonies: { kind: null, variations: [{ label: "Tycho", prompt: "x" }] }, + }, + }); + expect(w.categories.landscapes.kind).toBe("settings"); // built-in default + expect(w.categories.vehicles.kind).toBe("objects"); // built-in default + expect(w.categories.colonies.kind).toBe("other"); // custom fallback + }); + + it("drops the legacy default `characters` bucket and folds variations into canon", async () => { + const w = await seedWorld({ + categories: { + // Mimic a v3 client (or post-migration tab) accidentally sending the + // retired bucket — sanitize folds it into canon.characters[] and + // drops the bucket entirely. + characters: { + variations: [ + { label: "Ash", prompt: "young survivor with iron rebar" }, + { label: "Roan", prompt: "weathered scavenger" }, + ], + }, + }, + }); + expect(w.categories.characters).toBeUndefined(); + const ash = w.characters.find((c) => c.name === "Ash"); + const roan = w.characters.find((c) => c.name === "Roan"); + expect(ash).toBeDefined(); + expect(ash.prompt).toBe("young survivor with iron rebar"); + expect(roan).toBeDefined(); + }); + + it("WORLD_CATEGORIES no longer includes `characters`", () => { + expect(svc.WORLD_CATEGORIES).not.toContain("characters"); + expect(svc.WORLD_CATEGORIES).toEqual( + expect.arrayContaining(["landscapes", "environments", "structures", "vehicles"]), + ); + }); + + it("kind round-trips through update", async () => { + const w = await seedWorld({ + categories: { factions: { variations: [{ label: "Iron Reach", prompt: "x" }] } }, + }); + const patched = await svc.updateUniverse(w.id, { + categories: { factions: { kind: "characters", variations: [{ label: "Iron Reach", prompt: "x" }] } }, + }); + expect(patched.categories.factions.kind).toBe("characters"); + }); + }); + describe("sanitizers", () => { it("drops malformed variations on read", async () => { // Manually plant invalid state — sanitizeTemplate strips it on read. diff --git a/server/services/universeBuilderExpand.test.js b/server/services/universeBuilderExpand.test.js index fbe540230..d3918bbe7 100644 --- a/server/services/universeBuilderExpand.test.js +++ b/server/services/universeBuilderExpand.test.js @@ -142,10 +142,14 @@ describe("universeBuilderExpand.extractJson", () => { }); describe("universeBuilderExpand.normalizeCategories", () => { - it("returns all canonical categories with empty variations on empty input", () => { + it("returns all canonical categories with empty variations + kind on empty input", () => { const out = normalizeCategories({}); for (const key of WORLD_CATEGORIES) { - expect(out[key]).toEqual({ variations: [] }); + // schema v4 tags each bucket with a `kind` resolving its canon trunk; + // built-in defaults carry the expected mapping from + // WORLD_CATEGORY_DEFAULT_KINDS. + expect(out[key]).toMatchObject({ variations: [] }); + expect(typeof out[key].kind).toBe('string'); } }); @@ -160,10 +164,13 @@ describe("universeBuilderExpand.normalizeCategories", () => { }); it("truncates long string-shape labels at 80 chars", () => { + // `characters` was retired as a default bucket in schema v4 (canon owns + // characters now). Use a different built-in bucket to probe the length + // truncation behavior. const longText = "x".repeat(200); - const out = normalizeCategories({ characters: [longText] }); - expect(out.characters.variations[0].label).toHaveLength(80); - expect(out.characters.variations[0].prompt).toBe(longText); + const out = normalizeCategories({ vehicles: [longText] }); + expect(out.vehicles.variations[0].label).toHaveLength(80); + expect(out.vehicles.variations[0].prompt).toBe(longText); }); it("accepts the canonical { variations: [{label,prompt}] } shape", () => { @@ -212,8 +219,10 @@ describe("universeBuilderExpand.normalizeCategories", () => { }); it("treats a non-object category as empty variations (not a crash)", () => { - const out = normalizeCategories({ characters: "not an object" }); - expect(out.characters).toEqual({ variations: [] }); + // `characters` is retired — the sanitizer drops that key entirely. + // Probe with a custom key so the don't-crash behavior is the assertion. + const out = normalizeCategories({ factions: "not an object" }); + expect(out.factions).toMatchObject({ variations: [] }); }); }); diff --git a/server/vitest.config.js b/server/vitest.config.js index 05186769e..e98050e0a 100644 --- a/server/vitest.config.js +++ b/server/vitest.config.js @@ -7,8 +7,14 @@ export default defineConfig({ // helpers (normalize.js sidecar field resolution) have unit tests that // belong alongside the source, but the client itself has no test runner. // The server's vitest is the project's single test entrypoint, so we - // include the client *.test.js files here. - include: ['**/*.test.js', '../client/src/**/*.test.js'], + // include the client *.test.js files here. Also pick up migration tests + // from scripts/migrations/ so each one-shot migration can be verified + // against synthetic fixtures. + include: [ + '**/*.test.js', + '../client/src/**/*.test.js', + '../scripts/migrations/**/*.test.js', + ], coverage: { provider: 'v8', reporter: ['text', 'text-summary', 'html'],