Story 10: Maya chooses how her form was extracted#58
Merged
danielnaab merged 28 commits intomainfrom Apr 19, 2026
Merged
Conversation
This was referenced Apr 19, 2026
Placeholders until Stories 12 and 13 register actual variants. Mapping registry is placed under form-documents/ since Story 7 relocated PDF field-mapping there.
…per project Replaces the hardcoded extractor passed to createProjectService with an ExtractionContext that resolves a per-user variant at call time. Each extraction commits its variantId/modelId/spec version into forms/default/provenance.json so callers can later surface which model produced a given spec. Also mounts the /settings route on the app server so users can change their variant selection through the picker landed in Task 9.
When the extractor cache is populated (after running evaluate or in long-running test processes), extraction completes synchronously on cached PDFs and projects land on an import branch awaiting review rather than transitioning through extracting/error states. Add the pending-review state to the list of valid post-creation renders.
…ives The previous settings page used bare HTML with classes that had no CSS, so it rendered as unstyled markup. Rebuild it on top of the shared flex design system — Card / Radio (tile variant) / Button / Alert — and the page-level l-stack / l-cluster layout primitives so it matches the look and feel of ProjectOverview and other owner pages. - Add a breadcrumb + title header modelled on repo-header. - Wrap each task in a Card with heading, body, and tile-style radios. - Use Radio in its tile variant for rich per-variant option content. - Extend Radio's label prop from 'string' to 'Child' so callers can pass JSX (bold name + description + catalog link). The change is additive; all existing string-label callsites still type-check. - Scope highlight-on-anchor behavior to an attribute on the task wrapper (scroll-margin + accent border) instead of the unstyled task-section--highlight class. - Add src/entrypoints/app/routes/settings/styles.css for page-specific typography, header, and highlight rules, and wire it into the bundle. - Update settings-variants test to match the friendlier empty-state copy.
When browsing under /catalog/experiments/..., the left sidebar now shows an experiments-scoped navigation: a Back to Catalog link, Overview, any top-level pages (Roadmap), then one section per suite (PDF Field Extraction, Shaping Architecture) with each variant listed. Top-level `_name.md` files (e.g. _roadmap.md) are now reachable at `/catalog/experiments/<name>` (without the leading underscore). The legacy URL `/catalog/experiments/_name` still works because the handler tries both forms. Factored out titleCaseSlug so suite/variant titles fall back gracefully when a heading can't be parsed.
danielnaab
force-pushed
the
story-10/variant-picker
branch
from
aaa045a to
ee27963
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Trunk of the experiment roadmap. Delivers a user-toggleable variant picker for PDF extraction, plus reusable infrastructure every subsequent experiment story (#59–#66) builds on.
Maya uploads a PDF, sees a callout on
/newshowing which extraction model will run, clicks through to/settings/variants, switches to Haiku, uploads again, and the new spec carries "Extracted by Claude Haiku 4.5" with provenance committed to the project repo.What shipped
Core infrastructure
VariantRegistry/VariantMetadatatype aliases over existingStrategyRegistryfor cross-task reusevariant-preferencesservice — per-user, per-task, SQLite-backed, validates variantId against the registry, falls back to default on stale preferences (prevents orphaned projects)appendProvenance,readProvenance) — pure functional, non-mutatingTaskstring union +TaskRegistriesmapped type; empty stub registries for filling and field-mapping so picker tabs rendersrc/services/variant-preferences/User-visible
/settings/variantspicker — auth-required, redesigned with design-system primitives: per-task section with label + description + "View benchmarks →" link, tile radios for variant selection, accent-bar highlight on anchor-jump<VariantCallout>— new design-system component on/newshowing the currently-selected extraction model above the upload area with "Change model" / "See benchmarks" affordances<VariantBadge>— compact inline component with per-task verbs (Extracted by / Shaped by / Guided by / Mapped by), rendered on project overview, pending-review, and commit snapshots/catalog/experiments/...listing each suite and its variants, matching the design-system catalog patternExtraction wiring
ExtractionContext(DI) instead of a singleton extractor — resolves user's preferred variant per callforms/default/provenance.jsonwritten in the same git commit asspec.json/form.json/confidence.json/field-mapping.jsongetProvenance(owner, slug, task, branch?)exposed onProjectServiceEvaluation & catalog
evaluate comparererun across expanded fixture set: Opus 99.8% recall, Sonnet 62.1%, Haiku 57.8%/catalog/experiments/roadmap— live index of all follow-up experiment stories (Maya chooses her shaping model #59–Maya extracts via structured tool-use #66), their status, and ship targetsFollow-up stories filed
Eight user-stories under milestone "Final Project", all blocked on this PR:
Each story's acceptance criteria require a catalog entry with findings, so the benchmark evidence lives alongside the code.
Architecture notes
git log forms/default/provenance.json.VariantBadge,VariantCallout) use--flex-*tokens and cascade layers, no hand-rolled colors.Test plan
bun run check— 759/759 pass, type check clean, biome clean, rebased cleanly on origin/main/settings/variantsauth redirect, authed render, POST valid/invalid selection, project-service provenance round-tripbun run cli evaluate validatebun run cli evaluate compareFiles
27 commits, ~2400 lines added. Key additions:
src/services/variant-preferences/(new service)src/entrypoints/app/routes/settings/(picker route)src/design-system/components/flex-variant-badge/,flex-variant-callout/(new DS components)catalog/experiments/_roadmap.md, updated variant catalog pagesfixtures/i-9/,fixtures/w-9/with ground truthRoadmap context
notes/story-10-variant-picker/design.mdnotes/story-10-variant-picker/plan.md/catalog/experiments/roadmapBlocks: #59, #60, #61, #62, #63, #64, #65, #66