Improve RAG quality: content-grounded generation, BM25 retrieval, pipeline infra - #2
Merged
Merged
Conversation
…eline infrastructure Trainforge RAG improvements: - Content-grounded question generation with term/statement/relationship extraction - BM25 retrieval with n-gram boosting and configurable relevance thresholds - Bloom's taxonomy chunk strategy mapping and deduplication - New ContentExtractor, LeakCheckValidator, and QuestionQualityValidator Pipeline infrastructure: - Standalone MCP pipeline tools with full textbook-to-course workflow - CLI wiring and parameter routing fixes - Workflow config updates for new pipeline stages Courseforge: - Multi-file course generation script Schema & validation: - Migrate schema $id from HTTP URLs to URN identifiers - Add local RefResolver for $ref resolution without HTTP fetches - Add "quality" subdirectory support to LibV2 importer Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1ca55cb3e2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+71
to
+73
| correct = [ | ||
| c["text"] for c in q["choices"] if c.get("is_correct") | ||
| ] |
There was a problem hiding this comment.
The validator registers MCQ correct answers directly from choices[*].text with HTML still attached, but later checks stripped stem text, so exact/substring leak detection misses real leaks whenever answers are wrapped in tags (the generator emits <p>...</p> by default). In practice this lets answer-bearing stems pass the leak_check gate for HTML-formatted assessments because LeakChecker compares normalized strings, not rendered text.
Useful? React with 👍 / 👎.
| if statements: | ||
| stmt = statements[0] | ||
| # Randomly decide true vs false (use question_id hash for determinism) | ||
| make_false = hash(question_id) % 2 == 0 |
There was a problem hiding this comment.
Using Python's built-in hash(question_id) makes T/F polarity non-reproducible across processes because string hash randomization changes between interpreter runs unless PYTHONHASHSEED is fixed. That means the same input objectives can produce different answers on different runs, which undermines deterministic assessment generation and training-capture reproducibility.
Useful? React with 👍 / 👎.
Comment on lines
+1079
to
+1083
| staging_dir = COURSEFORGE_INPUTS | ||
| for staging_run in sorted(staging_dir.iterdir()): | ||
| if not staging_run.is_dir(): | ||
| continue | ||
| for src_file in staging_run.iterdir(): |
There was a problem hiding this comment.
This phase reads HTML from every subdirectory under Courseforge/inputs/textbooks instead of a run-specific staging directory, so once multiple textbook pipelines have run, content generation mixes prior runs' textbooks into the current course. That can contaminate generated modules with unrelated material and corrupt downstream packaging/assessment output.
Useful? React with 👍 / 👎.
This was referenced Apr 21, 2026
mdmurphy822
added a commit
that referenced
this pull request
Apr 27, 2026
Bug #2: _build_source_module_map round-robin fallback was emitting the alphabetically-first DART block as primary on every page; relegate fallback blocks to the contributing role and pin via regression test. Bug #3: extend lib/ontology/tech_anchors.py from the Wave 82 W3C set (9 anchors) to a Wave 84 set covering 23 surface-form regexes — serialization formats (TriG, N-Quads, RDF/XML), RDF foundationals (IRI, literal, datatype, blank-node, rdf-dataset), SHACL shapes (node-shape, property-shape), RDFS predicates (subClassOf, subPropertyOf, rdf:type), and Turtle prefix detection. Backed by parametrized fixture tests covering case sensitivity and false-positive guards (e.g., lowercase "trig" math function NOT matched). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mdmurphy822
added a commit
that referenced
this pull request
Apr 27, 2026
Three correctness fixes from the v0.2.2 review pass. 1. LibV2 SHACL import gate (importer.py) — remove non-functional gate. `CourseManifest.to_dict()` carries no ed4all: @type nodes, so the Courseforge SHACL shapes validated zero focus nodes and every NodeShape conformed vacuously. The gate was a silent no-op in production. The lower-level `_shacl_validator.validate_manifest_shacl` helper stays available for future callers that feed it real Courseforge JSON-LD. 2. Misconception ID seed (preference_factory.py, process_course.py) — bloom-less path now hashes a 2-field seed `{statement}|{correction}` instead of `{statement}|{correction}|` (trailing pipe). Pre-Wave-72 the trailing pipe rekeyed every legacy / pre-Wave-60 misconception, contradicting the docstring's explicit "pre-Wave-69 hash stable for the bloom-less path" claim. Both hash call sites now stay in lock-step via the same two-form branch. 3. LO dedup in `_build_objectives_metadata_for_graph` — two-pass restructure (process_course.py). Pre-Wave-72 each item ran Path 1 (JSON-LD) then Path 2 (dataclass fallback), interleaved across items. An early item's empty dataclass LO could shadow a later item's rich JSON-LD LO for the same ID, silently discarding `targetedConcepts[]`. Now Path 1 runs across all items first, then Path 2 fills any IDs still absent — the "JSON-LD preferred" docstring promise actually holds. Regression tests added for #2 (2-field seed shape) and #3 (legacy + rich items across pages). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mdmurphy822
pushed a commit
that referenced
this pull request
May 3, 2026
…6 Subtask 1)
Extends courseforge_jsonld_v1.schema.json with a Phase 6 ABCD framework
discrete-fields contract for learning objectives. Gives the course-
outliner agent a constrained-decoding-validated structured output target
and gives lib.validators.abcd_objective.AbcdObjectiveValidator (Subtask
4) a stable shape to gate verb-Bloom alignment against.
Schema changes:
- New $defs.AbcdObjective: required {audience, behavior, condition,
degree}; behavior is a nested {verb, action_object} object with both
fields required. additionalProperties:false on both levels for tight
shape enforcement.
- New LearningObjective.properties.abcd: $ref to AbcdObjective.
Required-vs-optional decision:
- abcd is OPTIONAL on LearningObjective (NOT added to required[]) so
legacy synthesized_objectives.json fixtures and pre-Phase-6 corpora
continue to validate. The AbcdObjective sub-object itself enforces
all four ABCD fields when abcd is present (per pre-resolved decision
#2 in plans/phase6_abcd_concept_extractor.md).
- Phase 7+ will promote abcd to LearningObjective.required once corpus
calibration confirms safe.
Verification:
- Draft 2020-12 schema-check: OK (jsonschema.Draft202012Validator.
check_schema passes).
- Legacy LO without abcd validates against LearningObjective.
- LO with abcd ({audience: "Students", behavior: {verb: "identify",
action_object: "cell parts"}, condition: "from a labeled diagram",
degree: "with 90% accuracy"}) validates.
- Incomplete abcd (missing condition + degree) fails with 2 errors.
- abcd.behavior missing action_object fails as expected.
- abcd with extra field rejected via additionalProperties:false.
- schemas/tests/ all pass: 414 passed, 4 skipped.
Plan reference: plans/phase6_abcd_concept_extractor.md Subtask 1.
https://claude.ai/code/session_01Se8hZK7RFScb6R4mfe9FgX
mdmurphy822
pushed a commit
that referenced
this pull request
May 3, 2026
…chive_to_libv2 (Phase 7c.5 SHIPPING BLOCKER) Phase 7c ST 17 (commit c61e608) promoted dart_chunks_sha256 and imscc_chunks_sha256 to required fields at the LibV2ManifestValidator boundary, but the producer side - _archive_to_libv2 in MCP/tools/pipeline_tools.py - had no kwarg plumbing for them. End-to-end `ed4all run textbook_to_course` would fail closed at the libv2_archival gate's libv2_manifest validation, since the manifest would be missing the two now-required hashes. This patch closes the gap by mirroring the Phase 6 ST 18 concept_graph_sha256 plumbing pattern (commit c3a9f72): 1. config/workflows.yaml::textbook_to_course::libv2_archival adds two new ``inputs_from`` entries that route ``dart_chunks_sha256`` from the ``chunking`` phase output (Phase 7b ST 11, _run_dart_chunking) and ``imscc_chunks_sha256`` from the ``imscc_chunking`` phase output (Phase 7c ST 16, _run_imscc_chunking) through to the archival helper. 2. MCP/tools/pipeline_tools.py::_archive_to_libv2 (the registry variant inside _build_tool_registry, which is what the workflow runner actually invokes) accepts the two new kwargs (defaulting to None for legacy back-compat) and persists them into manifest when the value matches the canonical 64-hex regex. Malformed values are dropped on the producer side so the validator's MISSING_* critical fires loudly rather than masking corruption with INVALID_*. 3. lib/validators/tests/test_libv2_manifest_concept_graph.py adds three new end-to-end producer-side tests: all-three-hashes round trip, kwarg-absent omission (back-compat), and malformed-hash dropping. Per Phase 7a investigation, _archive_to_libv2 has two manifest-build sites - the @mcp.tool() variant (line 1448) and the _build_tool_registry variant (line 5688). The Phase 6 ST 18 concept_graph_sha256 change only touched site #2 (the workflow runner calls the registry variant), so this patch follows the same pattern and updates only the registry variant. The @mcp.tool() variant has no concept_graph_sha256 plumbing either; if external MCP clients ever need the new fields, that can be a separate followup. Tests: - lib/validators/tests/test_libv2_manifest_concept_graph.py: 10 pass (4 new + 6 pre-existing). - lib/validators/tests/test_libv2_manifest_dual_chunkset.py: 10 pass (Phase 7c ST 20 integration suite still green; its manual manifest fixtures still apply since they exercise the validator side, which is unchanged). - MCP/tests/test_archive_libv2_chunker_version.py: 6 pass. https://claude.ai/code/session_01Se8hZK7RFScb6R4mfe9FgX
mdmurphy822
added a commit
that referenced
this pull request
Jun 23, 2026
…olution + frozen-result stamp Follow-on to 53883c0 (which landed gap #1: the V2Config local-adapter resolver). Live real-GPU testing surfaced two more gaps that each independently blocked local-real shipping: - Gap #2: _semantik_resolve_runtime_mode only checked .runtime_mode / .cascade["runtime_mode"], but the in-process run_full_cascade result carries it ONLY at cascade["conformance_audit"]["provenance"]["runtime_mode"] -> resolver returned None -> the R4 mock-trap failed closed EVEN ON A GENUINE REAL RUN. Now falls back to the conformance_audit.provenance location (the bridge top-level-key path is preserved). - Gap #3: the seam did setattr(result, "chapters", ...) on PipelineV2Result, a frozen dataclass -> FrozenInstanceError aborted the conversion. Now uses object.__setattr__ (frozen escape hatch) with a plain-setattr fallback for the mutable bridge result. Verified end-to-end (local, no NVIDIA): the seam runs runtime_mode=real, success=True on a region-capped slice in ~44s, writing accessible HTML (data-dart-* markers, wcag_status=passed) + sidecars (ship_with_flag from the theta stub). +121 lines of seam tests (22/22). Follow-ups (unchanged): theta v8 mode-collapse (needs retrain; runs ship flagged via DART_ALLOW_THETA_STUB=1); un-capped full-slice local gen exits 144 on 8GB VRAM (DGX-Spark target / VRAM tuning is the real fix). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QfgokVwS8UMGNAmWR1SohW
mdmurphy822
added a commit
that referenced
this pull request
Jun 23, 2026
… + B15 Resources (loop cycle 1)
Works through the first batch of the FRAMEWORK.pdf progress-review action set. No stubs
(real artifacts/contracts only); slug-free (dynamic discovery); byte-stable when the IB
flags are off; warning-day-1 gates with # TODO(calibration) deferred flips.
Calibration harness (KEYSTONE — unblocks the ~9 deferred gate-family critical-flips):
- scripts/calibration_harness.py — pure measurement (never mutates/flips). Slug-free
discovery of LibV2 rollup reports + Courseforge per-block GateResult reports + decision
JSONL; corpus-IDENTITY collapsing (the 10 timestamped runs of one textbook = 1 distinct
corpus); a 15-gate-family flip-criteria table with auditable expected-bands; emits
calibration_report.json with per-gate fire-rate + sample + flip_ready. On this box: 1
distinct corpus → every family flip_ready=false (honest). Flipping needs a 2nd
distinct-corpus run (+ ED4ALL_BLOCK_QUALITY_RUBRIC/IB flags on). 9 tests.
Keyboard + time-based-media a11y (real WCAG holes, not flag-flips) — lib/validators/rewrite_html_shape.py:
- BLOCK_KEYBOARD_OPERABLE (2.1.1): custom click-only non-native control w/o keyboard
affordance flagged; native button/a/input/details = escape hatch.
- BLOCK_FOCUS_VISIBLE (2.4.7): inline outline:none w/o replacement focus indicator flagged.
- B04 per-piece: MULTIMEDIA_{CONTROLS,CAPTIONS,AUDIO_DESC,TRANSCRIPT}_MISSING (1.2.2/1.2.4/
1.2.5) — AD check matches the actual renderer output (class="audio-description"). 19 tests.
B15 Resources (closes the last canonical-catalog gap — every B-code now has an Ed4All primary):
- resources block_type → framework_block B15 (block_catalog.yaml), IB5 default-off posture;
_render_resources_section (accessible descriptive links, never bare-URL); new
ResourceLinkPurposeValidator (2.4.4, RESOURCE_LINK_PURPOSE_UNCLEAR) wired warning-day-1 at
post_rewrite_validation in both two-pass workflows; planner nudge (further-reading shape);
JSON-LD enum + router-policy matrix + count tests 28->29; IB2 reconciliation table +
gate-count table re-derived (62/107/169). 19 tests.
Combined sweep: 723 passed. Pre-existing-unrelated: test_outline_seam_uses_block_validators
(fails on clean stash — IB3 outline-seam, not touched here).
Remaining in the action set: #4 anchored-rubric producer + #6 rollup->final_status wiring
(next iteration); #2 verb-triple flip + #6 enforcement are blocked on a 2nd-corpus FP
measurement (a GPU cycle). pipeline_tools.py decomposition stays deferred (monkeypatch
contract). .gitignore: extracted/ + calibration_report.json.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QfgokVwS8UMGNAmWR1SohW
mdmurphy822
added a commit
that referenced
this pull request
Jul 1, 2026
…eline infrastructure (#2) Trainforge RAG improvements: - Content-grounded question generation with term/statement/relationship extraction - BM25 retrieval with n-gram boosting and configurable relevance thresholds - Bloom's taxonomy chunk strategy mapping and deduplication - New ContentExtractor, LeakCheckValidator, and QuestionQualityValidator Pipeline infrastructure: - Standalone MCP pipeline tools with full textbook-to-course workflow - CLI wiring and parameter routing fixes - Workflow config updates for new pipeline stages Courseforge: - Multi-file course generation script Schema & validation: - Migrate schema $id from HTTP URLs to URN identifiers - Add local RefResolver for $ref resolution without HTTP fetches - Add "quality" subdirectory support to LibV2 importer Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
mdmurphy822
added a commit
that referenced
this pull request
Jul 1, 2026
Contamination audit #2 follow-up: 1. DART/templates/gold_standard.html — slim-reference rewrite. Replaced ~1130 lines of real paper content (Quantum Agents by Sultanow et al., arXiv:2506.01536) with a ~490-line synthetic reference demonstrating each structural pattern once (h1, abstract, TOC, 3 sections with h3 subheadings, definition block, algorithm block, accessible table, figure+figcaption, doc-footnote aside, references, footer). Preserves CSS bundle, skip-link, ARIA roles, schema.org JSON-LD, WCAG 2.2 AA scaffolding byte-for-byte — only the prose content is swapped for synthetic placeholders (Alice Sample / Bob Example / arXiv:0000.00000 etc.). 2. MCP/tools/_content_gen_helpers.py — 8 comment substitutions. Rewrote corpus-specific byline-filter examples (Maria Keet / Hung-Nghiep / Atsuhiro Takasu / A.W.) into abstract synthetic examples (Jane Doe / A.B. / J.K. / Author Name) while preserving each comment's technical teaching value. 3. DART/pdf_converter/claude_processor.py:293 — expanded docstring. Added explicit "currently unused — reserved" note for the gold_standard_template parameter plus TODO(wave-31) marker so future maintainers know the parameter is intentional scaffolding. Zero runtime behavior changes. Pure string/comment/markup substitution. Grep audit after: 0 matches on sultanow|tehrani|dutta|buchanan|khan|quant-ph|keet|hung.nghiep| atsuhiro|maria.keet|tony.bates|a\.w\..+bates across the 3 files. Tests: 1513 passed + 3 skipped (baseline match, delta=0). Integrity check: 8/8 PASSED. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mdmurphy822
added a commit
that referenced
this pull request
Jul 1, 2026
Four focused fixes following Waves 22-29, closing findings from the MCP audit #2 pass: 1. Fold @mcp.tool() package_imscc onto the mature multi-file packager. Pre-fold the tool flipped project_config.status and attempted a LibV2 copy without ever building the zip — a silent-failure mirror of the Wave 22 F2 extract_and_convert_pdf bug. Now delegates to Courseforge.scripts.package_multifile_imscc matching the Wave 27 registry-side fold: per-week LO-contract validation, course_metadata.json bundling, IMS CC v1.3 namespaces, week-grouped manifest. Legacy JSON envelope preserved so external clients see no contract change. 2. Add figures_dir to TOOL_SCHEMAS[extract_and_convert_pdf]. The Wave 17 signature gained figures_dir but the schema never tracked it, causing strict param-mapping to silently drop external callers' kwarg. Generic sig-vs-schema parity walker added in test_tool_schema_signature_parity.py prevents future regressions. 3. Runtime DeprecationWarning on create_textbook_pipeline_tool and run_textbook_pipeline_tool (both Wave-7 deprecated). Matches the CLI ed4all textbook-to-course stderr deprecation notice so the MCP + CLI surfaces are consistent. 4. Graceful deprecation for create_course_project (post-Wave-24 no agent mapping points at it; canonical path runs through extract_textbook_structure + plan_course_structure). Warning cites the replacement pair; schema description prefixed with [DEPRECATED — ...]. Function body remains functional for external clients. Smoke test: @mcp.tool() package_imscc against a 1-week fixture project produces a real 1.7 KB .imscc zip with 3 HTML pages + imsmanifest.xml and passes per-week LO-contract validation. Tests: +12 new (2277 -> 2289). Integrity check 8/8. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mdmurphy822
added a commit
that referenced
this pull request
Jul 1, 2026
From the 4-worker code review pass. Scope: non-architectural critical items. Defers the LocalDispatcher / runner wiring gap (#2), poison-pill asyncio.gather cancellation (#9), and LLM capture-wiring regression tests (#16) to a separate architectural wave. 1. **Source-router slug alignment** (`MCP/tools/pipeline_tools.py`): `_build_source_module_map` now lowercases + space-to-hyphens the sidecar stem, matching Wave 35's `_topic_source_references` and `ContentGroundingValidator`. Pre-Wave-36, a staging stem like `XYZ_201_synthesized` emitted router refs as `dart:XYZ_201#...` while the validator + content-generator lowercased — uppercase or space-containing corpora silently failed the `source_refs` gate. 2. **Retry backoff actually sleeps** (`MCP/core/executor.py`): `RetryPolicy` is now imported, instantiated in `_init_hardening` with `base_delay = config.retry_delay_seconds`, and `_execute_with_retries` awaits the computed delay between attempts. Pre-Wave-36 the loop re-dispatched immediately, which for rate-limited LLM calls meant firing `max_retries` requests inside the provider's cooldown window. Under pytest we short-circuit the sleep so the test suite doesn't stretch into minutes when exercising the retry paths. 3. **Root CLAUDE.md workflow list** — add the `training_synthesis` phase to the `textbook_to_course` narrative (phase 9, pushing `libv2_archival` → 10 and `finalization` → 11). Add `training-synthesizer` to the Trainforge Agents table. Pre-Wave-36 the phase was wired in `config/workflows.yaml` and `AGENT_TOOL_MAPPING` but invisible in the root guide. 4. **Root CLAUDE.md Active Gates table** — add a Phase column so operators can see which phase each gate fires on. Cross-checked against `config/workflows.yaml::validation_gates` via a YAML dump; the previous narrative conflated `content_generation` and `packaging` gates on `textbook_to_course` (`page_objectives` is on `packaging`, not `content_generation`). 5. **DART/convert.py docstring** — "WCAG 2.1 AA" → "WCAG 2.2 AA" to match every other surface (README, CLAUDE.md, converter code). 6. **LibV2 chunks.json → chunks.jsonl** in `LibV2/README.md:24` and `LibV2/CLAUDE.md:30,48`. The on-disk file is `chunks.jsonl`; docs mixed both names. Full suite: 2423 passed, 7 skipped. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mdmurphy822
added a commit
that referenced
this pull request
Jul 1, 2026
Self-review of the Wave 63 SHACL implementation surfaced six issues. This wave closes all of them, with regression tests for each, plus one pyld interaction bug caught by the end-to-end round trip. Fixes (by Wave 63 review item): #1 Closed-set vocab validation. bloomLevel / cognitiveDomain / hierarchyLevel / bloomRange now use sh:in against the canonical concept IRI set instead of sh:pattern prefix matching. Wave 63's prefix check accepted typos like <https://ed4all.dev/vocab/bloom#aplly> because they shared the namespace prefix; Wave 67 rejects them. #2 parentObjectiveId now carries sh:pattern ".*[/#](TO|CO)-\\d{2,}$" on the IRI's lexical form. Wave 63 only required sh:nodeKind sh:IRI, letting any URL through — including non-LO URIs. #3 New cfshapes:SectionShape targeting ed4all:Section. Requires schema:name (heading) and validates bloomRange items. Wave 63 had no Section shape at all; malformed sections validated silently. #4 NodeShape target classes now point at ed4all:CourseModule / ed4all:LearningObjective / ed4all:Section rather than their Schema.org equivalents. Wave 63 over-targeted — a Pearson-emitted schema:LearningResource in a mixed graph would trigger our required-predicate constraints and fail. Wave 67 stays scoped to OUR emit; Schema.org inference is still available via the Wave 65 vocabulary's rdfs:subClassOf axioms. #5 Property shapes now declare sh:node references (schema:teaches → LearningObjectiveShape, schema:hasPart → SectionShape, ed4all:hasMisconception → MisconceptionShape, ed4all:bloomDistribution → BloomDistributionShape, ed4all:targetsConcept → TargetedConceptShape). Wave 63 relied on class-based targeting alone; the parent-child shape relationship was implicit. Wave 67 makes it declarative. #6 CourseModuleShape now has an ed4all:hasMisconception property constraint (was absent in Wave 63). Ripple fixes surfaced along the way: * pyld @vocab + @container:@list/@set interaction bug: when two @context terms share the same @id and one uses @vocab + @container, the container-scoped @vocab resolution fails, producing literal @values instead of IRI references in the expanded RDF. This broke the end-to-end SHACL round trip. Wave 67 mitigation: bloomLevels and bloomVerbs (plural convenience fields for Wave 58) are suppressed from RDF projection via "bloomLevels": null / "bloomVerbs": null in the @context; the singular bloomLevel / bloomVerb carry the authoritative RDF predicate. bloomRange drops @type:@vocab and emits as string literals — SHACL uses sh:in against the string enum. The JSON wire format is unchanged; only the RDF projection of these three fields changes. * sections: @container switched from @list to @set so sh:node constraints traverse per-item (SHACL doesn't recurse rdf:List heads by default). JSON wire stays an array; order is preserved in practice by pyld compaction. RDF consumers that need strict ordering can add a schema:position field per section. Wave 62 / 64 test updates: * test_course_module_type_expands_to_schema_learning_resource → renamed to *_ed4all_course_module with the updated expansion assertion. Schema.org inference is now via vocabulary subClassOf, not direct @type alias. * test_jsonld_context_loader asserts ed4all:CourseModule in the expanded @type as proof-of-loader-served-context. New regression tests (test_courseforge_shacl_shapes.py): - External schema:LearningResource doesn't fire CourseModuleShape (over-targeting closed). - Typo bloom IRI in correct namespace fails (sh:in closed-set check). - Empty-fragment bloom IRI fails. - Non-canonical parentObjectiveId fails the pattern. - Canonical TO-01 parent ID passes. - Section without heading fails SectionShape. - Well-formed section with bloomRange passes. - bloomRange with typo value fails (string-literal sh:in). - Empty-string statement fails sh:minLength 1. Test suite: 2694 passed, 7 skipped (up from 2685; +9 new tests). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mdmurphy822
added a commit
that referenced
this pull request
Jul 1, 2026
Bug #2: _build_source_module_map round-robin fallback was emitting the alphabetically-first DART block as primary on every page; relegate fallback blocks to the contributing role and pin via regression test. Bug #3: extend lib/ontology/tech_anchors.py from the Wave 82 W3C set (9 anchors) to a Wave 84 set covering 23 surface-form regexes — serialization formats (TriG, N-Quads, RDF/XML), RDF foundationals (IRI, literal, datatype, blank-node, rdf-dataset), SHACL shapes (node-shape, property-shape), RDFS predicates (subClassOf, subPropertyOf, rdf:type), and Turtle prefix detection. Backed by parametrized fixture tests covering case sensitivity and false-positive guards (e.g., lowercase "trig" math function NOT matched). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mdmurphy822
added a commit
that referenced
this pull request
Jul 1, 2026
Three correctness fixes from the v0.2.2 review pass. 1. LibV2 SHACL import gate (importer.py) — remove non-functional gate. `CourseManifest.to_dict()` carries no ed4all: @type nodes, so the Courseforge SHACL shapes validated zero focus nodes and every NodeShape conformed vacuously. The gate was a silent no-op in production. The lower-level `_shacl_validator.validate_manifest_shacl` helper stays available for future callers that feed it real Courseforge JSON-LD. 2. Misconception ID seed (preference_factory.py, process_course.py) — bloom-less path now hashes a 2-field seed `{statement}|{correction}` instead of `{statement}|{correction}|` (trailing pipe). Pre-Wave-72 the trailing pipe rekeyed every legacy / pre-Wave-60 misconception, contradicting the docstring's explicit "pre-Wave-69 hash stable for the bloom-less path" claim. Both hash call sites now stay in lock-step via the same two-form branch. 3. LO dedup in `_build_objectives_metadata_for_graph` — two-pass restructure (process_course.py). Pre-Wave-72 each item ran Path 1 (JSON-LD) then Path 2 (dataclass fallback), interleaved across items. An early item's empty dataclass LO could shadow a later item's rich JSON-LD LO for the same ID, silently discarding `targetedConcepts[]`. Now Path 1 runs across all items first, then Path 2 fills any IDs still absent — the "JSON-LD preferred" docstring promise actually holds. Regression tests added for #2 (2-field seed shape) and #3 (legacy + rich items across pages). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mdmurphy822
pushed a commit
that referenced
this pull request
Jul 1, 2026
…6 Subtask 1)
Extends courseforge_jsonld_v1.schema.json with a Phase 6 ABCD framework
discrete-fields contract for learning objectives. Gives the course-
outliner agent a constrained-decoding-validated structured output target
and gives lib.validators.abcd_objective.AbcdObjectiveValidator (Subtask
4) a stable shape to gate verb-Bloom alignment against.
Schema changes:
- New $defs.AbcdObjective: required {audience, behavior, condition,
degree}; behavior is a nested {verb, action_object} object with both
fields required. additionalProperties:false on both levels for tight
shape enforcement.
- New LearningObjective.properties.abcd: $ref to AbcdObjective.
Required-vs-optional decision:
- abcd is OPTIONAL on LearningObjective (NOT added to required[]) so
legacy synthesized_objectives.json fixtures and pre-Phase-6 corpora
continue to validate. The AbcdObjective sub-object itself enforces
all four ABCD fields when abcd is present (per pre-resolved decision
#2 in plans/phase6_abcd_concept_extractor.md).
- Phase 7+ will promote abcd to LearningObjective.required once corpus
calibration confirms safe.
Verification:
- Draft 2020-12 schema-check: OK (jsonschema.Draft202012Validator.
check_schema passes).
- Legacy LO without abcd validates against LearningObjective.
- LO with abcd ({audience: "Students", behavior: {verb: "identify",
action_object: "cell parts"}, condition: "from a labeled diagram",
degree: "with 90% accuracy"}) validates.
- Incomplete abcd (missing condition + degree) fails with 2 errors.
- abcd.behavior missing action_object fails as expected.
- abcd with extra field rejected via additionalProperties:false.
- schemas/tests/ all pass: 414 passed, 4 skipped.
Plan reference: plans/phase6_abcd_concept_extractor.md Subtask 1.
https://claude.ai/code/session_01Se8hZK7RFScb6R4mfe9FgX
mdmurphy822
pushed a commit
that referenced
this pull request
Jul 1, 2026
…chive_to_libv2 (Phase 7c.5 SHIPPING BLOCKER) Phase 7c ST 17 (commit 5702095) promoted dart_chunks_sha256 and imscc_chunks_sha256 to required fields at the LibV2ManifestValidator boundary, but the producer side - _archive_to_libv2 in MCP/tools/pipeline_tools.py - had no kwarg plumbing for them. End-to-end `ed4all run textbook_to_course` would fail closed at the libv2_archival gate's libv2_manifest validation, since the manifest would be missing the two now-required hashes. This patch closes the gap by mirroring the Phase 6 ST 18 concept_graph_sha256 plumbing pattern (commit ce991c0): 1. config/workflows.yaml::textbook_to_course::libv2_archival adds two new ``inputs_from`` entries that route ``dart_chunks_sha256`` from the ``chunking`` phase output (Phase 7b ST 11, _run_dart_chunking) and ``imscc_chunks_sha256`` from the ``imscc_chunking`` phase output (Phase 7c ST 16, _run_imscc_chunking) through to the archival helper. 2. MCP/tools/pipeline_tools.py::_archive_to_libv2 (the registry variant inside _build_tool_registry, which is what the workflow runner actually invokes) accepts the two new kwargs (defaulting to None for legacy back-compat) and persists them into manifest when the value matches the canonical 64-hex regex. Malformed values are dropped on the producer side so the validator's MISSING_* critical fires loudly rather than masking corruption with INVALID_*. 3. lib/validators/tests/test_libv2_manifest_concept_graph.py adds three new end-to-end producer-side tests: all-three-hashes round trip, kwarg-absent omission (back-compat), and malformed-hash dropping. Per Phase 7a investigation, _archive_to_libv2 has two manifest-build sites - the @mcp.tool() variant (line 1448) and the _build_tool_registry variant (line 5688). The Phase 6 ST 18 concept_graph_sha256 change only touched site #2 (the workflow runner calls the registry variant), so this patch follows the same pattern and updates only the registry variant. The @mcp.tool() variant has no concept_graph_sha256 plumbing either; if external MCP clients ever need the new fields, that can be a separate followup. Tests: - lib/validators/tests/test_libv2_manifest_concept_graph.py: 10 pass (4 new + 6 pre-existing). - lib/validators/tests/test_libv2_manifest_dual_chunkset.py: 10 pass (Phase 7c ST 20 integration suite still green; its manual manifest fixtures still apply since they exercise the validator side, which is unchanged). - MCP/tests/test_archive_libv2_chunker_version.py: 6 pass. https://claude.ai/code/session_01Se8hZK7RFScb6R4mfe9FgX
mdmurphy822
added a commit
that referenced
this pull request
Jul 1, 2026
…olution + frozen-result stamp Follow-on to a09cb51 (which landed gap #1: the V2Config local-adapter resolver). Live real-GPU testing surfaced two more gaps that each independently blocked local-real shipping: - Gap #2: _semantik_resolve_runtime_mode only checked .runtime_mode / .cascade["runtime_mode"], but the in-process run_full_cascade result carries it ONLY at cascade["conformance_audit"]["provenance"]["runtime_mode"] -> resolver returned None -> the R4 mock-trap failed closed EVEN ON A GENUINE REAL RUN. Now falls back to the conformance_audit.provenance location (the bridge top-level-key path is preserved). - Gap #3: the seam did setattr(result, "chapters", ...) on PipelineV2Result, a frozen dataclass -> FrozenInstanceError aborted the conversion. Now uses object.__setattr__ (frozen escape hatch) with a plain-setattr fallback for the mutable bridge result. Verified end-to-end (local, no NVIDIA): the seam runs runtime_mode=real, success=True on a region-capped slice in ~44s, writing accessible HTML (data-dart-* markers, wcag_status=passed) + sidecars (ship_with_flag from the theta stub). +121 lines of seam tests (22/22). Follow-ups (unchanged): theta v8 mode-collapse (needs retrain; runs ship flagged via DART_ALLOW_THETA_STUB=1); un-capped full-slice local gen exits 144 on 8GB VRAM (DGX-Spark target / VRAM tuning is the real fix). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QfgokVwS8UMGNAmWR1SohW
mdmurphy822
added a commit
that referenced
this pull request
Jul 1, 2026
… + B15 Resources (loop cycle 1)
Works through the first batch of the FRAMEWORK.pdf progress-review action set. No stubs
(real artifacts/contracts only); slug-free (dynamic discovery); byte-stable when the IB
flags are off; warning-day-1 gates with # TODO(calibration) deferred flips.
Calibration harness (KEYSTONE — unblocks the ~9 deferred gate-family critical-flips):
- scripts/calibration_harness.py — pure measurement (never mutates/flips). Slug-free
discovery of LibV2 rollup reports + Courseforge per-block GateResult reports + decision
JSONL; corpus-IDENTITY collapsing (the 10 timestamped runs of one textbook = 1 distinct
corpus); a 15-gate-family flip-criteria table with auditable expected-bands; emits
calibration_report.json with per-gate fire-rate + sample + flip_ready. On this box: 1
distinct corpus → every family flip_ready=false (honest). Flipping needs a 2nd
distinct-corpus run (+ ED4ALL_BLOCK_QUALITY_RUBRIC/IB flags on). 9 tests.
Keyboard + time-based-media a11y (real WCAG holes, not flag-flips) — lib/validators/rewrite_html_shape.py:
- BLOCK_KEYBOARD_OPERABLE (2.1.1): custom click-only non-native control w/o keyboard
affordance flagged; native button/a/input/details = escape hatch.
- BLOCK_FOCUS_VISIBLE (2.4.7): inline outline:none w/o replacement focus indicator flagged.
- B04 per-piece: MULTIMEDIA_{CONTROLS,CAPTIONS,AUDIO_DESC,TRANSCRIPT}_MISSING (1.2.2/1.2.4/
1.2.5) — AD check matches the actual renderer output (class="audio-description"). 19 tests.
B15 Resources (closes the last canonical-catalog gap — every B-code now has an Ed4All primary):
- resources block_type → framework_block B15 (block_catalog.yaml), IB5 default-off posture;
_render_resources_section (accessible descriptive links, never bare-URL); new
ResourceLinkPurposeValidator (2.4.4, RESOURCE_LINK_PURPOSE_UNCLEAR) wired warning-day-1 at
post_rewrite_validation in both two-pass workflows; planner nudge (further-reading shape);
JSON-LD enum + router-policy matrix + count tests 28->29; IB2 reconciliation table +
gate-count table re-derived (62/107/169). 19 tests.
Combined sweep: 723 passed. Pre-existing-unrelated: test_outline_seam_uses_block_validators
(fails on clean stash — IB3 outline-seam, not touched here).
Remaining in the action set: #4 anchored-rubric producer + #6 rollup->final_status wiring
(next iteration); #2 verb-triple flip + #6 enforcement are blocked on a 2nd-corpus FP
measurement (a GPU cycle). pipeline_tools.py decomposition stays deferred (monkeypatch
contract). .gitignore: extracted/ + calibration_report.json.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QfgokVwS8UMGNAmWR1SohW
mdmurphy822
added a commit
that referenced
this pull request
Jul 7, 2026
…olution + frozen-result stamp Follow-on to 1987972 (which landed gap #1: the V2Config local-adapter resolver). Live real-GPU testing surfaced two more gaps that each independently blocked local-real shipping: - Gap #2: _semantik_resolve_runtime_mode only checked .runtime_mode / .cascade["runtime_mode"], but the in-process run_full_cascade result carries it ONLY at cascade["conformance_audit"]["provenance"]["runtime_mode"] -> resolver returned None -> the R4 mock-trap failed closed EVEN ON A GENUINE REAL RUN. Now falls back to the conformance_audit.provenance location (the bridge top-level-key path is preserved). - Gap #3: the seam did setattr(result, "chapters", ...) on PipelineV2Result, a frozen dataclass -> FrozenInstanceError aborted the conversion. Now uses object.__setattr__ (frozen escape hatch) with a plain-setattr fallback for the mutable bridge result. Verified end-to-end (local, no NVIDIA): the seam runs runtime_mode=real, success=True on a region-capped slice in ~44s, writing accessible HTML (data-dart-* markers, wcag_status=passed) + sidecars (ship_with_flag from the theta stub). +121 lines of seam tests (22/22). Follow-ups (unchanged): theta v8 mode-collapse (needs retrain; runs ship flagged via DART_ALLOW_THETA_STUB=1); un-capped full-slice local gen exits 144 on 8GB VRAM (DGX-Spark target / VRAM tuning is the real fix). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QfgokVwS8UMGNAmWR1SohW
mdmurphy822
added a commit
that referenced
this pull request
Jul 7, 2026
… + B15 Resources (loop cycle 1)
Works through the first batch of the FRAMEWORK.pdf progress-review action set. No stubs
(real artifacts/contracts only); slug-free (dynamic discovery); byte-stable when the IB
flags are off; warning-day-1 gates with # TODO(calibration) deferred flips.
Calibration harness (KEYSTONE — unblocks the ~9 deferred gate-family critical-flips):
- scripts/calibration_harness.py — pure measurement (never mutates/flips). Slug-free
discovery of LibV2 rollup reports + Courseforge per-block GateResult reports + decision
JSONL; corpus-IDENTITY collapsing (the 10 timestamped runs of one textbook = 1 distinct
corpus); a 15-gate-family flip-criteria table with auditable expected-bands; emits
calibration_report.json with per-gate fire-rate + sample + flip_ready. On this box: 1
distinct corpus → every family flip_ready=false (honest). Flipping needs a 2nd
distinct-corpus run (+ ED4ALL_BLOCK_QUALITY_RUBRIC/IB flags on). 9 tests.
Keyboard + time-based-media a11y (real WCAG holes, not flag-flips) — lib/validators/rewrite_html_shape.py:
- BLOCK_KEYBOARD_OPERABLE (2.1.1): custom click-only non-native control w/o keyboard
affordance flagged; native button/a/input/details = escape hatch.
- BLOCK_FOCUS_VISIBLE (2.4.7): inline outline:none w/o replacement focus indicator flagged.
- B04 per-piece: MULTIMEDIA_{CONTROLS,CAPTIONS,AUDIO_DESC,TRANSCRIPT}_MISSING (1.2.2/1.2.4/
1.2.5) — AD check matches the actual renderer output (class="audio-description"). 19 tests.
B15 Resources (closes the last canonical-catalog gap — every B-code now has an Ed4All primary):
- resources block_type → framework_block B15 (block_catalog.yaml), IB5 default-off posture;
_render_resources_section (accessible descriptive links, never bare-URL); new
ResourceLinkPurposeValidator (2.4.4, RESOURCE_LINK_PURPOSE_UNCLEAR) wired warning-day-1 at
post_rewrite_validation in both two-pass workflows; planner nudge (further-reading shape);
JSON-LD enum + router-policy matrix + count tests 28->29; IB2 reconciliation table +
gate-count table re-derived (62/107/169). 19 tests.
Combined sweep: 723 passed. Pre-existing-unrelated: test_outline_seam_uses_block_validators
(fails on clean stash — IB3 outline-seam, not touched here).
Remaining in the action set: #4 anchored-rubric producer + #6 rollup->final_status wiring
(next iteration); #2 verb-triple flip + #6 enforcement are blocked on a 2nd-corpus FP
measurement (a GPU cycle). pipeline_tools.py decomposition stays deferred (monkeypatch
contract). .gitignore: extracted/ + calibration_report.json.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QfgokVwS8UMGNAmWR1SohW
mdmurphy822
added a commit
that referenced
this pull request
Jul 7, 2026
…x GRACEFUL_STOP envelope P3 of plans/graceful-stop-checkpoint-on-command-2026-07.md. Every existing resume-sidecar loop now honors 'ed4all stop' at unit boundaries with the mechanical guarantee sidecar-records == provider-calls (worst-case loss = one in-flight LLM call). - pipeline_tools: stage-2 window synthesis + TO clusters, concept-extraction Stage-3 windows (STOP_MARKER return inside gathered coroutines; post-gather append-then-raise — the blocker-#1 pattern), outline tier, rewrite sequential/thread-pool/batched-lane pre-flight, objective-review + stage-3 GracefulStopRequested carve-outs before broad handlers, optional blanket checks in validators + dart_chunking. - objective_grounding: throttled StopPoller in the per-candidate NLI loop. - synthesize_training: sentinel stop co-located with the budget-exhausted break; surfaces in telemetry; phase not marked complete on stop. - Mailbox mode (blocker #2): servicer catches GracefulStopRequested -> GRACEFUL_STOP completion envelope (before the TOOL_RAISED catch-all) + pre-claim sentinel check; local_dispatcher maps GRACEFUL_STOP -> ExecutionResult PAUSED (no retry, no poison); task_mailbox poll loops race the sentinel so waiters return paused instead of sitting out the timeout (MailboxBrokeredBackend inherits). - Tests: 39 new stop tests across 7 files; R4 regression pins + dispatcher/ mailbox/executor/runner sweep = 214 passed, 0 failed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ContentExtractor,LeakCheckValidator, andQuestionQualityValidator$idfrom HTTP URLs to URN identifiers, add localRefResolverfor$refresolution, add "quality" subdirectory to LibV2 importerTest plan
pytest Trainforge/tests/test_content_grounded_generation.py— 24 tests passpytest Trainforge/tests/test_retrieval_improvements.py— 13 tests passgit diff origin/main --statshows 27 code files only)🤖 Generated with Claude Code