feat: imported-course mode for needs-analysis, assessment-design, course-builder - #9
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces 'Imported-course mode' for the needs-analysis, assessment-design, and course-builder skills, enabling specialized workflows for imported content such as auditing existing assessments and gap-fill generation. The manifest schema is updated with mode-tracking fields and audit note arrays, and the assessment-design skill is updated to use the idstack-manifest-merge tool for atomic writes. Feedback identifies misleading section descriptions in the assessment-design skill and corrects field name inconsistencies in the course-builder skill to ensure alignment with the canonical schema.
| and atomically updates the top-level `updated` timestamp. **This skill writes two | ||
| sections** — `assessments` and `learning_objectives.alignment_matrix.ilo_to_assessment` — | ||
| so call the merge tool twice (once per top-level section). |
There was a problem hiding this comment.
The description of the sections written by this skill is misleading. It identifies learning_objectives.alignment_matrix.ilo_to_assessment as a "section", but the idstack-manifest-merge tool operates on top-level sections (like learning_objectives). Furthermore, in Mode 3, the skill also writes to learning_objectives.alignment_matrix.gaps. It is clearer and more accurate to just name the top-level sections.
| and atomically updates the top-level `updated` timestamp. **This skill writes two | |
| sections** — `assessments` and `learning_objectives.alignment_matrix.ilo_to_assessment` — | |
| so call the merge tool twice (once per top-level section). | |
| and atomically updates the top-level `updated` timestamp. **This skill writes two | |
| sections** — `assessments` and `learning_objectives` — | |
| so call the merge tool twice (once per top-level section). |
| and atomically updates the top-level `updated` timestamp. **This skill writes two | ||
| sections** — `assessments` and `learning_objectives.alignment_matrix.ilo_to_assessment` — | ||
| so call the merge tool twice (once per top-level section). |
There was a problem hiding this comment.
The description of the sections written by this skill is misleading. It identifies learning_objectives.alignment_matrix.ilo_to_assessment as a "section", but the idstack-manifest-merge tool operates on top-level sections (like learning_objectives). Furthermore, in Mode 3, the skill also writes to learning_objectives.alignment_matrix.gaps. It is clearer and more accurate to just name the top-level sections.
and atomically updates the top-level `updated` timestamp. **This skill writes two
sections** — `assessments` and `learning_objectives` —
so call the merge tool twice (once per top-level section).
| 4. **Record outputs.** Update `course_content.generated_files` (additive) and `course_builder_notes.recommended_generation_targets` (the list, with status `generated | deferred | declined`). | ||
|
|
||
| When done, write the manifest via `bin/idstack-manifest-merge` (see Write Manifest below) and skip directly to the final summary. | ||
|
|
||
| Save the chosen mode under `course_builder_notes.mode` (`"build-new"` or `"gap-fill"`). |
There was a problem hiding this comment.
The instructions refer to course_builder_notes for the mode and recommended_generation_targets fields, but the manifest schema (and the generated_files field in the same sentence) uses course_content. This inconsistency will lead to the LLM writing data to a non-existent section.
| 4. **Record outputs.** Update `course_content.generated_files` (additive) and `course_builder_notes.recommended_generation_targets` (the list, with status `generated | deferred | declined`). | |
| When done, write the manifest via `bin/idstack-manifest-merge` (see Write Manifest below) and skip directly to the final summary. | |
| Save the chosen mode under `course_builder_notes.mode` (`"build-new"` or `"gap-fill"`). | |
| 4. **Record outputs.** Update `course_content.generated_files` (additive) and `course_content.recommended_generation_targets` (the list, with status `generated | deferred | declined`). | |
| When done, write the manifest via `bin/idstack-manifest-merge` (see Write Manifest below) and skip directly to the final summary. | |
| Save the chosen mode under `course_content.mode` (`"build-new"` or `"gap-fill"`). |
| 4. **Record outputs.** Update `course_content.generated_files` (additive) and `course_builder_notes.recommended_generation_targets` (the list, with status `generated | deferred | declined`). | ||
|
|
||
| When done, write the manifest via `bin/idstack-manifest-merge` (see Write Manifest below) and skip directly to the final summary. | ||
|
|
||
| Save the chosen mode under `course_builder_notes.mode` (`"build-new"` or `"gap-fill"`). |
There was a problem hiding this comment.
The instructions refer to course_builder_notes for the mode and recommended_generation_targets fields, but the manifest schema (and the generated_files field in the same sentence) uses course_content. This inconsistency will lead to the LLM writing data to a non-existent section.
4. **Record outputs.** Update `course_content.generated_files` (additive) and `course_content.recommended_generation_targets` (the list, with status `generated | deferred | declined`).
When done, write the manifest via `bin/idstack-manifest-merge` (see Write Manifest below) and skip directly to the final summary.
Save the chosen mode under `course_content.mode` (`"build-new"` or `"gap-fill"`).
Two corrections from Gemini Code Assist's review of this PR: - assessment-design: the manifest-write text said "writes two sections — assessments and learning_objectives.alignment_matrix.ilo_to_assessment" but the merge tool operates on top-level sections only, and Mode 3 also writes to learning_objectives.alignment_matrix.gaps. Clarified to "writes two top-level sections — assessments and learning_objectives" with a note about reading-then-merging the full learning_objectives section before passing it to the merge tool. - course-builder: gap-fill mode instructions referenced course_builder_notes.recommended_generation_targets and course_builder_notes.mode, but the canonical schema (and the course_content.generated_files reference in the same sentence) puts these under course_content. Real bug — Claude following the skill would write to a section that isn't whitelisted by bin/idstack-manifest-merge. Renamed both to course_content. Smoke test 153/153 unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…rse-builder Three skills assumed net-new course design and broke down for imported courses (TMC-430 test report issues #7, #13, #14). Each now branches early on import_metadata.source: - needs-analysis: skips the "is training the right intervention?" decision gate for credit-bearing imports (the registrar can't be told to remove the course); records justification automatically and runs a design-fit check. - assessment-design: adds Mode 3 (Audit Existing Assessments) alongside the existing Mode 1/Mode 2 split. Reads existing rubrics, classifies on Bloom's, compares to ILOs, surfaces alignment gaps. Does NOT propose new assessments unless asked. Mode 3 takes precedence over Mode 1. - course-builder: adds gap-fill mode triggered when import_metadata.source is set and course_content.modules is non-empty. Generates ONLY artifacts upstream skills flagged as missing, instead of regenerating syllabus, modules, and rubrics that already exist in the cartridge. All three skills announce the chosen mode at the start of the conversation, so the user can override if auto-detection is wrong. Canonical schema gains optional needs_analysis.mode, assessments.mode, course_content.mode (record the mode), assessments.audit_notes[] (Mode 3 output), and course_content.recommended_generation_targets[] (gap-fill output). Additive — no schema version bump. assessment-design also switches its manifest write to bin/idstack-manifest-merge (landed in v2.1.0.0), eliminating the misleading "include the COMPLETE schema structure" instruction. Smoke test grows from 150 to 153 assertions: each mode-aware skill template must reference import_metadata.source (drift guard). Resolves test-report issues #7, #13, #14. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two corrections from Gemini Code Assist's review of this PR: - assessment-design: the manifest-write text said "writes two sections — assessments and learning_objectives.alignment_matrix.ilo_to_assessment" but the merge tool operates on top-level sections only, and Mode 3 also writes to learning_objectives.alignment_matrix.gaps. Clarified to "writes two top-level sections — assessments and learning_objectives" with a note about reading-then-merging the full learning_objectives section before passing it to the merge tool. - course-builder: gap-fill mode instructions referenced course_builder_notes.recommended_generation_targets and course_builder_notes.mode, but the canonical schema (and the course_content.generated_files reference in the same sentence) puts these under course_content. Real bug — Claude following the skill would write to a section that isn't whitelisted by bin/idstack-manifest-merge. Renamed both to course_content. Smoke test 153/153 unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3e57254 to
116efe5
Compare
Summary
Three skills (needs-analysis, assessment-design, course-builder) assumed net-new course design and broke down for courses imported from a cartridge. The TMC-430 test report flagged this as Cluster #C — three issues with the same root cause. This PR adds an early branch on `import_metadata.source` to each affected skill.
All three announce the chosen mode at the start of the conversation, so the user can override the auto-detection if it's wrong.
Mode-detection contract
Both conditions must be true to enter the audit-existing/gap-fill branch:
Default is design-new mode — the conservative choice. Borderline states (e.g., `import_metadata.source` set but zero modules imported) ask the user via `AskUserQuestion` rather than guessing.
Schema additions (additive, no version bump)
New optional fields in `templates/manifest-schema.md`:
Existing manifests stay readable; downstream skills that don't read the new fields ignore them.
Other changes
Test plan
Risks (covered)
Out of scope (deferred — see TMC-430 test report)
🤖 Generated with Claude Code