feat: add QTI 3.0 schema validator

[rtibblesbot]

Summary

Adds validate_qti_item(), a standalone backend function that validates a QTI item's raw XML against the official QTI 3.0 item XSD schema (imsqti_itemv3p0p1_v1p0.xsd), independent of whether the XML came from the editor, ricecooker, or a direct sync-API write. Returns a QTIValidationResult (pass/fail plus structured errors with message/line/column) instead of raising.

The XSD tree (QTI item schema + its W3C/MathML/SSML dependencies) is vendored into the repo with all schemaLocation references rewritten to local files, so the schema compiles with zero network access — bulk publish and ricecooker upload validate many items per run. schema/refresh_schema.py re-downloads and re-applies those rewrites repeatably for future QTI point releases.

The compiled lxml.etree.XMLSchema is built once behind an lru_cache and reused across calls. XMLSchema.validate() writes to a shared error_log, so a lock serializes validate-and-read across concurrent calls to keep errors correctly attributed.

XML parsing uses a hardened lxml.etree.XMLParser (resolve_entities=False, no_network=True, load_dtd=False, huge_tree=False), built fresh per call, since this validator sits in the path of untrusted input (ricecooker uploads, direct sync-API writes), not just editor-authored XML.

References

Closes #6005

Reviewer guidance

Run pytest contentcuration/contentcuration/tests/utils/qti/ -v. New coverage:

• Valid items across multiple interaction types
• Invalid enum value and missing required attribute produce errors identifying the problem
• Malformed XML fails without raising
• External entities are not resolved (XXE)
• Two interaction types the pydantic layer never generates (qti-match-interaction, qti-order-interaction)
• Schema is compiled once and reused across validate_qti_item() calls

Other areas worth a close look:

• _secure_parser() in contentcuration/contentcuration/utils/assessment/qti/validation.py — the entity/network/DTD hardening, given untrusted input reaches this from ricecooker/sync-API.
• .pre-commit-config.yaml — added an exclude to check-added-large-files for the vendored schema/xsd/*.xsd files. The official imsqti_itemv3p0p1_v1p0.xsd is ~18MB (verified genuine spec content — per-attribute schematron assertions generated by IMS's own tooling, not accidental duplication).
• New dependency: lxml==6.1.1 in requirements.in/requirements.txt.

AI usage

Implemented with Claude Code from a maintainer-approved plan (.plans/index.md, issue #6005), using TDD per task. One vendored dependency, XInclude.xsd, turned out to be UTF-16 encoded. The repo's whitespace-fixing pre-commit hooks corrupted it on first commit attempt. I re-encoded it to UTF-8 and re-verified the schema still compiles fully offline. Ran the full QTI test suite and confirmed flake8/black compliance before opening this PR.

---

@rtibblesbot's comments are generated by an LLM, and should be evaluated accordingly

How was this generated?

• Ran pre-flight CI checks (lint, format, tests) and verified all pass
• Rebased onto the target branch and resolved any conflicts
• Reorganized commit history into clean, logical commits
• Audited the diff to ensure only issue-relevant files are changed
• Built PR body from the repository's PR template with evidence blocks

Status: 🟡 Waiting for feedback · updated 2026-07-03 02:21 UTC

[rtibbles]

This looks broadly correct to me - one question about the errors, one blocking comment about ongoing maintainability.

[rtibbles]

This documentation points to an uncommitted file to tell us how to do this. It would be far better to have a small Python script that can be rerun to do this repeatably and reliably.

[rtibblesbot]

Good point — added refresh_schema.py alongside the vendored schema. It downloads the QTI-item XSD and its W3C/MathML/SSML dependencies and reapplies the schemaLocation rewrites, so refreshing to a newer QTI point release is just running the script and re-running the test suite. Updated the README to point at it instead of the plan file. (5b56e52)

[rtibbles]

How does the schema.error_log get reset between validation runs? Is that built into the validate call?

[rtibblesbot]

Confirmed empirically: error_log is reset by validate() itself — each call clears the log before writing that call's results, whether it passes or fails (verified: validate(bad) → 1 error, validate(good) → empty log, validate(bad) again → the same 1 error, no accumulation). So no manual reset is needed between calls.

The remaining risk was concurrency, not staleness: since the schema instance is cached and shared, two threads calling validate() on it at the same time could race — one thread's validate() resets the log before the other has read its own results. That's exactly what the lock added in 257aa28 (already on this branch, in response to a self-review pass before your comment) serializes: validate() and the error_log read now happen atomically per call, so results stay correctly attributed under concurrent use. No further change needed here.

[rtibbles]

A bit more needed to make this fully maintainable.

[rtibbles]

Rerunning this script, while it produces nearly the same output as committed here, there are formatting differences that suggest that additional processing happened to them that isn't happening in this script.

After running them through pre-commit most of the differences disappeared, but XInclude.xsd still showed a binary difference, not sure what the diff was.

I think it would be helpful for this script to run pre-commit on the schema files after the download, and also nail down why XInclude.xsd is showing a diff?

[rtibblesbot]

Found it — XInclude.xsd is served by purl.imsglobal.org as UTF-16 with no XML declaration, unlike every other source file here (which are all plain UTF-8). That's why it kept showing a binary diff: the script wrote the raw UTF-16 bytes straight to disk while the committed copy had been normalized to UTF-8 by hand at some point.

Fixed in 804a8e4: refresh_schema.py now transcodes any UTF-16 download to UTF-8 (decode/re-encode, not a round-trip through lxml, so the original formatting isn't reflowed), and runs pre-commit run --files on the vendored .xsd tree after download+rewrite — that only triggers trailing-whitespace/end-of-file-fixer for these files, matching what had evidently been applied to the committed copies.

Verified: re-running the script now produces a byte-for-byte empty git diff against every committed vendored file, including XInclude.xsd.

[rtibbles]

Excellent, the refresh script now runs and lands clean! Great work.