DM-55799: Freeze masked image schema and check for schema inheritance - #90
Merged
Conversation
cell_coadd 1.0.0 inherits from masked_image and was published, and shipped in data, while masked_image was still at 1.0.0.dev0. A development schema may be changed in place freely, so an edit to masked_image would have rewritten the already-immutable cell_coadd document: the released schema could change underneath every reference to it. Freezing masked_image closes that. Its own dependencies -- image, mask and sky_projection -- are all already at 1.0.0, so it can be finalized without finalizing anything else first. cell_coadd's frozen document is byte-identical afterwards, because pydantic flattens a base class's fields into its subclass and the base's version never appears in the result. That is also why the problem was not visible in the published artifact: no $ref and no x-lsst-schema-url records the inheritance, so nothing that reads the document could see the dependency. The fixture carries over to the finalized version with its stamp normalized. It exercises 5 of 38 paths and never holds a sky_projection, and a frozen fixture cannot be widened without a SCHEMA_VERSION bump. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #90 +/- ##
==========================================
+ Coverage 67.85% 68.04% +0.18%
==========================================
Files 143 145 +2
Lines 20740 20876 +136
==========================================
+ Hits 14074 14205 +131
- Misses 6666 6671 +5 ☔ View full report in Codecov by Harness. |
TallJimbo
approved these changes
Aug 12, 2026
A frozen document inlines the content of everything the schema depends on, including each embedded sub-schema's version in its x-lsst-schema-url. So finalizing a schema whose dependencies are still in development publishes an immutable document that those dependencies remain free to change underneath, and the first time one of them is finalized the containing document changes and can no longer be written. write_frozen_schemas now checks this before writing, naming every offending dependency, and a test asserts that no finalized schema in the package has one. The check runs when a frozen file is first created rather than on every run, so an already-committed document is never re-validated. Inheritance is the route that made this easy to miss. pydantic flattens a base class's fields into its subclass, so a base contributes its fields to the frozen document without appearing in it as a $ref or an x-lsst-schema-url; cell_coadd inherits from masked_image and visit_image does too. No check that read the document could see the dependency at all, which is why schema_dependencies walks base classes as well as fields, and does so transitively -- a dependency reached only through a container, a union, or another schema counts the same. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two tests in test_cli.py named camera_frame_set because it happened to be a real schema still in development: one wrote an invalid fixture for it to prove 'fixtures refresh' turns a ValidationError into a clean error rather than a traceback, the other dirtied its fixture to prove a dirty fixture is detected on every run without being repaired. Both need refresh to actually process the fixture, and refresh acts only on a schema still in development, so both quietly depended on camera_frame_set never being finalized. That dependency cannot hold. Every schema is finalized during a release, so there are periods with no development schema in the package at all, and selecting one dynamically would make these tests skip exactly then. They now use a double that is never finalized, reached by passing its module as the CLI's --package, which selects that schema and no other. Freezing camera_frame_set with the old tests failed both after the ordinary fixture refresh cascade; with these it leaves the suite green. The validation test also stopped testing its own subject once the schema was finalized: refresh skipped the invalid fixture entirely and exited zero, because a .dev fixture no longer matches a finalized live version. The double's only field is required, so an empty tree fails to read and the error path under test is genuinely reached. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We froze cell_coadd 1.0.0 without realizing it depended on masked_image v1.0.0.dev0 because inheritance in python masked the version number.
This PR freezes masked_image but also checks at freeze time that no dev schemas are being included.
Checklist
doc/changes