v0.7.0

Breaking changes

--mask-ratio now defaults to auto, which resolves to 0.30 for a new run; v0.6.x used a float default of 0.50. Explicitly passing an option that the selected patch mode does not use is now rejected instead of silently ignored.

Migration:

• A new run that relied on the old 0.50 default gets 0.30. Pass --mask-ratio 0.5 explicitly to keep the previous behavior.
• Resuming a legacy checkpoint keeps its recorded ratio (0.50 when absent) and stays on consistency. Legacy checkpoints cannot resume as masked-feature or ibot.
• --patch-loss consistency with --masking-strategy or --ibot-prototypes now fails instead of ignoring the option.

Highlights

• Four patch-loss modes, mutually exclusive: none, consistency (default), masked-feature, and ibot (experimental).
  • consistency selects fully visible same-position patches for normalized cosine regression. It is masked-position patch consistency, not masked image modeling: the student still sees every pixel.
  • masked-feature (continuous masked feature prediction) replaces the selected student patch embeddings with a learnable mask token before positional encoding, so the student never sees the selected content while the position is preserved, and regresses them onto the L2-normalized mean of the teacher's final four block outputs.
  • ibot predicts the teacher's centered prototype distribution at masked positions with moving-average centering.
• Three real-masking strategies via --masking-strategy random|blockwise|hybrid, with exact mask counts, per-grid rectangle feasibility checks, and fail-fast on infeasible grids such as 4 x 4.
• --ibot-prototypes accepts any integer >= 2 (default 512) instead of a fixed 256/512/1024 list, so larger datasets can use larger dictionaries.
• --lambda-mask documents that ibot usually needs a lower weight (for example 0.25-0.5).

Compatibility

• Extraction, fine-tuning, CAM, and export are unchanged: the default path is still the raw CLS token, and they never load or invoke the training-only patch state.
• Pretraining and pretraining resume support standard timm ViT backbones only. EVA, Swin, and CNN backbones fail with an actionable error before the training loop.
• v0.7.0 resume is strict: patch mode, resolved ratio, masking strategy, prototype count, and the fixed target/center settings must match the checkpoint, and masked-feature/ibot checkpoints must carry their patch_objective state. Python, NumPy, PyTorch CPU, and CUDA RNG state round-trip.
• Existing instant-metrics CSVs with the recognized legacy pretrain header are migrated atomically to the new schema; unknown, duplicate-column, or malformed schemas fail without touching the file.

Notes

• masked-feature and ibot add one masked student forward per global view. On a CPU benchmark with vit_tiny_patch16_224, 2x224 global + 6x96 local crops, batch 16, they cost roughly 1.25x-1.42x the consistency step time, consistent with the design's ~1.39x forward-work proxy. This is a reported cost comparison, not a performance gate.
• These objectives are self-supervised contextual feature prediction. They do not by themselves establish anatomical part semantics or dense morphology supervision, and v0.7.0 makes no dense-representation claim.
• Loss values are not comparable across patch modes. The iBOT cross-entropy is not divided by log(K).