Skip to content

v0.10.0

Choose a tag to compare

@xtmtd xtmtd released this 28 Sep 15:19
· 8 commits to main since this release

Optional register tokens for eligible single-CLS ViTs, with register-aware checkpoint layouts and patch consumers.

Added

  • --register-tokens none|0|4 (default none) on pretrain: 4 adds four trainable registers and migrates the one-CLS pretrained backbone (the register token keeps its native initialization while every CLS/patch/block weight and position is preserved); none keeps the backbone's own architecture, i.e. no registers on a plain ViT or a native-register model's own count.
  • A shared read-only checkpoint loader with a register-layout decision table, rejection of non-canonical reg_token keys, and RNG-isolated pretrained=False probes; extract, cam and export all load through it.
  • Registered pretraining resume and registered fine-tune sources, including student/teacher synchronization of the newly initialized registers and explicit layout validation before any optimizer state is restored.
  • An attention-pool patch policy (num_prefix_tokens, register_tokens, prefix_excluded, attention_pooling_type) recorded in new pool checkpoints and saved to a policy-tagged sibling whose contents are validated even when the filename matches.
  • CPU FP32 PyTorch versus CPU ONNX Runtime numeric validation of exports (atol=1e-4, rtol=1e-3), recorded as validation_status, max_abs_diff, atol and rtol.
  • A design document and implementation plan under docs/superpowers/.

Changed

  • Every patch consumer (patch-topk, attention-query training, attention-pool extraction and the ViT CAM reshape) now drops all prefix tokens via backbone.num_prefix_tokens, so a CLS-free backbone keeps its first patch.
  • New checkpoints record config.register_tokens (the actual count) and args.register_tokens (the invocation value); fine-tuning re-saves the inherited layout and includes it in the v0.9.0 pseudo-label experiment identity.
  • vit_reshape_transform gained an optional prefix_tokens argument; the default is unchanged for direct callers.
  • README and README.cn.md document the option, the patch policy and the export validation.

Fixed

  • A failed or numerically mismatched export no longer publishes a model and can no longer leave a stale validated: true report behind; only this attempt's temporary files are cleaned up.
  • An incompatible attention-pool checkpoint at the policy-tagged path now fails naming its path instead of being masked by another fallback artifact.
  • config.register_tokens is validated as a non-negative integer at every checkpoint entry point, so a malformed value is rejected rather than treated as a legacy zero.

Upgrade notes

  • none is the default and preserves the previous construction, so existing zero-register workflows are unchanged; the 272 existing checkpoints were verified end to end (extract, CAM, ONNX export) with the corrected patch boundaries.
  • ONNX export now fails, instead of publishing a shape-only validated model, when ONNX Runtime is installed but the graph fails to run or disagrees numerically beyond the approved tolerances.
  • Patch-consumer outputs change for CLS-free/no-register checkpoints (the first patch is now included) and for CLS-bearing native-register checkpoints (registers are no longer treated as patches): regenerate affected patch-topk embeddings and attention pools.
  • Fine-tuning has no new switch; it inherits the source architecture and its register layout.