ACTok ResNet-50 and ConvNeXt-B checkpoints (PCA validated)

PCA-validated CNN checkpoints

These checkpoints were checked against historical project figures before release. All compared outputs use backbone features, not aggregation-head outputs. Strict loading and finite CPU inference were checked; benchmark scores were not reproduced.

File	Contents / teachers	Size	Historical PCA agreement
resnet_distill.pth	ResNet-50 backbone only	90.0 MiB	0.99997 mean raw RGB correlation vs. historical oranges PCA
convnext_b_norm_distill.pth	ConvNeXt-B, CNN teachers; backbone + aggregation modules	370.8 MiB	0.99780 vs. Figure 6(b)'s embedded JPEG
convnext_b_norm_distill_allvit.pth	ConvNeXt-B, ViT teachers; backbone + aggregation modules	364.6 MiB	0.9999999 vs. visualizations/fern.png, also Figure 6(c)

PCA comparisons use the historical OpenAI normalization, resize/crop settings, and no RGB sign/channel/color adjustments. The images are near-matches rather than pixel-exact: JPEG and plotting differences remain. The old ResNet run uses whitened linear PCA at 3072×3072; the ConvNeXt runs use cosine-kernel PCA at 4096×4096.

See the README and tools/validate_checkpoint_pca.py for strict backbone loading and reproduction commands. The ConvNeXt files contain extra aggregation modules, but the validated representation uses the student.* state. The published configuration and validation JSON assets include details. Verify downloaded files with CNN_SHA256SUMS.

Excluded: the _l2 ConvNeXt experiment did not reproduce the selected reference; resnet_distill_student.pth has no prediction heads and differs from the validated backbone. A distilled ViT-L student checkpoint was not found locally, so no ViT-L model is included.

ACTok ViT-B distilled checkpoint

ACTok ViT-B distilled checkpoint

This release publishes pretrain/vit_b_distill.pth from the project checkout: a ViT-B/16 student with learnable token, cross-model and inner-model interaction, projections, and prediction heads. No frozen teacher weights or optimizer state are included.

• File: vit_b_distill.pth (717,749,214 bytes; 684.5 MiB)
• Implementation: ModelAggregator_ViT.BackboneStuViT (original aggregation variant)
• Configuration: student="vit_b", num_blocks=1, in_dim=768, dim=768, token_dim=512, out_dims=[512,768,768], teacher_type=["clip","dinov3","mae"], img_size=224
• Features: DINO-head output [B,197,768] for 224×224 inputs; discard the first token for spatial patch features.
• Integrity: SHA-256 175956fc11674777027ac4b7efcea72e1268e94788a5b7d38dcad7fba0dd72c0

Use the updated loader and example in the repository README. The checkpoint differs from the ModelAggregator_ViT_combine.py variant selected by the current training entry point. It does not contain its original training configuration or benchmark scores, so this release does not claim an exact match to a specific row in the paper's result tables.

Validation: all 197 state-dict tensors loaded strictly, loaded values matched the source checkpoint, and a CPU inference pass returned finite [1,197,768] features. This is a loading and inference check, not a benchmark reproduction.

The additional assets contain the model configuration and SHA-256 checksum. Other teacher, comparison, and ablation weights remain local.