v0.7.1

Breaking changes

Embedding metrics are not numerically comparable with pre-v0.7.1 values. Four evaluator changes cause this:

• Cross-validated kNN and linear-probe scores now share one explicit shuffled StratifiedKFold(shuffle=True, random_state=42) whose fold count is min(5, smallest class count). A two-class dataset previously used only 2 folds; it now uses 5, so each fold trains on more data and the reported kNN/linear-probe values are systematically higher for the same model.
• Trainer and extract now apply the same sorted subsample order, so fold assignment, KMeans initialization, and tie-breaking agree between the two paths.
• mAP no longer counts the query itself as a relevant result.
• Metrics that cannot be computed are written as empty CSV fields instead of a fabricated 0.0 or a single-cluster KMeans fallback, and cross-validated metrics can no longer be written as NaN.

Existing field names are unchanged, and kNN_Acc_k1, kNN_Acc_k5, and kNN_Acc_k20 are retained. A requested k the data cannot support (for example kNN_Acc_k20 when the smallest training fold has fewer than 20 samples, or Recall@k with k > n_samples - 1) is now empty rather than silently renamed.

Highlights

• New Linear_Probing_Balanced_Acc field: class-balanced linear-probe accuracy taken from the same cross-validation pass as Linear_Probing_Acc, so it costs no extra fits. It appears in metrics.pretrain.csv, metrics.finetune.csv, extract's metrics.csv, and the Retrieval/Probe training plot.
• Correct retrieval and clustering edge cases: mAP ranks only non-query samples and drops queries with no non-self relevant item; clustering metrics are unavailable when there are fewer distinct normalized embeddings than clusters instead of fabricating a single cluster; Silhouette_Score is unavailable when silhouette is undefined.
• CLI no longer couples to external Click. Parameter-source checks compare by enum name, and cam/extract validate their enumerated options with native typer.BadParameter before any output directory is created, so commands keep working when typer>=0.26 vendors Click. --arch, --fig-format, and --device now list their accepted values in --help.
• extract rejects duplicate image rows in --label-csv before metric computation, instead of silently desynchronizing labels from embeddings.
• Metric contract documented in docs/superpowers/specs/2026-09-21-otuformer-v071-cli-metrics-corrections.md, with evaluator-vintage notes added to the historical comparison reports.

Compatibility

• Resuming a v0.7.0 logs/metrics.pretrain.csv or logs/metrics.finetune.csv migrates the recognized header atomically: an empty Linear_Probing_Balanced_Acc column is added and every existing row is preserved. A header that already matches the current schema is appended without rewriting; duplicate-column, malformed, and unrecognized headers fail before any append.
• Old rows leave the new field empty, and plots treat empty fields as gaps rather than zeros.
• extract writes metrics.csv only when --label-csv provides at least two classes; unlabelled extraction, and labelled extraction with a single class, still write embeddings.csv (and UMAP when eligible).
• No new dependency, and no change to training, checkpoint contents, or the embedding extraction path.

Notes

• Values recorded before v0.7.1 (for example the epoch-50 tables in docs/superpowers/specs/2026-03-30-*) keep their recorded numbers but are not comparable with v0.7.1 outputs.
• Training itself is unchanged: a controlled old-code vs new-code run with the same seed produced bit-identical weights, optimizer state, and RNG state. On the same checkpoint, only the four evaluator changes above move the numbers.
• Comparing two separate training invocations is not apples-to-apples on this pipeline: with --num-workers > 0 the augmentation RNG is per worker and batch-to-worker assignment is dynamic, so runs with identical arguments can diverge. Use --num-workers 0 when run-to-run reproducibility matters.