Repository navigation
v0.106.1: Trustworthy class-ordered-data warning
Classification label-diversity diagnostic
Mixlab warns when classification records look class-ordered within shards, a
condition that balanced global class_counts cannot reveal. The check only
fired when every sampled batch was single-label, so one batch straddling a
shard boundary cleared the flag and silenced the warning for the rest of the
run — and real class-ordered corpora cross a boundary every few batches, so the
diagnostic was quietest exactly when it mattered.
It now scores the mean distinct-label count across the sample and reports
mean/min/max, so occasional two- or three-label boundary batches no longer hide
near-degenerate ordering.
The threshold measures diversity above the one-label floor every batch
necessarily has, 1 + 0.25*(achievable - 1). A plain "25% of achievable" rule
demands both labels in every binary batch, which correctly shuffled data misses
by chance: measured over 400 seeded runs, that fired on 86% of 5%-minority
binary corpora and 74% of balanced binary at four rows per batch.
Low batch diversity has two causes the count alone cannot separate —
class-ordered records, or one class dominating the corpus — so the warning now
names both rather than asserting the first.
Other Changes
- chore:
make lintnow pins the same Go toolchain as the pre-commit hook,
so the two stop disagreeing; the go-vet-only fallback announces itself
instead of reporting "all checks passed" when golangci-lint is absent. - docs: corrections and additions across the module
CLAUDE.mdfiles.
Install
brew tap mrothroc/mixlab https://github.com/mrothroc/mixlab
brew install mixlabOr build from source:
CGO_ENABLED=1 go build -tags mlx -o mixlab ./cmd/mixlab/