ThinPFN 1.1.0: probability study, latency, one-line install

Same model, same weights, same sealed-test numbers as 1.0.0. This release adds evidence about the probabilities, makes the model installable in one line, and puts the tests under continuous integration.

pip install "git+https://github.com/maximer-v/thinpfn@v1.1.0"

Are the scores usable as probabilities?

New study on the validation split of the ten books, same contexts and baselines as the benchmark, no recalibration for any model (the sealed test split is not read again).

n	model	KS	Brier skill	calibration error (ECE)	books worse than the base rate
200	ThinPFN	0.315	+0.088	0.025	0 of 10
200	TabPFN v2	0.321	+0.073	0.032	2 of 10
500	ThinPFN	0.339	+0.104	0.023	0 of 10
500	TabPFN v2	0.351	+0.113	0.029	0 of 10
1,000	ThinPFN	0.365	+0.119	0.023	0 of 10
1,000	TabPFN v2	0.373	+0.128	0.027	0 of 10

• ThinPFN has the lowest calibration error of the four models at every context size and is never worse than the base-rate forecast on any book, even with 200 rows.
• TabPFN v2 separates slightly better from n = 500 (KS, Brier skill); most of that gap is one book, US home equity, the same book that carries the AUC losses.
• The predicted level follows the default rate of the context rows; a context sampled at a different rate than the scored population needs a correction outside the model.

XGBoost and logistic regression, per-book numbers and the reliability curve are in the README and in benchmark/results/calibration.*.

Latency

Measured time of one scoring call (benchmark/latency.py), default eight column orders:

context rows	query rows	CPU, 8 threads	desktop GPU
200	1	0.24 s	0.03 s
1,000	2,000	5.7 s	0.28 s
4,000	4,000	29 s	1.2 s

A single pass (n_perm=0) is about eight times faster.

Also in this release

• thinpfn.metrics: ks, ece, reliability_bins, calibration
• Tests on Python 3.10 and 3.12 in GitHub Actions, plus a check that a non-editable install loads the packaged weights
• Tests for use inside CalibratedClassifierCV and cross_val_score
• Paper: new probability-quality subsection, latency paragraph, updated limitations
• CHANGELOG.md

Released under CC BY-NC 4.0 (non-commercial).

ThinPFN 1.0.0

First public release.

• The synthetic prior: structural causal generators and a credit ledger simulator
• The 25.6M network, its trainer, and the released weights (thinpfn_v1, six GPU-hours on one desktop GPU)
• ThinPFNClassifier, a scikit-learn style front end
• The ten-book credit benchmark with split manifests and the sealed-test results against logistic regression, tuned XGBoost and TabPFN v2
• Walkthrough notebook and paper

Released under CC BY-NC 4.0 (non-commercial).