v8.4.0

Added

• TabPFNRegressor now accepts eval_metric and tuning_config arguments: passing tuning_config={"calibrate_temperature": True} makes fit() calibrate the temperature of the aggregated ensemble distribution on a holdout, sharpening or widening the predicted distribution as the data demands and improving every predict() output type, including the predicted quantiles. Set eval_metric to "nll" (the default, negative log-likelihood), "crps" (continuous ranked probability score, the same implementation used by the finetuning loss) to choose which quantity the calibration optimises; they weight the predicted distribution differently and pick noticeably different temperatures, so pick the metric you will be judged by. (#1172)
• In the previous setup, all GPUs in finetuning with DDP held all activations of all estimators in memory. This PR divides estimator activations across the available GPUs. (#1182)

Changed

• The temperature grid searched when calibrating TabPFNClassifier's softmax temperature now contains 1.0 exactly, so calibration can leave a distribution untouched. Previously the grid straddled 1.0 without including it, meaning a calibrated model always applied some correction even when none was warranted. Calibrated temperatures may therefore differ slightly from previous releases. (#1172)
• fit() cleans large tables with far less memory and time: the redundant float64 copies are gone, cutting both transient memory and wall time by about two thirds on a 5.3 GB all-numeric table. (#1173)
• fit() uses less memory on tables with categorical columns: the encoded array is now assembled in place instead of being stacked and then reordered, cutting transient memory by about a quarter on a half-string table. (#1174)
• fit() no longer slows to a crawl on wide tables with categorical columns under pandas < 3: the dtype casts no longer rebuild the frame one column at a time, which took minutes on a 333,333 x 400 table and now takes seconds. (#1180)
• Reduce peak GPU memory during KV-cache construction by quantizing layers as they are built and temporarily staging completed estimator caches on CPU in memory-saving mode. (#1183)

Fixed

• TabPFNRegressor.predict_batched now raises NotImplementedError when the estimator was constructed with a tuning_config, instead of silently returning uncalibrated predictions. The ensemble temperature is calibrated on each dataset's own holdout, so a fused batch has no single temperature to apply; score such datasets individually with predict. This matches the existing guard in TabPFNClassifier.predict_proba_batched. (#1172)

v8.3.0

Added

• Fine-tuning now supports validation_frequency to run validation and early-stopping checks every N epochs. (#811)

• Add ManyClassDecoder.attention_weights, the canonical per-training-row attention distribution of the multiclass decoder head, so interpretability tooling can read out which training rows drive a prediction without reimplementing the head's internal forward pass. The method exists only on multiclass models that use this decoder. (#1142)

• Add fp8 kv cache dtype. (#1157)

• fit() now warns when a column of X looks like free text. (#1159)

• Add an opt-in built-model cache to load_model, enabled via the TABPFN_MODEL_CACHE_SIZE environment variable (default off). When set, repeated loads of the same checkpoint reuse the constructed model instead of rebuilding the architecture and re-running load_state_dict. Only the non-mutating (cache_trainset_representation=False) build is cached. (#1162)

• Add TabPFNRegressor.predict_batched, the regression counterpart to TabPFNClassifier.predict_proba_batched. It preprocesses each (X_train, y_train, X_test) triple exactly as fit + predict does, stacks the datasets on the model's batch dimension and scores them with a single fused forward per estimator, then decodes each dataset with its own target standardisation and per-estimator border transforms. Returns one entry per dataset in input order, each with the same structure predict would return for that dataset. Datasets must share array shapes; constant-target datasets are answered analytically.

  Both batched methods now raise NotImplementedError for inference_precision=torch.float64 instead of silently computing the fused forward at float32 and returning float32-precision results. (#1164)

• Add an attention-backend registry; the in-tree FA3/torch-MPS/MLX paths now route through it. Behavior unchanged unless a backend is registered. (#1165)

• Add a test that enable_torch_compile traces tabpfn without graph breaks. (#1166)

Changed

• Speed up cached prediction on Hopper GPUs with FlashAttention-3 installed, by splitting attention over the key/value sequence when few test rows attend over a large training cache. (#1168)
• n_estimators now defaults to "auto" on TabPFNClassifier and TabPFNRegressor. Feature-coverage auto-scaling (raising n_estimators on wide datasets so every feature is seen by some estimator) applies only to "auto" — an explicitly passed n_estimators is always used exactly as given, and warns at fit time if it is too small for every feature to be covered. (#1171)

Fixed

• Fix activation checkpointing during v2, v2.5, and v2.6 fine-tuning after the state-container memory optimization. (#1138)

• • Fixed fit() crashing during temperature calibration or threshold tuning
    (tuning_config) when a rare class is absent from the tuning holdout, by
    passing the full label set to log_loss explicitly.
  • Fixed fit() crashing when random_state is a np.random.Generator and
    tuning is enabled, by converting the generator to a static seed before it
    reaches StratifiedKFold.

  (#1140)

• fixed the randomness in the truncated SVD to make runs more reproducible. (#1167)

• Fixed the many class decoder to only work with the relevant class count. (#1175)

• Skip the Claude code review workflow on bot-authored release PRs, which previously failed the claude-review check on every release. (#1177)

Deprecated

• auto_scale_n_estimators is deprecated and will be removed in v9. It only ever applied to n_estimators="auto", where auto_scale_n_estimators=False is equivalent to passing n_estimators=8; pass an explicit n_estimators instead to opt out of feature-coverage scaling. Passing False now emits a FutureWarning at fit time. (#1171)

v8.2.0

Added

• Add examples/input_gradients.py showing how to extract gradients of TabPFN predictions with respect to the input data via differentiable_input=True on the breast-cancer dataset. (#1128)

Changed

• Improved fine-tuning: retuned the example scripts (Higgs test AUC 0.8247 → 0.8322, California housing test MSE 0.1350 → 0.1328), validation_split_ratio=None/0 now disables validation entirely, best-checkpoint saving now only applies with early_stopping=True, and small remainder data chunks no longer crash the context/query split. (#1101)
• Run the consistency tests with float64 inference precision and regenerate the reference predictions, reducing floating-point divergence across hardware/BLAS backends. (#1111)
• Expose the kv_cache_dtype as explicit argument. (#1120)
• inference_precision="auto" now uses bfloat16 autocast on CPUs with native bf16 support (Intel AMX / AVX512-BF16, AMD Zen 4+), giving ~2x faster CPU inference at unchanged accuracy. CPUs without fast bf16 keep running in float32. (#1122)
• Raised the default CPU sample limit from 1000 to 5000 for the v3 model (other versions keep the 1000 limit). With bf16 autocast on modern CPUs, v3 inference on datasets up to 5000 rows now runs in a reasonable time (~30s at 5000 rows). The "may be slow" warning threshold scales with the limit (1000 for v3, 200 otherwise). The override (ignore_pretraining_limits=True / TABPFN_ALLOW_CPU_LARGE_DATASET=1) is unchanged. (#1123)
• Enabled autocast also for mps. (#1124)
• Changed the minimal required version of torch to use MPS to 2.6 (#1129)
• Speed up preprocessing for large number of features (#1136)
• Speedup preprocessing for large numeric columns. (#1137)

Fixed

• Fixed a RuntimeWarning: overflow encountered in cast in SafePowerTransformer's
  Yeo-Johnson inverse transform, caused by the clip bound being computed in the
  output dtype instead of float64. (#1105)
• Fix timing on GPU to reflect full GPU work. (#1113)
• Fix _repair_borders widening the top bar-distribution border downwards for negative targets, which left the borders non-ascending when a quantile target transform collapsed the top gap. (#1127)

v8.1.0

Added

• KV Cache support for 2.5 and 2.6 single file models. (#1039)
• Add TabPFNClassifier.predict_proba_batched(X_list, y_list, X_test_list) to score several independent datasets in a single fused forward per estimator (stacking them on the model's batch dimension), equivalent to fitting and predicting each dataset separately but much faster when launching many small predicts. (#1045)
• Add an opt-in PASSTHROUGH_INF inference-config option (default False), set via the inference_config argument of TabPFNClassifier/TabPFNRegressor (or, for the finetuned estimators, via the inference_config entry of their extra_*_kwargs). When enabled, ±inf values are no longer rejected during fit()/predict(); they are carried through preprocessing (replaced with NaN for the steps that cannot handle them and restored afterwards) so they reach the model, which handles them natively. (#1055)
• README architecture and attention diagrams for TabPFN-3 (Prior Labs colour scheme). (#1060)
• Add calculate_cache_size for TabPFN v3 to compute the resident cache memory (ICL KV cache, decoder activations, distribution-embedder inducing states, and scaler stats) for a given train-set size, column count, ensemble size, and dtype — without running inference. (#1087)
• Add a public tabpfn.finetuning.main_process_first() context manager for multi-GPU (torchrun) scripts: the main process runs the with-block first while the other ranks wait at a barrier, then the other ranks run it — useful for one-time work such as dataset downloads that should warm a shared cache. The process group it initializes is reused by the subsequent fit(). (#1094)
• Chunk large test sets during cached (fit_mode="fit_with_cache") inference to bound peak GPU memory, controlled by the new TABPFN_MAX_BATCHED_TEST_ROWS setting (default 32768; set to 0 to disable). Chunking is mathematically equivalent. (#1096)

Changed

• TabPFN-2 and TabPFN2.5 now use the single file implementation, deprecate 'base'. (#1052)
• Fine-tuning now targets the package default model version (settings.tabpfn.model_version) instead of a hardcoded older one, and FinetunedTabPFNClassifier/FinetunedTabPFNRegressor accept an optional model_version to override it — so a fine-tuned model is no longer silently compared against a different-generation base. (#1064)
• Reduce memory usage for v2.x architectures. Enable flash attention on MPS for v2_6. (#1070)

Fixed

• Add TabPFNRegressor.fit_with_differentiable_input(X, y) so gradients can flow from a downstream loss back through the regressor into upstream torch modules feeding X (and y, when it carries grads). Mirrors the existing classifier-side path — previously TabPFNRegressor.fit raised ValueError("Differentiable input is not supported for regressors yet.") and there was no differentiable counterpart. (#923)
• Support save/load for estimators fitted with fit_mode="fit_with_cache". Previously save_fit_state / load_from_fit_state raised NotImplementedError for KV-cache inference engines. (#977)
• Fix save_fitted_tabpfn_model/save_fit_state moving the live estimator's bar distribution modules to CPU, which broke subsequent predict calls (e.g. output_type="median"/"quantiles") on CUDA/MPS devices. (#1030)
• Fixed AdaptiveQuantileTransformer losing output_distribution and random_state when cloned by sklearn (e.g. inside ColumnTransformer.fit), which made the quantile_norm* presets silently produce uniform output. All transformers now run sklearn's standard estimator checks. (#1031)
• Fixed fit() hanging forever when stratified row subsampling allocates a class more slots than it has rows (e.g. an ultra-rare class with SUBSAMPLE_SAMPLES set); such classes are now minimally oversampled instead. (#1034)
• Fixed norm_and_kdi returning a feature schema that undercounts the output columns: the FeatureUnion emits two columns per input column, so the schema and num_added_features under-reported, letting the ensemble's feature-budget planning silently exceed max_features_per_estimator. (#1035)
• Fixed an inverted enable_gqa condition in the torch-MPS attention fast path that would crash every forward of models with asymmetric query/KV head counts (including the default TabPFN v3 checkpoint) on Apple Silicon once torch satisfies the MPS flash-attention version gate (>= 2.13). (#1037)
• Fix fitted model saving for paths whose parent directories contain .tabpfn_fit. (#1048)
• Fix the README's save/load example to call save_tabpfn_model(reg, ...) with the estimator instead of reg.model_, which would have raised at runtime. (#1053)
• Remove all-NaN columns as constant features so they no longer leak NaNs into downstream preprocessing. (#1061)
• Fine-tuning with early stopping no longer returns a model worse than the base when no epoch improves over the default; the original weights are now restored. (#1064)
• Fix the README save/load FAQ to render correctly on GitHub (replace Sphinx :func: roles with code spans) and document the TABPFN_MPS_MEMORY_FRACTION environment variable. (#1065)
• Fix incorrect model output on MacOS 26 on M1 when using the MPS device. (#1077)
• Fix predict_proba_batched raising RuntimeError: mat1 and mat2 must have the same dtype, but got Half and Float under inference_precision=torch.float16 on GPU. The batched inference engine now casts the model to the forced dtype, not just the inputs. (#1083)
• Fix the fine-tuning examples crashing or redundantly downloading their dataset once per rank when launched with torchrun --nproc-per-node=N on a cold sklearn cache; the dataset fetch is now wrapped in main_process_first() so only the main process downloads. (#1094)
• Fix cross-device save/load tests failing on GPU by only requiring functional equivalence, not bit-identical predictions, across devices. (#1097)
• Fix Windows CI crash (illegal instruction) by skipping the bfloat16 autocast KV-cache test on Windows without CUDA. (#1103)

Deprecated

• Remove base architecture - per-model file implementations are now the single source of truth. (#1056)
• Remove the now unused InferenceEngineCacheKV. All models now use InferenceEngineExplicitKVCache. (#1057)

v8.0.8

Breaking Changes

• Dropped support for Python 3.9; the minimum required version is now Python 3.10. The eval-type-backport dependency (only needed on 3.9) has been removed. (#1038)

Added

• Add a single file implementation of TabPFNv2. Not activated by default yet. (#995)
• Add a keep_cache_on_device option to TabPFNClassifier/TabPFNRegressor (defaults to True). When fit_mode="fit_with_cache", setting it to False offloads each per-estimator KV cache to CPU as it is built and moves it back to the device on demand, lowering resident device memory at the cost of per-call transfers. (#1009)
• Added official support for Python 3.14 (already exercised by the CI test matrix). (#1038)

Changed

• Improve peak memory of single file model implementations. (#1019)
• Removed the per_feature option from PreprocessorConfig.name. (#1036)

Fixed

• Fixed regressor ensemble members sharing a single mutable target_transform instance. With in-process preprocessing (n_preprocessing_jobs=1), each member's in-place fit clobbered the fitted state of the others, silently corrupting predictions whenever members were fitted on different targets (e.g. with row subsampling active). Each ensemble config now owns a deep copy of the transform. (#1029)
• Fixed two GPU-preprocessing divergences from the CPU reference: TorchSoftClipOutliers silently skipped outlier clipping when predicting a single sample in KV-cache mode (predictions depended on test batch size), and TorchAddSVDFeaturesStep added an SVD column for single-feature datasets where the CPU pipeline adds none (predictions differed between ENABLE_GPU_PREPROCESSING on and off). (#1033)
• Fixed a fit-time crash when a DataFrame mixed a plain numpy bool column with a non-numeric string column (string-valued category or pandas string dtype). coerce_nullable_dtypes_to_numpy now coerces numpy bool columns to float64, not only nullable extension dtypes. (#1040)

v8.0.7

Changed

• At predict time, an encoded column whose dtype differs from fit is now coerced to its fit-time dtype (and warns). For a numeric-categorical column arriving as strings, numeric-looking strings ("1.0") now match their fit category instead of all being treated as unseen. (#1015)

Fixed

• Fix a crash in the chunked-inference OOM recovery path that called torch.mps.empty_cache() unconditionally, raising Cannot execute emptyCache() without MPS backend on non-MPS devices (CUDA GPUs, CPU-only Linux) and turning a recoverable out-of-memory into a hard failure. (#1007)
• Fixed two crashes from inconsistent column dtypes: fit raising Cannot cast object dtype to float64 when a nullable extension dtype (Int64/Float64/boolean) sits next to a string categorical column, and predict raising a TypeError when a column was string/categorical at fit but arrives numeric. (#1015)

v8.0.6

Added

• Add auto_scale_n_estimators constructor argument (default True) to auto-scale n_estimators for full feature coverage on wide datasets, capped at 32. (#1000)

v8.0.5

Fixed

• Fixed a could not convert string to float crash when a feature declared categorical via categorical_features_indices is all-missing during fit but has real string values at predict. Such columns are now kept categorical instead of being demoted to a constant numeric column, so they route through the ordinal encoder consistently between fit and predict. (#1002)

v8.0.4

Added

• Add SafeTensors checkpoint loading. TabPFN can now load model checkpoints from .safetensors files in addition to the legacy .ckpt format, with non-tensor metadata (architecture name, model config, inference config) embedded in the safetensors header. (#981)
• Register tabpfn-v3-classifier-v3_20260506_ood.ckpt and tabpfn-v3-regressor-v3_20260506_ood.ckpt so they can be loaded from Hugging Face by filename. (#982)
• Add a visualisation utility to plot the predicted distribution (regression) in tabpfn.visualization (#987)

Changed

• Remove the feature selection cell from the TabPFN_Demo_Local example notebook. (#978)
• Quantize KV cache to int8 for fit_mode="fit_with_cache" on TabPFN-3 models. Reduces ICL KV cache memory ~2 with no accuracy loss. (#983)

Fixed

• Fixed a could not convert string to float crash when a categorical/string feature is all-missing during fit but has real string values at predict, caused by a fit/predict dtype-routing asymmetry in the ordinal encoder. (#992)

v8.0.3

Changed

• Significantly reduced import tabpfn time (roughly halved: ~2.4s → ~1.1s warm, and ~9s → ~5s on a cold first import) by no longer importing torch._dynamo/torch._inductor or scikit-learn's estimator-check test machinery at import time. (#972)