You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Per-fold feature-importance CSV ({model}_feature_importance_per_fold.csv) with one row per outer CV fold and, for linear models, the intercept of each fold
Documentation on reading linear coefficients: they refer to standardized and one-hot encoded features and are taken before probability calibration; the feature-importance page now shows how they relate to the calibrated probabilities and how to extract them from the saved models
Configuration validation: model names are checked against the available models (a misspelt name used to be skipped at run time with a warning while the run reported success), outer_folds and inner_folds must be at least 2, verbosity must be 0, 1 or 2, and an empty targets is rejected
Warning when a categorical feature has high cardinality (at least 20 distinct values and more than half as many as rows), since it is one-hot encoded into one column per level
training_metadata.json records calibration_bins and the resolved threshold_method (oof or cv instead of auto); respredai evaluate uses the recorded bin count for ECE and MCE
Temporal validation reports progress in the CLI
Checkpoints store the per-fold reliability-curve data, so a resumed or re-loaded run still produces reliability curves
Changed
extract_feature_importance_from_models() also returns the per-fold intercepts
Logistic-regression hyperparameter grid expressed with l1_ratio on scikit-learn 1.8 or newer (same 71 combinations), avoiding the deprecation of penalty and its removal planned for 1.10
Model names are case-insensitive and normalised to their canonical spelling (lr selects LR), duplicates are dropped; applies to [Pipeline] models, run --models, train --models and feature-importance --model
continuous_features may be empty or omitted when every feature is categorical
With imputation disabled, the missing-value check covers the feature columns only and names the offending columns; subgroup columns may contain missing values, which are reported as Unknown
Feature-name sanitisation also replaces [ and ], which XGBoost rejects in feature names, in addition to < and >
With threshold_method = cv the saved fold model is the (calibrated) classifier itself with the tuned threshold stored beside it, as for oof; TunedThresholdClassifierCV is used only to search the threshold. With a group_column that search runs on group-aware inner splits, while the calibrator refit inside the search uses plain stratified folds
Removed
tqdm dependency (unused); torch is no longer required unless the tabpfn extra is installed
Fixed
import respredai failed without torch, which has not been a dependency of the base package since tabpfn became an optional extra; torch is now imported only when the TabPFN model is built
Resuming a run from a partially completed checkpoint raised KeyError and wrote a corrupted checkpoint (duplicated metric rows, missing fold models, completed_folds equal to the total); the per-fold bookkeeping is now recorded in one step after every fallible computation
threshold_method = cv (also selected by auto from 1000 training rows) with a group_column failed every fold, because sklearn rejects groups= on TunedThresholdClassifierCV without metadata routing; the group-aware splits are now passed as a precomputed list
respredai train with a group_column and calibrate_probabilities = true crashed with "indices are out-of-bounds" in the conformal step
Temporal validation swallowed per-target failures and the CLI reported success; a failed model/target now raises RuntimeError like the CV pipeline
The HTML report assumed CI95_* columns, so confidence intervals disappeared when confidence_level was not 0.95 while the headers still said 95%
Progress bars used outer_folds instead of folds times repeats with outer_cv_repeats > 1
Loading several configurations in one process added a log file handler each time, so later runs also logged to the earlier files
feature-importance dropped the sign of LR and Linear_SVC coefficients, so protective (negative) coefficients were reported as positive
With calibrate_probabilities = true, feature importance came from a single calibration sub-model instead of the whole fold model; it is now averaged over all sub-models