Skip to content

v1.9.4

Latest

Choose a tag to compare

@EttoreRocchi EttoreRocchi released this 30 Sep 16:01

Added

  • Per-fold feature-importance CSV ({model}_feature_importance_per_fold.csv) with one row per outer CV fold and, for linear models, the intercept of each fold
  • Documentation on reading linear coefficients: they refer to standardized and one-hot encoded features and are taken before probability calibration; the feature-importance page now shows how they relate to the calibrated probabilities and how to extract them from the saved models
  • Configuration validation: model names are checked against the available models (a misspelt name used to be skipped at run time with a warning while the run reported success), outer_folds and inner_folds must be at least 2, verbosity must be 0, 1 or 2, and an empty targets is rejected
  • Warning when a categorical feature has high cardinality (at least 20 distinct values and more than half as many as rows), since it is one-hot encoded into one column per level
  • training_metadata.json records calibration_bins and the resolved threshold_method (oof or cv instead of auto); respredai evaluate uses the recorded bin count for ECE and MCE
  • Temporal validation reports progress in the CLI
  • Checkpoints store the per-fold reliability-curve data, so a resumed or re-loaded run still produces reliability curves

Changed

  • extract_feature_importance_from_models() also returns the per-fold intercepts
  • Logistic-regression hyperparameter grid expressed with l1_ratio on scikit-learn 1.8 or newer (same 71 combinations), avoiding the deprecation of penalty and its removal planned for 1.10
  • Model names are case-insensitive and normalised to their canonical spelling (lr selects LR), duplicates are dropped; applies to [Pipeline] models, run --models, train --models and feature-importance --model
  • continuous_features may be empty or omitted when every feature is categorical
  • With imputation disabled, the missing-value check covers the feature columns only and names the offending columns; subgroup columns may contain missing values, which are reported as Unknown
  • Feature-name sanitisation also replaces [ and ], which XGBoost rejects in feature names, in addition to < and >
  • With threshold_method = cv the saved fold model is the (calibrated) classifier itself with the tuned threshold stored beside it, as for oof; TunedThresholdClassifierCV is used only to search the threshold. With a group_column that search runs on group-aware inner splits, while the calibrator refit inside the search uses plain stratified folds

Removed

  • tqdm dependency (unused); torch is no longer required unless the tabpfn extra is installed

Fixed

  • import respredai failed without torch, which has not been a dependency of the base package since tabpfn became an optional extra; torch is now imported only when the TabPFN model is built
  • Resuming a run from a partially completed checkpoint raised KeyError and wrote a corrupted checkpoint (duplicated metric rows, missing fold models, completed_folds equal to the total); the per-fold bookkeeping is now recorded in one step after every fallible computation
  • threshold_method = cv (also selected by auto from 1000 training rows) with a group_column failed every fold, because sklearn rejects groups= on TunedThresholdClassifierCV without metadata routing; the group-aware splits are now passed as a precomputed list
  • respredai train with a group_column and calibrate_probabilities = true crashed with "indices are out-of-bounds" in the conformal step
  • Temporal validation swallowed per-target failures and the CLI reported success; a failed model/target now raises RuntimeError like the CV pipeline
  • The HTML report assumed CI95_* columns, so confidence intervals disappeared when confidence_level was not 0.95 while the headers still said 95%
  • Progress bars used outer_folds instead of folds times repeats with outer_cv_repeats > 1
  • Loading several configurations in one process added a log file handler each time, so later runs also logged to the earlier files
  • feature-importance dropped the sign of LR and Linear_SVC coefficients, so protective (negative) coefficients were reported as positive
  • With calibrate_probabilities = true, feature importance came from a single calibration sub-model instead of the whole fold model; it is now averaged over all sub-models