You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Added named pipelines (the new pipelines module and Pipeline class). pipeline = classification in the "general" section (or the project's
section) fills in the settings of the analyst, critic, and artist, so a
study can be a label, a seed, and a pipeline. The pipelines are classification, regression, and count_regression, which compare
models in an experiment, and classification_inference, regression_inference, count_inference, panel_classification, and panel_regression, which fit one model for its coefficients (without
scaling or resampling, and with each category encoded against a
reference level). Settings that you write win over the pipeline's, a
worker's own section can name a pipeline for that worker, the pipeline
sets "task" if it is not set, and the filled-in settings are in Project.idea and the exported "settings.json". Add a Pipeline to pipelines.PIPELINES to name your own.
Added three choosers, which the pipelines use and "auto" now resolves to
in steps named "impute", "encode", and "evaluate" (and in a worker named
"critic"): auto_imputer fills missing values (with iterative_impute
when many numeric values are missing, median_impute otherwise, and mode_impute for categories); auto_encoder encodes each categorical
feature by its number of levels (one_hot for 15 or fewer, target for
more with a label of two classes or a regressed one, count for a label
of more classes, tfidf for free text, and date_parts for dates); and auto_evaluator applies the evaluators that suit the task and the model.
auto_model knows the needs of ols, logit_sm, probit, and mnlogit (such as the number of classes), which it considers when they
are named in "choices".
Fixed smote_tomek and smote_enn marking every training row as
synthetic, because they remove some of the original rows after adding
rows. Validators then had no real rows to cross-validate, and choosers
profiled none. Each row they keep that equals an original row is now
real.
Fixed fixest predicting missing values (and so a scorecard failing
with "Input contains NaN") for rows whose level of a fixed effect has no
estimate: a level that was not in the training rows, or one that
pyfixest dropped (a singleton, or a level separated by having rows of
only one class). Such a row is now predicted with that fixed effect at
the average of its estimated levels. The estimates of the levels are in
the model's effect_levels_. A fixest model can therefore be
validated by group_k_fold (each held-out group is predicted with the
average fixed effect), which used to fail with "could not predict".
Fixed permutation_importance, shap_partial_dependence, and decision_regions failing for a model that needs columns besides its
features, such as the fixed effects and cluster of a fixest model. permutation_importance and shap_partial_dependence hold those columns
at each row's own values (as shap_importance does), and decision_regions holds them at their most common values in the real
training rows (as it holds the other features at their medians). permutation_importance also works for models that are not scikit-learn
estimators, which scikit-learn's scorers rejected.
Fixed the report of a project saying that an experiment that conserves
memory chose "the best of 1 paths". It now gives the number of paths
compared.
Inferers (such as dr_learner and partially_linear) now leave out the
rows that a sampler (such as random_over or smote) added to the
training rows, which counted some rows more than once and biased the
estimates and their standard errors. The data is put back afterwards.