Skip to content

Release: 0.5.2

Choose a tag to compare

@github-actions github-actions released this 29 Oct 22:53
  • Added named pipelines (the new pipelines module and Pipeline class).
    pipeline = classification in the "general" section (or the project's
    section) fills in the settings of the analyst, critic, and artist, so a
    study can be a label, a seed, and a pipeline. The pipelines are
    classification, regression, and count_regression, which compare
    models in an experiment, and classification_inference,
    regression_inference, count_inference, panel_classification, and
    panel_regression, which fit one model for its coefficients (without
    scaling or resampling, and with each category encoded against a
    reference level). Settings that you write win over the pipeline's, a
    worker's own section can name a pipeline for that worker, the pipeline
    sets "task" if it is not set, and the filled-in settings are in
    Project.idea and the exported "settings.json". Add a Pipeline to
    pipelines.PIPELINES to name your own.
  • Added three choosers, which the pipelines use and "auto" now resolves to
    in steps named "impute", "encode", and "evaluate" (and in a worker named
    "critic"): auto_imputer fills missing values (with iterative_impute
    when many numeric values are missing, median_impute otherwise, and
    mode_impute for categories); auto_encoder encodes each categorical
    feature by its number of levels (one_hot for 15 or fewer, target for
    more with a label of two classes or a regressed one, count for a label
    of more classes, tfidf for free text, and date_parts for dates); and
    auto_evaluator applies the evaluators that suit the task and the model.
  • auto_model knows the needs of ols, logit_sm, probit, and
    mnlogit (such as the number of classes), which it considers when they
    are named in "choices".
  • Fixed smote_tomek and smote_enn marking every training row as
    synthetic, because they remove some of the original rows after adding
    rows. Validators then had no real rows to cross-validate, and choosers
    profiled none. Each row they keep that equals an original row is now
    real.
  • Fixed fixest predicting missing values (and so a scorecard failing
    with "Input contains NaN") for rows whose level of a fixed effect has no
    estimate: a level that was not in the training rows, or one that
    pyfixest dropped (a singleton, or a level separated by having rows of
    only one class). Such a row is now predicted with that fixed effect at
    the average of its estimated levels. The estimates of the levels are in
    the model's effect_levels_. A fixest model can therefore be
    validated by group_k_fold (each held-out group is predicted with the
    average fixed effect), which used to fail with "could not predict".
  • Fixed permutation_importance, shap_partial_dependence, and
    decision_regions failing for a model that needs columns besides its
    features, such as the fixed effects and cluster of a fixest model.
    permutation_importance and shap_partial_dependence hold those columns
    at each row's own values (as shap_importance does), and
    decision_regions holds them at their most common values in the real
    training rows (as it holds the other features at their medians).
    permutation_importance also works for models that are not scikit-learn
    estimators, which scikit-learn's scorers rejected.
  • Fixed the report of a project saying that an experiment that conserves
    memory chose "the best of 1 paths". It now gives the number of paths
    compared.
  • Inferers (such as dr_learner and partially_linear) now leave out the
    rows that a sampler (such as random_over or smote) added to the
    training rows, which counted some rows more than once and biased the
    estimates and their standard errors. The data is put back afterwards.