RELEASE_NOTES.rst

Release Notes

FACET 2.1

FACET 2.1 introduces the :class:`.NativeLearnerInspector` for inspecting native scikit-learn models and pipelines.

We still recommend using :mod:`sklearndf` models and learner pipelines along with FACET's :class:`.LearnerSelector` for hyperparameter tuning; however the new :class:`.NativeLearnerInspector` can be useful for inspecting models that have been trained using scikit-learn directly.

2.1.1

This is a maintenance release to catch up with FACET 2.0.1.

2.1.0

• API: new :class:`.NativeLearnerInspector` class for inspecting native scikit-learn regressors, classifiers, and pipelines with a regressor or classifier as the final estimator

FACET 2.0

FACET 2.0 brings numerous API enhancements and improvements, accelerates model inspection by up to a factor of 50 in many practical applications, introduces a new, more flexible and user-friendly API for hyperparameter tuning – with support for scikit-learn's native hyperparameter searchers – and improves the styling of all visualizations.

FACET 2.0 requires :mod:`pytools` 2.0 and :mod:`sklearndf` 2.2, and is now fully type-checked by |mypy|.

2.0.1

• API: class :class:`.LearnerInspector` now supports inspecting individual regressors and classifiers; it is no longer necessary to wrap them into a :class:`.RegressorPipelineDF` or :class:`.ClassifierPipelineDF` instance with empty preprocessing
• FIX: replace a call to method get_text_heights() of :class:`matplotlib.axes.Axes`, which is deprecated as of :mod:`matplotlib` 3.6

2.0.0

facet.data

• API: class :class:`.Sample` raises an exception if the name of any used column is not a string
• API: class :class:`.RangePartitioner` supports new optional arguments lower_bound and upper_bound in method :meth:`~.RangePartitioner.fit` and no longer accepts them in the class initializer

facet.explanation

• REFACTOR: moved explainer factories from module :mod:`facet.inspection` to new module :mod:`facet.explanation`.
• API: new explainer factories :class:`.ExactExplainerFactory` and :class:`.PermutationExplainerFactory`, in addition to the :class:`.TreeExplainerFactory` and :class:`.KernelExplainerFactory` introduced in FACET 1.0

facet.inspection

• API: new :class:`.FunctionInspector` class for inspecting arbitrary functions, using a :class:`.ExactExplainerFactory` by default
• API: :class:`.LearnerInspector` no longer uses learner crossfits and instead inspects models using a single pass of SHAP calculations, usually leading to performance gains of up to a factor of 50
• API: return :class:`.LearnerInspector` matrix outputs as :class:`~pytools.data.Matrix` instances
• API: diagonals of feature synergy, redundancy, and association matrices are now nan instead of 1.0
• API: the leaf order of :class:`~pytools.data.LinkageTree` objects generated by feature_…_linkage methods of :class:`.LearnerInspector` is now the same as the row and column order of :class:`~pytools.data.Matrix` objects returned by the corresponding feature_…_matrix methods of :class:`.LearnerInspector`, minimizing the distance between adjacent leaves. The old sorting behaviour of FACET 1.x can be restored using method :meth:`~pytools.data.LinkageTree.sort_by_weight`

facet.selection

• API: :class:`.LearnerSelector` replaces FACET 1.x class LearnerRanker, and provides a new, more flexible and user-friendly API for hyperparameter tuning
• API: :class:`.LearnerSelector` introduces support for any CV searcher implementing scikit-learn's CV search API, including scikit-learn's native searchers such as :class:`~sklearn.model_selection.GridSearchCV` or :class:`~sklearn.model_selection.RandomizedSearchCV`
• API: new classes :class:`.ParameterSpace` and :class:`.MultiEstimatorParameterSpace` offer a more convenient and robust mechanism for declaring options or distributions for hyperparameter tuning
• API: new class :class:`.LearnerSelector` supports a new, more flexible and user-friendly API for hyperparameter tuning

facet.simulation

• API: simulations no longer depend on learner crossfits and instead are carried out as a single pass on the full dataset, using the standard error of mean predictions to obtain confidence intervals that less conservative yet more realistic
• VIZ: minor tweaks to simulation plots and reports generated by :class:`.SimulationDrawer`

facet.validation

• API: removed class FullSampleValidator

Other

• VIZ: significant updates to the styling of all visualizations, especially those generated for output of :class:`.LearnerInspector`, using the all-new versions of :mod:`pytools` matrix and dendrogram drawers
• API: class LearnerCrossfit is no longer needed in FACET 2.0 and has been removed
• API: support new :obj:`~pytools.fit.fitted_only` decorator introduced in :mod:`pytools` 2.1.

FACET 1.2

FACET 1.2 adds support for sklearndf 1.2 and scikit-learn 0.24. It also introduces the ability to run simulations on a subsample of the data used to fit the underlying crossfit. One example where this can be useful is to use only a recent period of a time series as the baseline of a simulation.

1.2.3

• BUILD: pin down matplotlib version to < 3.6 and scipy version to < 1.9 to ensure compatibility with pytools 1.2 and sklearndf 1.2

1.2.2

• catch up with FACET 1.1.2

1.2.1

• FIX: fix a bug in :class:`.UnivariateProbabilitySimulator` that was introduced in FACET 1.2.0
• catch up with FACET 1.1.1

1.2.0

• BUILD: added support for sklearndf 1.2 and scikit-learn 0.24
• API: new optional parameter subsample in method :meth:`.BaseUnivariateSimulator.simulate_feature` can be used to specify a subsample to be used in the simulation (but simulating using a crossfit based on the full sample)

FACET 1.1

FACET 1.1 refines and enhances the association/synergy/redundancy calculations provided by the :class:`.LearnerInspector`.

1.1.2

• DOC: use a downloadable dataset in the getting started notebook
• FIX: import catboost if present, else create a local module mockup
• FIX: correctly identify if sample_weights is undefined when re-fitting a model on the full dataset in a LearnerCrossfit
• BUILD: relax package dependencies to support any numpy version 1.`x` from 1.16

1.1.1

• DOC: add reference to FACET research paper on the project landing page
• FIX: correctly count positive class frequency in UnivariateProbabilitySimulator

1.1.0

• API: SHAP interaction vectors can (in part) also be influenced by redundancy among features. This can inflate quantifications of synergy, especially in cases where two variables are highly redundant. FACET now corrects interaction vectors for redundancy prior to calculating synergy. Technically we ensure that each interaction vector is orthogonal w.r.t the main effect vectors of both associated features.
• API: FACET now calculates synergy, redundancy, and association separately for each model in a crossfit, then returns the mean of all resulting matrices. This leads to a slight increase in accuracy, and also allows us to calculate the standard deviation across matrices as an indication of confidence for each calculated value.
• API: Method :meth:`.LearnerInspector.shap_plot_data` now returns SHAP values for the positive class of binary classifiers.
• API: Increase efficiency of ModelSelector parallelization by adopting the new :class:`pytools.parallelization.JobRunner` API provided by :mod:`pytools`
• BUILD: add support for :mod:`shap` 0.38 and 0.39

FACET 1.0

1.0.3

• FIX: restrict package requirements to gamma-pytools 1.0 and sklearndf 1.0, since FACET 1.0 is not compatible with gamma-pytools 1.1

1.0.2

This is a maintenance release focusing on enhancements to the CI/CD pipeline and bug fixes.

• API: add support for |shap| 0.36 and 0.37 via a new :class:`.BaseExplainer` stub class
• FIX: apply color scheme to the histogram section in :class:`.SimulationMatplotStyle`
• BUILD: add support for :mod:`numpy` 1.20
• BUILD: updates and changes to the CI/CD pipeline

1.0.1

Initial release.