Repository navigation
ISLP 0.6.0 on PyPI: https://pypi.org/project/ISLP/0.6.0/
ModelSpec was audited before it moves to its own package in ISLP 0.7.
This release makes it a conforming scikit-learn transformer, fixes its
categorical encodings, and adds polars and pyarrow input, sparse model
matrices and tuning of the parameters of its terms. Some results change:
see Changes in behaviour.
Deprecations
ModelSpecmoves to its own package in ISLP 0.7. Every
ModelSpec.fitgives aFutureWarningsaying so:from ISLP.models import ModelSpec(and the other tools ofISLP.modelsthat build model
matrices) will keep working, but other import paths, such as
ISLP.models.model_spec, may not. The default ofinterceptchanges to
Falsethen, in ISLP 0.7, not in 0.6 as 0.5.2 announced: aModelSpec
relying on the default gets the same one warning, advising to pass
intercept=True(e.g. for statsmodels) orintercept=False. Silence the
warning withwarnings.filterwarnings('ignore', message='ModelSpec will move')(#69).
Changes in behaviour
- Ordered categoricals are coded with contrasts, in the order of their
categories, like any other categorical; they were a single column of
level codes. A score is opt-in, withordinal('x')(now exported from
ISLP.models),categorical_encoders={'x': Ordinal()}, or
{'ordinal': Ordinal()}for every orderedpd.Categorical.ordinal()
requires an explicit order: an ordered categorical, a numeric column, or
levels=[...]; a string column or an unordered categorical raises
rather than being scored alphabetically. In the Ch07 lab,
MS(['education', poly('age', d)])gives 4 dummy columns; use
ordinal('education')for the linear term (#62). - A string column used in a term raises a
ValueErrorasking to make
it categorical (astype('category')orcontrast()); it went into the
model matrix as text. Columns no term uses may be anything (#60). - Categorical encodings: unused categories are dropped (they raised);
a missing value gives a row ofNaN(it was coded as the baseline
level);contrast(..., None)and callable contrasts follow the order of
the categories, as'drop'and'sum'did (they were sorted). A level
not seen at fit raises, naming it (#60). - The dense model matrix is all
float64forpd.DataFrameinput, as
fornp.ndarrayinput; an integer column wasint64(#67). - Stricter input: 1-d, empty, complex and sparse
Xraise with
scikit-learn's messages, and annp.ndarraywith a different number of
columns at transform raises; apd.DataFrameis matched by name, as
before (#61).Column.get_columnsof aColumnwhose encoder is not fit
raises, asking forfit=True: fitting aModelSpecfits a copy of its
terms (#59, #61).
New features
- A fitted
ModelSpeccan be fit again, cloned and refit, used in
several specifications, and cross-validated:fitfits copies of the
encoders of its terms, leavingtermsunchanged. A model fit elsewhere
is a term withFrozenEstimator(#59). - Tune the parameters of the terms, e.g.
GridSearchCV(pipe, {'modelspec__terms__0__df': [2, 4, 8]})forns('x', df=4); parts of
interactions areterms__<i>__<j>__<parameter>, and a named term
terms__<name>__<parameter>. Terms made bypoly(),ns(),bs()and
pca()are made again, so their names follow (ns(x, df=8)) (#68). - polars, pyarrow and other dataframes (by narwhals) as
X; the model
matrix is returned in the same backend. A polarsEnumkeeps its order;
the levels of an unordered categorical are sorted, as pandas infers them
(#63). - Sparse model matrices with
ModelSpec(sparse_output=True): a
scipy.sparse.csr_matrix, categorical encodings and their interactions
built sparse throughout. For 200,000 rows and a categorical with 20,000
levels, 44 MB where the dense design would be 32 GB (#64). - scikit-learn conformance:
get_feature_names_out(and so
set_output),n_features_in_andfeature_names_in_, defaults
terms=()andcategorical_encoders=None, and theallow_nantag.
ModelSpec()passes scikit-learn'scheck_estimator, but for one check
documented in the tests (#61).
Bug fixes
- Refitting a
ModelSpec,cloneof a fitted one, two specifications
sharing terms, and cross-validation of a fitted one raised "encoder has
already been fit previously";contrast()terms kept the levels of their
first fit (#59). BSpline.fitchanged its parameterslower_boundandupper_bound,
so a refit used the bounds of the first fit (#59).ordinal()sorted levels alphabetically (hi < lo < mid): it used
OrdinalEncoderon values stripped of their dtype (#62).set_params(categorical_encoders=...)was ignored, and
check_is_fittedpassed on an unfitModelSpec(#61, also fixed in
#48).ContrastandOrdinalflattened a 2-column array into one column
(#61); an encoder with sparse output (e.g.OneHotEncoder()) gave a 0-d
object array (#64).build_sequencewith a boolean column: the designs kept it boolean,
so an ANOVA over them failed (#63).- Predicting a single row with
poly(),ns()orbs()raised
IndexError:Poly,BSplineandNaturalSplinemade one value 0-d
withnp.squeeze(#71).
Performance
fitdoes not build the model matrix, and the dense model matrix is
built once, in place. For 2,000,000 rows and a 50-level categorical,
fit_transformtakes 0.3s and 1.7 GB, where it took 1.3s and 2.5 GB;
fitalone 0.06s. For a polars or pyarrow frame only the columns the
terms use are converted (#67).
Documentation
- A page comparing
ModelSpecwith scikit-learn'sColumnTransformer,
patsy and formulaic, executed at build time: terms as Python objects,
features of features, interactions, tuning withGridSearchCV, sparse
model matrices, polars input, and the defaults that differ (#66). - The guide's section on ordinal variables, and two of its cells broken by
the stricter fitting (#61, #62). Executed pages do not show warnings
(#69).
Packaging
- Requires
narwhals>=1.26(#63). - CI runs the tests on Linux (Python 3.12) for pull requests, into any
branch, and on macOS, Linux and Windows for merges into main, the weekly
schedule and by hand (#65). polars and pyarrow are in thedevgroup
for the tests (#63); polars in the docs' requirements (#66).
Requires Python 3.11 or later, as 0.5.0.
Install: pip install ISLP==0.6.0.
Full Changelog: v0.5.2...v0.6.0