Skip to content

Skrub expressions: flexible pipelines with multiple tables, previews and easy hyperparameter tuning - #1233

Merged
rcap107 merged 266 commits into
skrub-data:mainfrom
jeromedockes:expressions
Apr 9, 2025
Merged

rcap107 merged 266 commits into
skrub-data:mainfrom
jeromedockes:expressions

Conversation

@jeromedockes

@jeromedockes jeromedockes commented Feb 5, 2025 •

Copy link
Copy Markdown
Member

This pull request adds the skrub expressions, which provide a new way to build complex machine-learning estimators.

  • they can handle multiple inputs of any type and perform arbitrary transformations (not limited to a sequence of scikit-learn transformers)
  • The design matrix and targets (which get split for cross-validation) don't have to be available from the start, they can be intermediate results inside the pipeline.
  • it is easy to specify ranges for any value (eg estimator hyperparameters) used in the transformation, and tune them with scikit-learn's grid-search or randomized-search
  • estimators can be built interactively, while inspecting the results of intermediate steps

The expressions represent a computation and can be evaluated to produce a result (the estimator's output). The simplest expressions are variables, which represent an input to the ML estimator. They are created with var

>>> import skrub

>>> a = skrub.var("a")
>>> b = skrub.var("b")
>>> b
<Var 'b'>

Variables can be combined with operators, functions, constants etc. to build up more complex expressions.

>>> c = a + b
>>> c
<BinOp: add>

Variables can be given a value, used to display preview results and as a source of data for fitting & validating estimators:

>>> a = skrub.var("a", 7)
>>> b = skrub.var("b", 1)
>>> c = a + b
>>> c
<BinOp: add>
Result:
―――――――
8

We can use the attributes, methods, operators of the underlying objects:

>>> (skrub.var('a', '   hello...')[:-3] + ', world!').strip().capitalize()
<CallMethod 'capitalize'>
Result:
―――――――
'Hello, world!'

Skrub functionality is accessed through the special attribute .skb

>>> c.skb.eval({"a": 10, "b": 5})
15
>>> c.skb.draw_graph()

graph

We can obtain a fittable estimator out of the expression.

>>> estimator = c.skb.get_estimator()
>>> estimator.fit_transform({"a": 2, "b": 1})
3

In the example above the transformation is stateless but expressions can contain scikit-learn estimators:

>>> import skrub.datasets

>>> dataset = skrub.datasets.fetch_employee_salaries()

>>> features = skrub.X(dataset.X).skb.apply(
...     skrub.TableVectorizer(high_cardinality=skrub.StringEncoder())
... )
>>> features
<Apply TableVectorizer>
Result:
―――――――
      gender_F  gender_M  ...  date_first_hired_total_seconds  year_first_hired
0          1.0       0.0  ...                    5.277312e+08            1986.0
1          0.0       1.0  ...                    5.900256e+08            1988.0
2          1.0       0.0  ...                    6.274368e+08            1989.0
3          0.0       1.0  ...                    1.399248e+09            2014.0
4          0.0       1.0  ...                    1.173053e+09            2007.0
...        ...       ...  ...                             ...               ...
9223       1.0       0.0  ...                    1.446509e+09            2015.0
9224       1.0       0.0  ...                    5.966784e+08            1988.0
9225       0.0       1.0  ...                    9.885888e+08            2001.0
9226       0.0       1.0  ...                    1.157414e+09            2006.0
9227       0.0       1.0  ...                    1.327882e+09            2012.0

[9228 rows x 143 columns]

(A transformer can easily be applied to a subset of columns with the cols parameter)

Any value can be replaced by a choice, which is then used to build a hyperparameter grid:

>>> from sklearn.ensemble import HistGradientBoostingRegressor, RandomForestRegressor

>>> hgb = HistGradientBoostingRegressor(
...     learning_rate=skrub.choose_float(0.01, 0.8, log=True, name="lr")
... )
>>> rf = RandomForestRegressor(
...     n_estimators=skrub.choose_int(8, 128, log=True, name="N trees")
... )
>>> regressor = skrub.choose_from({"hgb": hgb, "rf": rf}, name="regressor")

>>> y = skrub.y(dataset.y)
>>> pred = features.skb.apply(regressor, y=y)
>>> pred
<Apply HistGradientBoostingRegressor>
Result:
―――――――
      current_annual_salary
0              67678.566435
1              96062.630076
2             107967.887714
3              54766.663259
4              88773.361530
...                     ...
9223           73232.819304
9224          154673.199482
9225          101625.833270
9226          143800.018771
9227           74987.836900

>>> print(pred.skb.describe_param_grid())
- regressor: 'hgb'
  lr: choose_float(0.01, 0.8, log=True, name='lr')
- regressor: 'rf'
  N trees: choose_int(8, 128, log=True, name='N trees')

>>> search = pred.skb.get_randomized_search(fitted=True, n_jobs=8)
>>> search.results_
   mean_test_score regressor        lr  N trees
0         0.914610       hgb  0.185372      NaN
1         0.911956       hgb  0.206369      NaN
2         0.909182       hgb  0.309576      NaN
3         0.902986        rf       NaN    113.0
4         0.901762        rf       NaN     92.0
5         0.901231       hgb  0.055610      NaN
6         0.900955        rf       NaN     88.0
7         0.900353        rf       NaN     40.0
8         0.891114        rf       NaN     14.0
9         0.768370       hgb  0.014172      NaN

(we can also get plotly parallel coordinate plots)

There's a lot more to say so please refer to the examples & reference documentation.

very much WIP, I'm opening the PR so the first example renders and we can discuss it.

There's still some work to do especially on the examples, but IRL we decided to merge it before the upcoming sprint and improve the examples in follow-up pull requests.

@GaelVaroquaux

Copy link
Copy Markdown
Member

Very much WIP, but already a cool example to look at https://output.circle-artifacts.com/output/job/34c076e7-2a4d-4616-946d-bfb5c27d2167/artifacts/0/doc/auto_examples/10_expressions.html#sphx-glr-auto-examples-10-expressions-py
🤩

Comment thread examples/10_expressions.py Resolved
@jeromedockes jeromedockes mentioned this pull request Feb 18, 2025
@jeromedockes

Copy link
Copy Markdown
Member Author

thanks @GaelVaroquaux !!

If you want, you can add a "section" to the front page, below the "Interactive Data Exploration" section. But if you prefer, we can do this in a follow up PR.

good idea. I was thinking maybe we can manage to get this PR in before the sprint and in this case this would be a good thing to do in the sprint. but if it's not merged by then I'll add it in this PR. WDYT?

Small comment: I think that I'd prefer having the API docs of the Expressions symbols not in a separate page, but in the main page of the API docs. Right now, it's too easy to miss it while scrolling through the API page

ok that's what I had at first but I thought the table of contents was getting really long ... I'll put it back but maybe the organization of this page could also be something to work on in the sprint. for example maybe all fetchers could be in a separate page

@GaelVaroquaux

Copy link
Copy Markdown
Member

If you want, you can add a "section" to the front page, below the "Interactive Data Exploration" section. But if you prefer, we can do this in a follow up PR.

I was thinking maybe we can manage to get this PR in before the sprint and in this case this would be a good thing to do in the sprint.

Sounds right. Let's do this!

@jeromedockes jeromedockes changed the title Flexible pipelines with multiple tables, previews and easy hyperparameter tuning Skrub expressions: flexible pipelines with multiple tables, previews and easy hyperparameter tuning Apr 9, 2025
@jeromedockes
jeromedockes marked this pull request as ready for review April 9, 2025 09:57
@jeromedockes

Copy link
Copy Markdown
Member Author

circleCI is failing to checkout the repository, I don't think this is specific to skrub because the same problem was happening on hidimstat this morning

@rcap107
rcap107 merged commit e8f29ea into skrub-data:main Apr 9, 2025
@GaelVaroquaux

GaelVaroquaux commented Apr 9, 2025 via email

Copy link
Copy Markdown
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants