Repository navigation
Skrub expressions: flexible pipelines with multiple tables, previews and easy hyperparameter tuning - #1233
Conversation
|
Very much WIP, but already a cool example to look at https://output.circle-artifacts.com/output/job/34c076e7-2a4d-4616-946d-bfb5c27d2167/artifacts/0/doc/auto_examples/10_expressions.html#sphx-glr-auto-examples-10-expressions-py |
|
thanks @GaelVaroquaux !!
good idea. I was thinking maybe we can manage to get this PR in before the sprint and in this case this would be a good thing to do in the sprint. but if it's not merged by then I'll add it in this PR. WDYT?
ok that's what I had at first but I thought the table of contents was getting really long ... I'll put it back but maybe the organization of this page could also be something to work on in the sprint. for example maybe all fetchers could be in a separate page |
Sounds right. Let's do this! |
Rewording and splitting example 11 into two
|
circleCI is failing to checkout the repository, I don't think this is specific to skrub because the same problem was happening on hidimstat this morning |
|
Wohooo!
|
This pull request adds the skrub expressions, which provide a new way to build complex machine-learning estimators.
The expressions represent a computation and can be evaluated to produce a result (the estimator's output). The simplest expressions are variables, which represent an input to the ML estimator. They are created with
varVariables can be combined with operators, functions, constants etc. to build up more complex expressions.
Variables can be given a value, used to display preview results and as a source of data for fitting & validating estimators:
We can use the attributes, methods, operators of the underlying objects:
Skrub functionality is accessed through the special attribute
.skbWe can obtain a fittable estimator out of the expression.
In the example above the transformation is stateless but expressions can contain scikit-learn estimators:
(A transformer can easily be applied to a subset of columns with the
colsparameter)Any value can be replaced by a choice, which is then used to build a hyperparameter grid:
(we can also get plotly parallel coordinate plots)
There's a lot more to say so please refer to the examples & reference documentation.
very much WIP, I'm opening the PR so the first example renders and we can discuss it.There's still some work to do especially on the examples, but IRL we decided to merge it before the upcoming sprint and improve the examples in follow-up pull requests.