Change the grid search order to optimize caching in the new Pipeline

[knarf-hackatal2016]

Version: current master branch (19.)

GridSearchCV fits all the classifiers on the first split, then all the classifers on the second split, etc.

This is suboptimal for caching in the pipeline, since most intermediate transformers will be fitted with same input when the grid search runs with several jobs (since no caching has been done yet).

The order should be: fit the first classifier on all splits, then second classifier on all splits, etc

[jnothman]

Pull request for this at #8322

[knarf-hackatal2016]

Woops, missed this part of the discussion.

Awesome, closing the issue. Aside from that caching is working great for me.

[jnothman]

I'd actually like this open to keep track. Thanks_

[jnothman]

Good to hear caching is working for you!

[GaelVaroquaux]

Why is this suboptimal? In other terms, what does changing the order of traversal bring?

[knarf-hackatal2016]

Let's say you have 2 transformers A, B and a classifier C.

Consider the pipelne: (P): A -> B -> C .

Let's use GridSearchCV on (P) with a 3 folds CV.

Let's say A will be tested with two sets of parameters A' and A'', B with B' and B'', C with C' and C''.

Let's say you have two jobs.

At start the two first jobs will fit A' -> B' -> C' and A' -> B' -> C'' on the same split. It means A' and B' will be fitted twice with the same input data. Which is what caching should avoid.

Now if you start with A' -> B' -> C' on first and second splits: it's ok because the input data is different. It will be cached at every step. When later you fit A' -> B' -> C'', A' and B' should already have been cached before and can already be loaded.