Support pipelines as graphs of components

[dsherry]

In #229 and related slack discussion today, @jeremyliweishih and @angela97lin were chatting about whether we should require pipelines to have exactly one estimator.

Currently, it appears we expect a pipeline to consist of a single string of components to be evaluated linearly, with exactly one estimator at the end. In PipelineBase::predict, we call transform on every component in the line, then call predict on the estimator. self.estimator is currently used in predict, predict_proba and feature_importances in ways which are consistent with that expectation.

We can imagine pipelines where:

1. The estimator isn't the final component in the pipeline
2. There are multiple estimators in the pipeline (Perhaps its easier or more clear to do that instead of defining a custom component to implement that)
3. There are multiple preprocessing pathways of components (which presumably eventually merge together)
4. There's no estimator at all; the pipeline in question is just doing some preprocessing or feature extraction (this wouldn't be useful in the automl leaderboard as we currently envision it)

Questions:

• Which of those cases should we plan to support?
• What invariants should we check on pipeline structure?
• How can we change our code to enforce them but allow some of the cases described here?
• The answers here may take a while to implement. What if anything do we do in the meantime?

[dsherry]

I will try to find some concrete examples of why this is a useful feature to support. We should justify the value before working on it.

For now, I wanted to share a picture I drew which at least shows how this would work.

The top part shows a graph where we train a regression model (xgboost in the picture) on preprocessed data, then take the residual of that first model and fit a second model (random forest in the picture) on that, using the same preprocessed data that was given to the first model. Then the graph sums the outputs of the two models and returns that. This pattern could be interesting to try, although we may not end up using xgboost or random forest in particular.

The bottom part shows a list of the components in the graph. The order in the list isn't important to the algorithm we'd write to fit and evaluate the graph, it just defines the IDs of each node. (although we could certainly use component hashing for indexing instead of using the components' indexes in the list)

In other words, a component graph consists of two things: 1. a list of components, and 2. a graph structure describing how those components are connected.

If we go with this structure, we'd need an API for specifying the nodes and edges in a graph. We'd also need a way to show all that information to users, so they can introspect on the graphs.

In order to evaluate a pipeline which uses a graph like this, we'd need an algorithm that could traverse the graph and evaluate the components in order of their use. That will be a fun design challenge; actually, there's probably a few third-party packages which we could use to do this. Dask supports task graphs.

[dsherry]

@kmax12 just posted this new sklearn feature in the evalml-dev channel: html representations of pipelines. I'm posting it here because it shows a great example of a simple multi-path pipeline: there's one preprocessing path for numeric inputs and another for categorical inputs, and then both are provided to the estimator.