Add facility to quickly define a linear transformers-predictor pipeline

[ablaom]

I think a macro is the easiest way to do this, given the existing learning networks API. Syntax would look something like:

composite_model =  @pipeline transformer1 transformer2 predictor

The things on the right are models and the result composite_model is just another model whose hyper parameters are called "transformer1", "transformer2", "predictor" and the (mutable) values of these are set to to transformer1, transformer2, predictor. Mutating these would mutate composite_model.

[fkiraly]

I think this is great, but would imitate a flaw of the vanilla sklearn design: you don't specify which variables/covariates the transformer ought to be applied to.

Makes sense to apply it in default to "all applicable" (e.g., one-hot to all categorical), but that's not always what one wants (e.g., PCA only to the variables coming from the questionnaire etc).

Potential solution is to specify pairs, transformer & variable names?

[ablaom]

Good point! However, making a column-selective transformer is, in my view, just a different kind of learning network, for which we could have another macro (or composite model type). So, e.g., you would do:

pca = PCA(n=2)  # instantiate a PCA model
restricted_pca = @restrict pca features=[:x1, :x2, :x3, :x4]    # defines and instantiates a restricted PCA model
composite_model = @pipeline restricted_pca random_forest   # defines and instantiates transformer-predictor composite                

[fkiraly]

But would that not kill all other features entirely rather than just apply PCA only to these?

[ablaom]

No, no. The incoming data into two, applies PCA to one part, then reassembles. Maybe "restrict" is a bad name. Perhaps "selective" is better. I am not going into the implementation here.

[fkiraly]

Ah, makes sense.
Though if you use "select" it would leave open how to tell the pipeline to transform resulting variables, i.e., new ones that are produced by the first transformer.

There's at least one of the two issues, depending on the design:
(a) "by default apply to all, sequentially" - this has the problem that a simple pipeline either kills variables, or has problems with selection
(b) "by default, add variables" - this has the problem that it it not straightforward how to chain transformers, since you would need to refer to the resulting variables somehow

Your comment indicates that you favour (b)? Or, do you have an idea which altogether avoids the issues? E.g., a default convention for output variables?

[ablaom]

Implemented some time ago. Query ?@pipeline for details.