Rework SV logic

[LilianBoulard]

I noticed some inconsistencies in the behavior of the SuperVectorizer, which this PRs aims to fix:

• The _auto_cast method was called on the data passed to transform, which meant that types of the fit_transform output data could be different from the types of the transform output data.
• The types_ variable was overwritten during transform, which shouldn't happen as this is a mapping of the types learnt during fit.
• "False missing" values (for a lack of a better term) were only replaced in the _auto_cast function, which made sense as it was used in the transform as well. Now, it has been moved to its own function
• Imputation is not consistent. We might want to rework this logic.

[GaelVaroquaux]

This PR has conflicts and failing tests

[LilianBoulard]

So, the failing test was because of an issue that affects pandas version 1.1.5 (our min dependency until now), and possibly earlier versions (I haven't checked).
Pandas 1.2.0 was released on December 26th, 2020, only a few weeks after 1.1.5 (December 7th), and fixes this bug, so I guess bumping is not a big deal.

[GaelVaroquaux]

This looks overall good, but we do have the problem of terminology of "cat" vs "str". It seems to me that it should be "str" quite often.

[LilianBoulard]

So, the PR is overall stable, but there's a last thing I'm concerned about: the imputation strategy.

Currently, the logic is to impute missing values categorical / string with a custom value "missing", depending on the impute_missing parameter.

There is currently no imputation for missing values in datetime and numerical columns.
I'm kind of curious as to why this has not been a problem so far: do most models support missing values out of the box ?
What do you think @GaelVaroquaux ?

[GaelVaroquaux]

There is currently no imputation for missing values in datetime and numerical columns.

This may need to be done by a sophisticated imputation methods (unlike with categorical data), so we leave it to the user to add an Imputer in the pipeline. Another option is to use a model that natively supports missing values.

[LilianBoulard]

Okay so these last few commits add some doc to clarify to the user what we impute and what we don't.

Edit: one last thing we need to address is the way the remainder is used, as currently it cannot be applied to the *_transformer parameters because before, it was used when None was passed. Now, None is transformed to a default transformer as pointed out in the doc.

[LilianBoulard]

Everything looks good to me!
I'm waiting for your feedback, and I'll merge it right away :)

[GaelVaroquaux]

I'm not sure that I am following the logic fully. If I understand things right, if I put numerical_transformer="pasthrough" (a typo on "passthrough", the numerical_transformer ends up being the high_card_cat_transformer. This sounds like a dangerous behavior, as it will lead to people having bugs in their pipelines.

[LilianBoulard]

Ah, no, sorry that was en error, it should be self.numerical_transformer_ = self.numerical_transformer

[LilianBoulard]

And so if there was a typo, it will crash down the line when the ColumnTransformer's fit_transform will call _validate_transformers.

[GaelVaroquaux]

OK, thanks. Please add a comment that says that.

[GaelVaroquaux]

Please add a small docstring to describe to the developer what this function does.
It does not need to be a properly formatted docstring describing all parameters, as this is an internal function, we just need to describe why this function

[GaelVaroquaux]

Two things to modify:

• The docstring to add
• The error on the numerical vs categorical transformer

After this, we merge. Don't do more changes, or I'll find other comments :)