Support string / categorical targets for binary and multiclass problems

[angela97lin]

Continuation of #917, closes #215

[dsherry]

Great stuff! The impl looks ready to go. I left a comment about a line in ClassificationPipeline.predict_proba which I'm not sure I follow yet, and a comment about docstrings.

Its great that we have coverage for binary classification estimators’ class order, thanks for adding.

Here's a few test cases and potential changes I'd like us to address before merging:

• What happens when the target passed to fit and score disagree, and there are new unique values passed to score, or we passed str to fit and now int to score? We should raise an error explaining the problem. I think what would currently happen is a stack trace out of LabelEncoder, right?
• Try calling binary and multiclass pipeline fit, predict, predict_proba and score with various types of target (int, float, bool, category, string), and test that either the encode/decode still functions properly, or we get a clear error message.
• If we’re going to modify predict_proba col names in this PR, as it appears we are, we should have coverage for that change too. My suggestion would be to push it to Include target column name and class labels in predict/predict_proba output #645 , unless I'm misunderstanding.

Happy to talk more about this if you'd like!

[angela97lin]

@dsherry

• What happens when the target passed to fit and score disagree, and there are new unique values passed to score, or we passed str to fit and now int to score? We should raise an error explaining the problem. I think what would currently happen is a stack trace out of LabelEncoder, right?

Yup, we get a pretty clear error out of LabelEncoder but I'll catch it and rethrow with our own exception. Added test_new_unique_targets_in_score as a test for this.

• Try calling binary and multiclass pipeline fit, predict, predict_proba and score with various types of target (int, float, bool, category, string), and test that either the encode/decode still functions properly, or we get a clear error message.

I already had what's now test_targets_data_types for string and categorical testing automl searches. I added bool and float/int checks. How does that look?

• If we’re going to modify predict_proba col names in this PR, as it appears we are, we should have coverage for that change too. My suggestion would be to push it to Include target column name and class labels in predict/predict_proba output #645 , unless I'm misunderstanding.

On second thought, I'm going to update the tests and address this in #645. I know you added a potential code snippet, but I think rather than introduce even more code to fix/patch something, might as well punt it after all!

[dsherry]

@angela97lin the tests you added look great! I left one suggested for an additional check. Well done on this, there were lots of details to chase down.

It occurs to me, would be nice to have a target type test for regression too, mirroring your classification test. That one would just expect that categorical and str types fail. Not required for merge.

[angela97lin]

@dsherry Thanks for the feedback! Added in the extra assert but RE regression, I opened up #946 to ensure that we have a clear error thrown to the user and then test that; I can tackle that issue in this epic as well but wanted this merged first :)

[dsherry]

@angela97lin sounds good!