Fix a bug where categorical features created by interaction caused Training step 3 to crash #111
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ficients bug This bug is triggered when a categorical feature is interacted with another feature. The resulting feature is also categorical, but its name may not end with _onehotencoded because the user can specify its name in the config file. So we can't rely on column names ending with _onehotencoded to specify which features are categorical.
…g step 3 to crash These features are categorical but may not end with the suffix "_onehotencoded". We now check the data type of the columns in training_features_prepped and take any column with type "vector" to be categorical and explode it. This seems pretty robust. I also improved the test for this bug a bit.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #110.
We caught this bug because we set
strict=Truein the call tozip(). Hooray!The user names these features in the config file. So they probably don't end in
_onehotencodedlike other categorical variables. To determine if a variable is categorical, we now check if its data type is"vector". If so, then we strip any_onehotencodedsuffix off of it and explode it fortraining_feature_importances. This handles both normal and interacted categorical variables nicely.