Feature/groupby2 #58
Merged
Feature/groupby2 #58
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Feature/groupby
|
Remove file to upload changed version
dev-rinchin
reviewed
Jan 11, 2023
dev-rinchin
requested changes
Jan 11, 2023
dev-rinchin
reviewed
Jan 11, 2023
dev-rinchin
reviewed
Jan 12, 2023
dev-rinchin
reviewed
Jan 12, 2023
dev-rinchin
reviewed
Jan 12, 2023
dev-rinchin
reviewed
Jan 12, 2023
dev-rinchin
reviewed
Jan 12, 2023
|
btw, I would like you to add some unit tests for groupby features |
|
Stale pull request message |
dev-rinchin
reviewed
Mar 28, 2023
dev-rinchin
reviewed
Mar 28, 2023
examples/demo14.py
Outdated
|
|
||
| train, test = train_test_split(data, test_size=2000, random_state=42) | ||
|
|
||
|
|
There was a problem hiding this comment.
remove redundant empty lines
dev-rinchin
reviewed
Mar 28, 2023
examples/demo14.py
Outdated
| general_params={"use_algos": [["lgb"]]}, | ||
| gbm_pipeline_params={"use_groupby": True, "groupby_triplets": groupby_triplets}, | ||
| ) | ||
| _ = automl.fit_predict(train, roles=roles) |
There was a problem hiding this comment.
automl.fit_predict(train, roles=roles)
dev-rinchin
reviewed
Apr 6, 2023
Co-authored-by: Rinchin <57899558+dev-rinchin@users.noreply.github.com>
dev-rinchin
reviewed
Apr 27, 2023
examples/demo14.py
Outdated
| gbm_pipeline_params={"use_groupby": True, "groupby_triplets": groupby_triplets}, | ||
| ) | ||
| automl.fit_predict(train, roles=roles) | ||
|
|
There was a problem hiding this comment.
add comment for feature_scores
examples/demo14.py
Outdated
|
|
||
| # Custom pipeline with groupby features defined by importance | ||
| print("\nTry custom pipeline with groupby features defined by importance:\n") | ||
|
|
There was a problem hiding this comment.
add comment for custom pipeline
examples/demo14.py
Outdated
|
|
||
|
|
||
| pipe = LGBAdvancedPipeline( | ||
| use_groupby=True, pre_selector=selector, groupby_types=["delta_median", "std"], groupby_top_based_on="importance" |
| @@ -429,7 +429,7 @@ def get_gbms( | |||
| pre_selector: Optional[SelectionPipeline] = None, | |||
| ): | |||
|
|
|||
| gbm_feats = LGBAdvancedPipeline(**self.gbm_pipeline_params) | |||
| gbm_feats = LGBAdvancedPipeline(**self.gbm_pipeline_params, feats_imp=pre_selector) | |||
There was a problem hiding this comment.
add pre_selector to linear_l2_feats init in get_linear
| @@ -348,7 +348,7 @@ def get_gbms( | |||
| ): | |||
|
|
|||
| text_gbm_feats = self.get_nlp_pipe(self.gbm_pipeline_params["text_features"]) | |||
| gbm_feats = LGBAdvancedPipeline(output_categories=False, **self.gbm_pipeline_params) | |||
| gbm_feats = LGBAdvancedPipeline(feats_imp=pre_selector, output_categories=False, **self.gbm_pipeline_params) | |||
There was a problem hiding this comment.
do we need eats_imp=pre_selector in get_linear?
| group_col: str, | ||
| numeric_cols: Optional[List[str]] = None, | ||
| categorical_cols: Optional[List[str]] = None, | ||
| used_transforms: Optional[List[str]] = None, |
| self._features = [f"{self._fname_prefix}__{self.group_col}__{t}__{f}" for f, t in self.transformations_list] | ||
| self._features_mapping = {self._features[i]: k for i, k in enumerate(self.transformations_list)} | ||
|
|
||
| self._group_ids_dict = self._calculate_group_ids(dataset) |
|
|
||
| def _set_feature_indices(self): | ||
| feat_idx = dict() | ||
| feat_idx[self.group_col] = 0 |
There was a problem hiding this comment.
do not hardcode values
There was a problem hiding this comment.
The thing is that we don't have the original feature names in there, because they are affected by different encoders. So that, hardcoding idx is the only way to link them with original names. To fix this we need some general improvements: save original feature names through one-to-one transformations.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Groupby features introduced: delta mean, delta median, min, max, std, cat mode. Config files updated, the features are not used by default.