When I try to combine two or more selection transformers(by sklearn pipeline) with the variables parameter.
Due to the first selection transformer already dropping some features, I would encounter KeyError.
For example:
I want to combine DropHighPSIFeatures(split by 'position') and DropHighPSIFeatures(split by 'date') selection transformer.
I don't want 'date' and 'position' to compute by the other transformer, so I need to set variables for them.
Things are doing well when fitting the first transformer but get KeyError when fitting the second transformer.
I have two ideas:
-
Add an 'ignore_variables_not_exist'(or another name) bool parameter that can automatically
change the variables parameter to the intersection of variables parameter and columns of input data.
-
Add an 'exclude'(or another name) bool parameter that makes the variables parameter be excluded.
Not sure which one is best or if this feature is needed. I will start working on it if you think this is needed.
When I try to combine two or more selection transformers(by sklearn pipeline) with the variables parameter.
Due to the first selection transformer already dropping some features, I would encounter KeyError.
For example:
I want to combine DropHighPSIFeatures(split by 'position') and DropHighPSIFeatures(split by 'date') selection transformer.
I don't want 'date' and 'position' to compute by the other transformer, so I need to set variables for them.
Things are doing well when fitting the first transformer but get KeyError when fitting the second transformer.
I have two ideas:
Add an 'ignore_variables_not_exist'(or another name) bool parameter that can automatically
change the variables parameter to the intersection of variables parameter and columns of input data.
Add an 'exclude'(or another name) bool parameter that makes the variables parameter be excluded.
Not sure which one is best or if this feature is needed. I will start working on it if you think this is needed.