You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, we could add multiple duplicate components from the same actions. We should try to prune duplicate components, if possible:
# current behavior. But this might cause errors, trying to drop 'some col' multiple times
actions = [DataCheckAction(DataCheckActionCode.DROP_COL, {"columns": ['some col']}),
DataCheckAction(DataCheckActionCode.DROP_COL, {"columns": ['some col']})]
_make_component_list_from_actions(actions) == [DropColumns(columns=['some col']),
DropColumns(columns=['some col'])]
# desired behavior
actions = [DataCheckAction(DataCheckActionCode.DROP_COL, {"columns": ['some col']}),
DataCheckAction(DataCheckActionCode.DROP_COL, {"columns": ['some col']})]
_make_component_list_from_actions(actions) == [DropColumns(columns=['some col'])] # only once
There's some support in the DataChecks class to not return duplicate actions, but if a user just used multiple DataCheck objects, we will still end up with duplicates in _make_component_list_from_actions
The text was updated successfully, but these errors were encountered:
dsherry
changed the title
Update _make_component_list_from_actions to not produce duplicate components
Data check actions: don't produce duplicate components
Mar 25, 2021
This is not blocked on #1929 -- we can lump the dropped cols into one DropColumns component. However, it may make more sense if each action generates its own component, which we'd need to do #1929 for in order to put multiple DropColumns into a pipeline.
Currently, we could add multiple duplicate components from the same actions. We should try to prune duplicate components, if possible:
There's some support in the
DataChecks
class to not return duplicate actions, but if a user just used multiple DataCheck objects, we will still end up with duplicates in_make_component_list_from_actions
The text was updated successfully, but these errors were encountered: