Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: - MuFidelity does not work for Tabular data and time series #151

Closed
AntoninPoche opened this issue Dec 8, 2023 · 0 comments · Fixed by #152
Closed

[Bug]: - MuFidelity does not work for Tabular data and time series #151

AntoninPoche opened this issue Dec 8, 2023 · 0 comments · Fixed by #152
Assignees
Labels
bug Something isn't working

Comments

@AntoninPoche
Copy link
Collaborator

AntoninPoche commented Dec 8, 2023

Module

None

Current Behavior

When applying MuFidelity on tabular data or time series, the number of features set to 0 with the mask does not correspond to subset_percent. Indeed, the formula to compute the subset_size depends on grid_size ** 2 which is adapted to images.

Therefore, the number of features set to zero is much higher than expected, all features may be set to zero if grid_size is big. For tabular data or time series, the default value corresponds to the number of features, which is too big and results in fidelity scores of only zeros.

Expected Behavior

Remove the attribute self.subset_size line 91.

Then in lines 201, 208, and 220:
Replace: subset_masks = tf.argsort(subset_masks, axis=-1) > self.subset_size
By: subset_masks = subset_masks > self.subset_percent
!!! See the direction of the > or < depending on the meaning of subset percent.

Version

1.3.1

Environment

No response

Relevant log output

No response

To Reproduce

See Attributions: Time Series and Regression tutorial, in the metric part, for MuFidelity, set subset_percent to 0.5 (perturb half) and grid_size to 48 (treat each time step differently).

It will return only zeros, but the configuration is far from absurd.

@AntoninPoche AntoninPoche added the bug Something isn't working label Dec 8, 2023
@AntoninPoche AntoninPoche self-assigned this Dec 12, 2023
@AntoninPoche AntoninPoche linked a pull request Dec 12, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant