Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistence in attacking returns #644

Open
yitao416 opened this issue Sep 14, 2021 · 5 comments
Open

Inconsistence in attacking returns #644

yitao416 opened this issue Sep 14, 2021 · 5 comments
Assignees

Comments

@yitao416
Copy link

The very example at your official website is taking the first return as advs.

image

Well, the latest attack method suggests using the second return as the better option (clipped).
image

I think it is better to make this more clear for users.

Thanks.

@jangop
Copy link

jangop commented Jan 25, 2022

@yitao416 Which “latest attack method” are you referring to?

I agree that returning adversarials and clipped_adversarials can be confusing, especially seeing that the differences aren't documented, and one might assume that the clipping refers to the bounds specified when initializing fmodel.

Furthermore, when epsilons=None, maybe clipped_adversarials (the second return value) should be None as well, since no clipping takes place. I do not like mixing return types (even Optional[Tensor]), but I suppose that's the price to pay for always returning both variants.

@yitao416
Copy link
Author

@yitao416 Which “latest attack method” are you referring to?

I agree that returning adversarials and clipped_adversarials can be confusing, especially seeing that the differences aren't documented, and one might assume that the clipping refers to the bounds specified when initializing fmodel.

Furthermore, when epsilons=None, maybe clipped_adversarials (the second return value) should be None as well, since no clipping takes place. I do not like mixing return types (even Optional[Tensor]), but I suppose that's the price to pay for always returning both variants.

The latest attack method I refer to is version 3.3.1.

The second picture is from the Example section from the Github page: https://github.com/bethgelab/foolbox

@zimmerrol
Copy link
Member

You are right that we should put this in the documentation.
@jangop If you have the time and already an idea how to implement your suggested changes please feel free to open a PR for it:)

@zimmerrol zimmerrol self-assigned this Jan 27, 2022
@jangop
Copy link

jangop commented Jan 28, 2022

You are right that we should put this in the documentation. @jangop If you have the time and already an idea how to implement your suggested changes please feel free to open a PR for it:)

Sure. However, what would be the desired outcome? What was/is the rationale behind always returning both unclipped and clipped adversarials? If a user knows which one they want, they could probably take care of the clipping themselves.

I still believe it useful to return None if epsilons=None, because in that case, clipping is not just the identity but meaningless. That part is simple to implement, but it might break around 50 % of existing code.

@zimmerrol
Copy link
Member

I would just suggest updating the documentation to include this information.

@zimmerrol zimmerrol reopened this Oct 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants