Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to use in TF2 #89

Open
Shubhajit1988 opened this issue May 16, 2022 · 5 comments
Open

Unable to use in TF2 #89

Shubhajit1988 opened this issue May 16, 2022 · 5 comments

Comments

@Shubhajit1988
Copy link

Dear sir,
When I call 'adversary.run_standard_evaluation', it shows 'ModelAdapter' object is not callable

@fra31
Copy link
Owner

fra31 commented May 17, 2022

Hi,

it's a bit hard to say with little context, but I guess it might be because the flag is_tf_model=True is not set (see here).

@isaacwasserman
Copy link

I can confirm this behavior. In each of the attacks, model(x) is called. I believe this should be model.predict(x) instead.

@fra31
Copy link
Owner

fra31 commented Jun 27, 2022

Hi,

could you please point to the exact points where this happens? I think there should always be an if statement to distinguish whether the model is a TF one and then call model.predict.

@isaacwasserman
Copy link

It happens in checks.py:

def check_dynamic(model, x, is_tf_model=False, logger=None):
    if is_tf_model:
        msg = 'the check for dynamic defenses is not currently supported'
    else:
        msg = None
    sys.settrace(tracefunc)
    model(x)
    sys.settrace(None)
    #for k, v in funcs.items():
    #    print(k, v)
    if any([c > 0 for c in funcs.values()]):
        msg = 'it seems to be a dynamic defense! The evaluation' + \
            ' with AutoAttack might be insufficient.' + \
            f' See {checks_doc_path} for details.'
    if not msg is None:
        if logger is None:
            warnings.warn(Warning(msg))
        else:
            logger.log(f'Warning: {msg}')
    #sys.settrace(None)

@fra31
Copy link
Owner

fra31 commented Jul 30, 2022

Sorry for the late reply. This #92 should fix it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants