Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Readme.MD Usage typo issue #86

Closed
fealty94 opened this issue Apr 13, 2023 · 1 comment
Closed

Readme.MD Usage typo issue #86

fealty94 opened this issue Apr 13, 2023 · 1 comment
Labels

Comments

@fealty94
Copy link

fealty94 commented Apr 13, 2023

In Readme.MD, there is typo in Usage.
Below is the code.

Usage

from sam import SAM

model = YourModel()
base_optimizer = torch.optim.SGD  # define an optimizer for the "sharpness-aware" update
optimizer = SAM(model.parameters(), base_optimizer, lr=0.1, momentum=0.9)

for input, output in data:
  loss = loss_function(output, model(input))  # use this loss for any training statistics
  loss.backward()
  optimizer.first_step(zero_grad=True)
  
  loss_function(output, model(input)).backward()  # make sure to do a full forward pass
  optimizer.second_step(zero_grad=True)

I think the arguments of loss_function have to be reordered.
First argument of loss_function in pytorch might be output of the model,
Second argument of loss_function in pytorch might be "label".

so I think loss_function(model(input),output) will be the right answer.

Some people like me maybe confused with this order.

Thanks for sharing your great code.

@stale
Copy link

stale bot commented May 9, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label May 9, 2023
@stale stale bot closed this as completed May 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant