save grad_scaler if use amp for better resume

[xiaohu2015]

The pr is about #4509
when use amp, it is better to save/resuem scaler state_dict https://pytorch.org/tutorials/recipes/recipes/amp_recipe.html#saving-resuming

cc @datumbox

[facebook-github-bot]

💊 CI failures summary and remediations

As of commit 42c1c4f (more details on the Dr. CI page):

---

• 1/1 failures introduced in this PR

---

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

CodeQL / build (1/1)

Step: "Perform CodeQL Analysis" (full log | diagnosis details | 🔁 rerun)

2021-11-12T09:56:10.6351445Z ##[error]source repository for pull request could not be determined

2021-11-12T09:56:09.8303807Z TRAP files cleaned up (3ms).
2021-11-12T09:56:09.8308131Z Cleaning up scratch directory...
2021-11-12T09:56:09.8326612Z Scratch directory cleaned up (0ms).
2021-11-12T09:56:09.8327347Z Cleaning up temporary directory...
2021-11-12T09:56:09.8327943Z Temporary directory cleaned up (0ms).
2021-11-12T09:56:09.9465461Z ##[endgroup]
2021-11-12T09:56:09.9465789Z 
2021-11-12T09:56:09.9701760Z ##[group]Uploading results
2021-11-12T09:56:09.9702710Z Processing sarif files: ["/home/runner/work/vision/results/cpp.sarif","/home/runner/work/vision/results/python.sarif"]
2021-11-12T09:56:10.2983063Z Uploading results
2021-11-12T09:56:10.6351445Z ##[error]source repository for pull request could not be determined
2021-11-12T09:56:10.6383729Z RequestError [HttpError]: source repository for pull request could not be determined
2021-11-12T09:56:10.6385698Z     at /home/runner/work/_actions/github/codeql-action/v1/node_modules/@octokit/request/dist-node/index.js:66:23
2021-11-12T09:56:10.6386590Z     at runMicrotasks (<anonymous>)
2021-11-12T09:56:10.6387457Z     at processTicksAndRejections (internal/process/task_queues.js:93:5)
2021-11-12T09:56:10.6388883Z     at async Job.doExecute (/home/runner/work/_actions/github/codeql-action/v1/node_modules/bottleneck/light.js:405:18) {
2021-11-12T09:56:10.6389869Z   name: 'HttpError',
2021-11-12T09:56:10.6390281Z   status: 422,
2021-11-12T09:56:10.6390675Z   headers: {
2021-11-12T09:56:10.6391439Z     'access-control-allow-origin': '*',
2021-11-12T09:56:10.6394225Z     'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset',

---

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

[datumbox]

LGTM, thanks!