add test cases for GradScaler on CPU

[CaoE]

Stack from ghstack (oldest at bottom):

• -> add test cases for GradScaler on CPU #109994

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/109994

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit c2b2fec with merge base a43c283 ():

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / linux-jammy-py3-clang12-executorch / test (executorch, 1, 1, linux.2xlarge) (gh)
  Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[ezyang]

You should seriously just consider making a dedicated test file for this though

[CaoE]

Do you mean making a dedicated test file like test_grad_scaler.py for all GradScaler related tests above ? It's okay for me. I will move such tests to a new file test_grad_scaler.py.

[ezyang]

Any code changes besides changing device? (A diff perhaps?)

[CaoE]

No, only the device change.
The simple code would also fail (will also fail on CUDA with device="cuda")：
torch._dynamo.exc.TorchRuntimeError: Failed running call_function <built-in method sparse_coo_tensor of type object at 0x7f105afdb9a0>(*(FakeTensor(..., size=(2, 3), dtype=torch.int64), FakeTensor(..., size=(3,)), (2, 3)), **{'device': 'cpu', 'dtype': torch.float32}): The tensor has a non-zero number of elements, but its data is not allocated yet. Caffe2 uses a lazy allocation, so you will need to call mutable_data() or raw_mutable_data() to actually allocate memory.

@onlyNativeDeviceTypes
@dtypes(torch.float)
def test_grad_scaling_unscale_sparse(self, device, dtype):
    i = torch.tensor([[0, 1, 1],
                      [2, 0, 2]], device="cpu", dtype=torch.int64)
    v = torch.tensor([16., 32., 64.], device="cpu", dtype=torch.float)
    s = torch.sparse_coo_tensor(i, v, torch.Size([2, 3]), device="cpu", dtype=torch.float)

[ezyang]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here