Swap CUDA 10.1 and CPU CI for windows

[janeyx99]

This change temporarily disables CUDA testing on PRs, but keeps it on master.
This is likely to increase the number of reverts, but this is necessary as a stop-gap measure to cap the CI costs growth.

[facebook-github-bot]

💊 CI failures summary and remediations

As of commit bf65dd8 (more details on the Dr. CI page):

---

💚 💚 Looks good so far! There are no failures yet. 💚 💚

---

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

[facebook-github-bot]

@janeyx99 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

[facebook-github-bot]

@janeyx99 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

[ngimel]

Hm, given how many cuda-related PRs and especially builds are broken on windows only, I'm afraid it will increase churn a lot.

[zasdfgbnm]

Does this mean no CI will be running for CUDA+Windows for PR? I feel worried about it as in the past Windows CI has been useful. (I saw Windows only failures ~twice last month)

[ptrblck]

This change temporarily disables CUDA testing on PRs, but keeps it on master.

If I understand it correctly, Windows CI would only be triggered after the PR was merged into the master branch?
How temporary would it be, as this could indeed increase the churn of reverting faulty PRs a lot, as already mentioned, and could additionally make fixes to PRs hard (they won't be tested before the re-merge of the "fixed PR" in CI).

[seemethere]

I think based on our current cost for running Windows gpu executors it's just not tenable to run Windows CUDA tests on every PR.

After this PR lands I'll do a follow up PR to add a CI label that we can use to trigger windows CUDA testing on PRs if requested.

It'll be similar to #54018

[facebook-github-bot]

@janeyx99 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

[facebook-github-bot]

@janeyx99 merged this pull request in 1d3a9bf.

[imaginary-person]

If failures on Windows CUDA checks do not usually result from the fact that Windows CUDA CI checks use a Tesla T4 GPU (SM_75), whereas Linux CUDA CI checks use an SM_52 GPU, how about also having a (regular) Windows CI check with an SM_52 GPU, if an executor with an SM_52 GPU is significantly cheaper than an executor with an SM_75 GPU?

In that case, @seemethere can make two labels for Windows CUDA CI checks - one each for executors with an SM_52 & an SM_75 GPU respectively, so that one can choose the SM_52 one (cheaper), if the difference in CUDA Compute Capability would be deemed to not be a source of potential failures for an in-progress PR.