Add libtorch nightly build for CUDA 12.8

[tinglvv]

Try removing sm50 and sm60 to shrink binary size, and resolve the ld --relink error

"Architecture support for Maxwell, Pascal, and Volta is considered feature-complete and will be frozen in an upcoming release." from 12.8 release note.

Also updating the runner for cuda 12.8 test to g4dn (T4, sm75) due to the drop in sm50/60 support.

#145570

cc @atalman @malfet @ptrblck @msaroufim @eqy @nWEIdia

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/146265

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (2 Unrelated Failures)

As of commit 62ef609 with merge base 16e202a ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• linux-binary-manywheel / manywheel-py3_12-cuda11_8-test / test (gh) (detected as infra flaky with no log or failing log classifier)
• linux-binary-manywheel / manywheel-py3_13t-xpu-test (gh) (detected as infra flaky with no log or failing log classifier)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[tinglvv]

Test failures in libtorch and manywheel with CUDA Error:
Tesla M60 with CUDA capability sm_52 is not compatible with the current PyTorch installation. The current PyTorch install supports CUDA capabilities sm_70 sm_75 sm_80 sm_86 sm_90 sm_100 sm_120 compute_120.

Reason is we are removing sm50 and sm60 from 12.8 binary in this PR, to resolve the ld --relink error in #145792 (comment).

And current upstream CI test runs on Tesla M60 which is sm_50.
See runs_on: linux.4xlarge.nvidia.gpu --> instance_type: g3.4xlarge --> Tesla M60 (https://aws.amazon.com/blogs/aws/new-next-generation-gpu-powered-ec2-instances-g3/).

Proposing solution:
Use g4dn or g5 AWS runners to run the test for 12.8, g4dn is T4 GPU Turing (sm_75) and g5 is Ampere (sm_80). If we were to deprecate Volta (sm_70) in 12.8 as well, then need to get a g5 runner.
Runner choices https://github.com/tinglvv/test-infra/blob/main/.github/scale-config.yml#L101

cc @atalman @ptrblck

[Skylion007]

Can we not use the linker script with --relink keep the old arch support?

[tinglvv]

Can we not use the linker script with --relink keep the old arch support?

Hi @Skylion007 , right the --relink would work. Meanwhile, we are actually also deprecating the sm_50,60,70 for cuda 12.8 (they will be deprecated officially in future cuda releases), and this would resolve the build error.

[Skylion007]

This will drop support for 1080 and similar consumer chips, right? We are finally starting to drop GPU arches that are commonly used and can run modern architectures in inference. These are very common in university clusters.

SM70 only supports GV100s right? Why not support SM60 so torch supports more devices?

Is 12.9 dropping all these cuda arches completely? Or is this just to unblock the binary size issues? Seems like there might be a longer term alternative to fixing the 1GB libtorch limit such getting the linker script to work with relink, LTO might reduce binary size sufficiently to save one of the arch's, or just splitting the binaries.

[ptrblck]

@Skylion007 Future CUDA versions will drop sm_50-sm_70 completely as @tinglvv explained and CUDA 12.8 deprecated these.

We are finally starting to drop GPU arches that are commonly used and can run modern architectures in inference. These are very common in university clusters.

Universities and other users stuck on older GPUs or drivers are still able to use PyTorch binaries built with an older CUDA toolkit (e.g. 12.6.3 or 11.8). We are keeping PyTorch binaries with CUDA 11.8 alive for 2+ years for this reason.

[tinglvv]

Fix for the build failures - use g4dn runners for 12.8 binary testing. (sm_75)
Hi @atalman , could you help update the runners? Thanks!

[tinglvv]

@pytorchbot rebase

[pytorchmergebot]

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[pytorchmergebot]

Successfully rebased cu128-libtorch-build onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout cu128-libtorch-build && git pull --rebase)

[atalman]

lgtm

[tinglvv]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here