Skip to content

Add dual s3/r2 upload to torchaudio#4179

Merged
atalman merged 1 commit intomainfrom
revert-4144-dual_upload_remove
Feb 19, 2026
Merged

Add dual s3/r2 upload to torchaudio#4179
atalman merged 1 commit intomainfrom
revert-4144-dual_upload_remove

Conversation

@atalman
Copy link
Copy Markdown
Contributor

@atalman atalman commented Feb 19, 2026

This brings back dual upload

@atalman atalman requested a review from a team as a code owner February 19, 2026 15:03
@pytorch-bot
Copy link
Copy Markdown

pytorch-bot Bot commented Feb 19, 2026

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/audio/4179

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@atalman atalman merged commit 11ed357 into main Feb 19, 2026
44 of 45 checks passed
@atalman atalman deleted the revert-4144-dual_upload_remove branch February 19, 2026 15:19
pytorchmergebot pushed a commit to pytorch/pytorch that referenced this pull request Feb 19, 2026
Similar to:
vision: pytorch/vision#9393
audio: pytorch/audio#4179

TLDR. Reason for using dual upload:

[Sippy](https://developers.cloudflare.com/r2/data-migration/sippy/) on demand Cloudflare migration is enabled on our R2 account.
I do see we are running installs from R2 successfully :[ https://github.com/meta-pytorch/torchcomms/actions/runs/21960287284/job/63435607477#step:12:53](https://github.com/meta-pytorch/torchcomms/actions/runs/21960287284/job/63435607477#step:12:53)

Log:
Collecting torch
  Downloading[ https://download-r2.pytorch.org/whl/nightly/cu129/torch-2.11.0.dev20260212%2Bcu129-cp314-cp314-manylinux_2_28_x86_64.whl.metadata](https://download-r2.pytorch.org/whl/nightly/cu129/torch-2.11.0.dev20260212%2Bcu129-cp314-cp314-manylinux_2_28_x86_64.whl.metadata) (29 kB)
Collecting filelock (from torch)

However when I naviagate to /whl/nightly/cu129 r2 folder in R2 account I don’t see any torch-2.11 files. Does it mean on every request files are being synced from AWS ? And hence either getting timeouts or 499 or 404 errors. From Sippy documentation once synced, files should be stored on our r2 folder.
Example of a failure: https://github.com/meta-pytorch/torchcomms/actions/runs/21960287284/job/63435607377

Pull Request resolved: #175352
Approved by: https://github.com/seemethere, https://github.com/malfet, https://github.com/zxiiro
norx1991 pushed a commit to pytorch/pytorch that referenced this pull request Feb 24, 2026
Similar to:
vision: pytorch/vision#9393
audio: pytorch/audio#4179

TLDR. Reason for using dual upload:

[Sippy](https://developers.cloudflare.com/r2/data-migration/sippy/) on demand Cloudflare migration is enabled on our R2 account.
I do see we are running installs from R2 successfully :[ https://github.com/meta-pytorch/torchcomms/actions/runs/21960287284/job/63435607477#step:12:53](https://github.com/meta-pytorch/torchcomms/actions/runs/21960287284/job/63435607477#step:12:53)

Log:
Collecting torch
  Downloading[ https://download-r2.pytorch.org/whl/nightly/cu129/torch-2.11.0.dev20260212%2Bcu129-cp314-cp314-manylinux_2_28_x86_64.whl.metadata](https://download-r2.pytorch.org/whl/nightly/cu129/torch-2.11.0.dev20260212%2Bcu129-cp314-cp314-manylinux_2_28_x86_64.whl.metadata) (29 kB)
Collecting filelock (from torch)

However when I naviagate to /whl/nightly/cu129 r2 folder in R2 account I don’t see any torch-2.11 files. Does it mean on every request files are being synced from AWS ? And hence either getting timeouts or 499 or 404 errors. From Sippy documentation once synced, files should be stored on our r2 folder.
Example of a failure: https://github.com/meta-pytorch/torchcomms/actions/runs/21960287284/job/63435607377

Pull Request resolved: #175352
Approved by: https://github.com/seemethere, https://github.com/malfet, https://github.com/zxiiro
EmanueleCoradin pushed a commit to EmanueleCoradin/pytorch that referenced this pull request Mar 30, 2026
Similar to:
vision: pytorch/vision#9393
audio: pytorch/audio#4179

TLDR. Reason for using dual upload:

[Sippy](https://developers.cloudflare.com/r2/data-migration/sippy/) on demand Cloudflare migration is enabled on our R2 account.
I do see we are running installs from R2 successfully :[ https://github.com/meta-pytorch/torchcomms/actions/runs/21960287284/job/63435607477#step:12:53](https://github.com/meta-pytorch/torchcomms/actions/runs/21960287284/job/63435607477#step:12:53)

Log:
Collecting torch
  Downloading[ https://download-r2.pytorch.org/whl/nightly/cu129/torch-2.11.0.dev20260212%2Bcu129-cp314-cp314-manylinux_2_28_x86_64.whl.metadata](https://download-r2.pytorch.org/whl/nightly/cu129/torch-2.11.0.dev20260212%2Bcu129-cp314-cp314-manylinux_2_28_x86_64.whl.metadata) (29 kB)
Collecting filelock (from torch)

However when I naviagate to /whl/nightly/cu129 r2 folder in R2 account I don’t see any torch-2.11 files. Does it mean on every request files are being synced from AWS ? And hence either getting timeouts or 499 or 404 errors. From Sippy documentation once synced, files should be stored on our r2 folder.
Example of a failure: https://github.com/meta-pytorch/torchcomms/actions/runs/21960287284/job/63435607377

Pull Request resolved: pytorch#175352
Approved by: https://github.com/seemethere, https://github.com/malfet, https://github.com/zxiiro
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants