dist.yml: Download optional/experimental tarballs for GitHub Release assets

[mkoeppe]

• Fixes GH Actions: Also put optional tarballs in the GH release assets #37752, at least for the non-"huge" tarballs (we skip the "huge" tarballs here).
• More options for sage -package download (a subset of what sage -package list supports)
• Also remove some zombie package files

Test run: https://github.com/mkoeppe/sage/actions/workflows/dist.yml

📝 Checklist

• The title is concise and informative.
• The description explains in detail what this PR is about.
• I have linked a relevant issue or discussion.
• I have created tests covering the changes.
• I have updated the documentation accordingly.

⌛ Dependencies

• Depends on GH Actions: Build platform-independent wheels of sagemath-environment, sage-setup, sage-sws2rst for PyPI #37099 (merged to resolve merge conflict)

[mkoeppe]

https://github.com/mkoeppe/sage/actions/runs/8585436817/job/23526820138#step:9:864

Total file count: 368 ---- Processed file #367 (99.7%)
[Error: ENOSPC: no space left on device, write] {
  errno: -28,
  code: 'ENOSPC',
  syscall: 'write'
}

[github-actions]

Documentation preview for this PR (built with commit a055882; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

[mkoeppe]

Removed the fix for the symengine_py upstream_url here; it's now on #37726.

[kwankyu]

Are these Release assets distributed from somewhere else or just from the github site?

Is it okay to bloat release assets? Some people may not prefer swollen tarballs with optional and experimental packages. I don't know as I never downloaded the tarballs for my use...

[mkoeppe]

Are these Release assets distributed from somewhere else or just from the github site?

All these files are also on our mirrors.

Making them available as release assets makes it faster and more robust for users to download the files because of the use of GitHub's CDN.

Is it okay to bloat release assets? Some people may not prefer swollen tarballs with optional and experimental packages. I don't know as I never downloaded the tarballs for my use...

This PR does not change what is in the large self-contained Sage release tarball. The tarball will still only contain the upstream tarballs of standard packages.

[mkoeppe]

Per https://docs.github.com/en/repositories/releasing-projects-on-github/about-releases#storage-and-bandwidth-quotas:

Each file included in a release must be under 2 GiB. There is no limit on the total size of a release, nor bandwidth usage.

[kwankyu]

Are these Release assets distributed from somewhere else or just from the github site?

All these files are also on our mirrors.

You mean that the files contained in the Release assets, not the Release assets themselves?

This PR just makes Release assets bigger by adding optional (not huge) and experimental packages. So if someone was using the Release assets, then he would experience longer download time after this PR. Can you estimate how much longer?

Anyway, I agree that the Release assets don't need to be the same with the Sage release tarball...

[mkoeppe]

On Fri, May 10, 2024 at 4:04 PM Kwankyu Lee ***@***.***> wrote: Are these Release assets distributed from somewhere else or just from the github site? All these files are also on our mirrors. You mean that the files contained in the Release assets, not the Release assets themselves?

"Release assets" are the files that you see listed in a GitHub release such as https://github.com/sagemath/sage/releases/tag/10.3

This PR just makes Release assets bigger by adding optional (not huge) and experimental packages.

It does not make any asset bigger; it just adds more of them.

So if someone was using the Release assets, then he would experience longer download time after this PR.

No.

[kwankyu]

Ah, I see. Thanks.

[kwankyu]

"Release assets" are the files that you see listed in a GitHub release such as https://github.com/sagemath/sage/releases/tag/10.3

I was only looking at release_dist artifact. Nice to see the Release assets :-)

LGTM.

[mkoeppe]

Thank you!