[bug] slow download speed / max download speed (GH purposely limiting in some cases)?

[nmichlo]

What happened?

Slow download speed on v4 of the action.

1Gb file took 12 minutes to download on a 500mbps line.

What did you expect to happen?

This should take ~20s at most.

How can we reproduce it?

Upload a large file and download it again.

Anything else we need to know?

self-hosted macOS runner.

Downloads used to be faster.

• manually checked download speed and it is at 500mbps
• re-ran download action a few times to make sure

What version of the action are you using?

v4

What are your runner environments?

self-hosted

Are you on GitHub Enterprise Server? If so, what version?

No response

[rwader-swi]

Lots of people are facing the same issue. Ironically v3 is faster but its gonna be deprecated and support will be completely removed. Can you all please fix the v4 issues before deprecating v3? @bethanyj28

[rasharab]

Hit this issue today.
5GB file took over 20 minutes to download.
Ironically, took 40 seconds to upload using v4 of upload-artifact.
We're thinking of abandoning using these actions as they're just too non-performant. (moving to s3)

[nmichlo]

Moving to S3/alternative seems like it will be the most reliable solution.

We spent considerable time consolidating on a single system for handling code review and CI/CD, but GitHub so far is proving to be more of a hinderance than it is actually useful. Would recommend not using GitHub in the first place with all the issues and lack of support.

[gmankab]

same issue

[nmichlo]

From what I can tell

• The download speed limits seem to apply when a workflow is triggered automatically instead of from a personal access token.

  • seems like this change was intentional from GH side and was just not well documented or communicated...
  • e.g. on: {pull_request: {types: [synchronize]}}

• As soon as a personal access token triggers the workflow then download speed is no longer limited to 1MBPS

  • e.g. automated version bump pushes tag using PAT, and another action listens for on: {push: {tags: ["*"]}} from this.

---

As a HACKY workaround, if you are not wanting to use a PAT to trigger or you don't want to switch to S3.

Then it's possible to use tools like aria2 to speed up the download.
Although there are a number of steps to this, and it probably could break in future.

1. find artifacts using GitHub script (won't go into details here) OR easier pass urls from outputs of actions/upload-artifact to your download step e.g. steps.upload.outputs.artifact-url
2. resolve the URL which requires authentication to a temporary unathenticated download URL with:
   final_url="$(curl -L -I -s -H "Authorization: token ${GITHUB_TOKEN}" -o /dev/null -w "%{url_effective}" "$url")"
3. download this URL with aria2:
   aria2c -x 16 -s 16 -j 1 -k 1M "${final_url}"
4. now downloading at 1MBPS per split/connection, so ~16MBPS