Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow cache retrieval speeds on the self-hosted runners #1143

Open
igorwojda opened this issue Mar 29, 2023 · 3 comments
Open

Slow cache retrieval speeds on the self-hosted runners #1143

igorwojda opened this issue Mar 29, 2023 · 3 comments

Comments

@igorwojda
Copy link

igorwojda commented Mar 29, 2023

In my setup the cache retrieval step is the longest step for each action:

image

It takes around 3-4 minutes to download and extract a 515MB cache file:

image

Host OS
Ubuntu 22

Host Internet connection speed:
This file wget http://speedtest.novoserve.com/files/10GB.bin download speed is 40-55MB/s

Host CPU:
Intel(R) Xeon(R) Silver 4310 CPU @ 2.10GHz (12 cores)

Host I/O
hdparm -Tt /dev/sda

  • Timing cached reads: 9070 MB/s
  • Timing buffered disk reads: 1600MB/s

d if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=dsync

  • Write: 937 MB/s

Config

      - uses: actions/cache@v3
        with:
          path: ~/.gradle
          key: gradle-${{ hashFiles('gradle-checksums.txt') }}-${{ hashFiles('gradle/wrapper/gradle-wrapper.properties') }}
          restore-keys: |
            gradle-${{ hashFiles('gradle-checksums.txt') }}-

Questions

  • Is there any way to speed up the network cache retrieval?
  • Is there a way to configure a local disk cache for the self-hosted runners? (ideally shared between runners)
@igorwojda
Copy link
Author

FIY: It looks like all runners are reusing Gradle cache (/home/ubuntu/.gradle), so for my setup actions/cache is not required at all (still it would be nice to have the ability to utilize local cache).

@amadou-6e
Copy link

Has a local cache been added by now. Build and security test take me 6 mins upload and download take me 50min. This is unnecessary as I want to have my cache on my runner and not somewhere in the stars

@redvinaa
Copy link

redvinaa commented Dec 2, 2024

+1
@igorwojda please consider reopening this, as the questions proposed are still very valid

@igorwojda igorwojda reopened this Dec 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants