Skip to content

Use CDN for ci-caches on download #72027

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 12, 2020
Merged

Use CDN for ci-caches on download #72027

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
54b7d45
Use CDN for ci-caches on download
Mark-Simulacrum May 8, 2020
9a4e718
Configure cache domain for GHA
Mark-Simulacrum May 11, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ jobs:
CI_JOB_NAME: "${{ matrix.name }}"
SCCACHE_BUCKET: rust-lang-gha-caches
TOOLSTATE_REPO: "https://github.com/pietroalbini/rust-toolstate"
CACHE_DOMAIN: ci-caches-gha.rust-lang.org
if: "github.event_name == 'pull_request'"
strategy:
matrix:
Expand Down Expand Up @@ -146,6 +147,7 @@ jobs:
TOOLSTATE_PUBLISH: 1
CACHES_AWS_ACCESS_KEY_ID: AKIA46X5W6CZOMUQATD5
ARTIFACTS_AWS_ACCESS_KEY_ID: AKIA46X5W6CZH5AYXDVF
CACHE_DOMAIN: ci-caches-gha.rust-lang.org
if: "github.event_name == 'push' && github.ref == 'refs/heads/try' && github.repository == 'rust-lang-ci/rust'"
strategy:
matrix:
Expand Down Expand Up @@ -255,6 +257,7 @@ jobs:
TOOLSTATE_PUBLISH: 1
CACHES_AWS_ACCESS_KEY_ID: AKIA46X5W6CZOMUQATD5
ARTIFACTS_AWS_ACCESS_KEY_ID: AKIA46X5W6CZH5AYXDVF
CACHE_DOMAIN: ci-caches-gha.rust-lang.org
if: "github.event_name == 'push' && github.ref == 'refs/heads/auto' && github.repository == 'rust-lang-ci/rust'"
strategy:
matrix:
Expand Down Expand Up @@ -606,6 +609,7 @@ jobs:
TOOLSTATE_PUBLISH: 1
CACHES_AWS_ACCESS_KEY_ID: AKIA46X5W6CZOMUQATD5
ARTIFACTS_AWS_ACCESS_KEY_ID: AKIA46X5W6CZH5AYXDVF
CACHE_DOMAIN: ci-caches-gha.rust-lang.org
if: "github.event_name == 'push' && github.ref == 'refs/heads/master' && github.repository == 'rust-lang-ci/rust'"
steps:
- name: checkout the source code
Expand Down
10 changes: 6 additions & 4 deletions src/ci/docker/run.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ dist=$objdir/build/dist

source "$ci_dir/shared.sh"

CACHE_DOMAIN="${CACHE_DOMAIN:-ci-caches.rust-lang.org}"

if [ -f "$docker_dir/$image/Dockerfile" ]; then
if [ "$CI" != "" ]; then
hash_key=/tmp/.docker-hash-key.txt
Expand All @@ -38,9 +40,7 @@ if [ -f "$docker_dir/$image/Dockerfile" ]; then
cksum=$(sha512sum $hash_key | \
awk '{print $1}')

s3url="s3://$SCCACHE_BUCKET/docker/$cksum"
url="https://$SCCACHE_BUCKET.s3.amazonaws.com/docker/$cksum"
upload="aws s3 cp - $s3url"
url="https://$CACHE_DOMAIN/docker/$cksum"

echo "Attempting to download $url"
rm -f /tmp/rustci_docker_cache
Expand All @@ -65,7 +65,9 @@ if [ -f "$docker_dir/$image/Dockerfile" ]; then
-f "$dockerfile" \
"$context"

if [ "$upload" != "" ]; then
if [ "$CI" != "" ]; then
s3url="s3://$SCCACHE_BUCKET/docker/$cksum"
upload="aws s3 cp - $s3url"
digest=$(docker inspect rust-ci --format '{{.Id}}')
echo "Built container $digest"
if ! grep -q "$digest" <(echo "$loaded_images"); then
Expand Down
2 changes: 2 additions & 0 deletions src/ci/github-actions/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ x--expand-yaml-anchors--remove:
- &public-variables
SCCACHE_BUCKET: rust-lang-gha-caches
TOOLSTATE_REPO: https://github.com/pietroalbini/rust-toolstate
CACHE_DOMAIN: ci-caches-gha.rust-lang.org

- &prod-variables
SCCACHE_BUCKET: rust-lang-gha-caches
Expand All @@ -51,6 +52,7 @@ x--expand-yaml-anchors--remove:
# (caches, artifacts...).
CACHES_AWS_ACCESS_KEY_ID: AKIA46X5W6CZOMUQATD5
ARTIFACTS_AWS_ACCESS_KEY_ID: AKIA46X5W6CZH5AYXDVF
CACHE_DOMAIN: ci-caches-gha.rust-lang.org

- &base-job
env: {}
Expand Down