Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI Updates #1668

Merged
merged 5 commits into from
Feb 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
87 changes: 55 additions & 32 deletions .github/workflows/build_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,7 @@ on:
jobs:
build-test-cyclus:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
pull-requests: write

strategy:
fail-fast: false
matrix:
Expand All @@ -39,16 +36,17 @@ jobs:
conda,
]

steps:
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
services:
registry:
image: registry:2
ports:
- 5000:5000

steps:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
driver-opts: network=host

- name: Checkout Cyclus
uses: actions/checkout@v4
Expand All @@ -59,8 +57,7 @@ jobs:
with:
file: docker/Dockerfile
cache-from: type=registry,ref=ghcr.io/cyclus/cyclus_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cyclus:ci-layer-cache
cache-to: type=registry,ref=ghcr.io/cyclus/cyclus_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cyclus:ci-layer-cache,mode=max
tags: ghcr.io/cyclus/cyclus_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cyclus:ci-image-cache
tags: localhost:5000/cyclus:local
push: true
build-args: |
pkg_mgr=${{ matrix.pkg_mgr }}
Expand All @@ -81,14 +78,14 @@ jobs:
with:
context: ${{ github.workspace }}/cycamore
file: ${{ github.workspace }}/cycamore/docker/Dockerfile
build-contexts: |
ghcr.io/cyclus/cyclus_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cyclus=docker-image://localhost:5000/cyclus:local
cache-from: type=registry,ref=ghcr.io/cyclus/cycamore_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cycamore:ci-layer-cache
cache-to: type=registry,ref=ghcr.io/cyclus/cycamore_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cycamore:ci-layer-cache,mode=max
tags: ghcr.io/cyclus/cycamore_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cycamore:ci-image-cache
tags: localhost:5000/cycamore:local
push: true
build-args: |
pkg_mgr=${{ matrix.pkg_mgr }}
ubuntu_version=${{ matrix.ubuntu_versions }}
cyclus_tag=ci-image-cache@${{ steps.build-cyclus.outputs.digest }}

- name: Checkout Cymetric
if: ${{ github.event_name == 'pull_request' && steps.build-cycamore.outcome == 'success' }}
Expand All @@ -105,25 +102,51 @@ jobs:
with:
context: ${{ github.workspace }}/cymetric
file: ${{ github.workspace }}/cymetric/docker/Dockerfile
build-contexts: |
ghcr.io/cyclus/cycamore_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cycamore=docker-image://localhost:5000/cycamore:local
cache-from: type=registry,ref=ghcr.io/cyclus/cymetric_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cymetric:ci-layer-cache
cache-to: type=registry,ref=ghcr.io/cyclus/cymetric_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cymetric:ci-layer-cache,mode=max
tags: ghcr.io/cyclus/cymetric_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cymetric:ci-image-cache
push: true
build-args: |
pkg_mgr=${{ matrix.pkg_mgr }}
ubuntu_version=${{ matrix.ubuntu_versions }}
cycamore_tag=ci-image-cache@${{ steps.build-cycamore.outputs.digest }}

- name: PR Comment
if: ${{ github.event_name == 'pull_request' }}
uses: thollander/actions-comment-pull-request@v2
- name: Export Environment Variables
if: github.event_name == 'pull_request'
run: |
echo "CYCAMORE_BUILD_STATUS=${{steps.build-cycamore.outcome == 'success' && '**Success** :white_check_mark:' ||
steps.build-cycamore.outcome == 'failure' && '**Failure** :x:' ||
'**Skipped due to upstream failure** :warning:'}}" >> "$GITHUB_ENV"

echo "CYMETRIC_BUILD_STATUS=${{steps.build-cymetric.outcome == 'success' && '**Success** :white_check_mark:' ||
steps.build-cymetric.outcome == 'failure' && '**Failure** :x:' ||
'**Skipped due to upstream failure** :warning:'}}" >> "$GITHUB_ENV"

echo "ARTIFACT_NAME=${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}" >> "$GITHUB_ENV"

- name: Construct Artifact
if: github.event_name == 'pull_request'
run: |
echo "
##### Build \`FROM cyclus_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cyclus\`
- Cycamore: ${{ env.CYCAMORE_BUILD_STATUS }}
- Cymetric: ${{ env.CYMETRIC_BUILD_STATUS }}" > ${{ env.ARTIFACT_NAME }}.txt

- name: Upload Artifact
if: github.event_name == 'pull_request'
uses: actions/upload-artifact@v4
with:
name: ${{ env.ARTIFACT_NAME }}
path: ${{ env.ARTIFACT_NAME }}.txt

upload-pr-number:
runs-on: ubuntu-latest
if: github.event_name == 'pull_request'
steps:
- name: Save PR number to file
run: |
echo "${{ github.event.number }}" > pr_number

- name: Upload artifact
uses: actions/upload-artifact@v4
with:
comment_tag: ${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}
message: |
## Downstream build statuses using cyclus_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}
- Cycamore: ${{steps.build-cycamore.outcome == 'success' && '*Success* :white_check_mark:' ||
steps.build-cycamore.outcome == 'failure' && '**Failure** :x:' ||
'**Skipped due to upstream failure** :warning:'}}
- Cymetric: ${{steps.build-cymetric.outcome == 'success' && '*Success* :white_check_mark:' ||
steps.build-cymetric.outcome == 'failure' && '**Failure** :x:' ||
'**Skipped due to upstream failure** :warning:'}}
name: pr_number
path: pr_number
7 changes: 5 additions & 2 deletions .github/workflows/code_coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ on:
- '.github/workflows/changelog_test.yml'
- 'doc/**'
- 'CHANGELOG.rst'
push:
branches:
- main

jobs:
build-and-test:
Expand All @@ -35,7 +38,7 @@ jobs:
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Checkout Cyclus
Expand All @@ -48,7 +51,7 @@ jobs:
uses: docker/build-push-action@v5
with:
cache-from: type=registry,ref=ghcr.io/cyclus/cyclus_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cyclus-coverage:ci-layer-cache
cache-to: type=registry,ref=ghcr.io/cyclus/cyclus_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cyclus-coverage:ci-layer-cache,mode=max
cache-to: type=registry,ref=ghcr.io/cyclus/cyclus_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cyclus-coverage:ci-layer-cache,mode=max,ignore-error=true
file: docker/Dockerfile
target: coverage-report
outputs: type=local,dest=.
Expand Down
32 changes: 32 additions & 0 deletions .github/workflows/pr_comment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: Comment on PR

on:
workflow_run:
workflows: ["Build/Test Cyclus"]
types:
- completed

jobs:
pr-comment:
runs-on: ubuntu-latest
if: github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.conclusion == 'success'
steps:
- name: Download artifacts
uses: actions/download-artifact@v4
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
run-id: ${{ github.event.workflow_run.id }}
merge-multiple: true

- name: Merge artifacts and get PR number
run: |
echo "### Downstream Build Status Report" > artifacts_merged.md
cat ./*.txt >> artifacts_merged.md
echo "PR_NUMBER=$(cat pr_number)" >> "$GITHUB_ENV"

- name: PR Comment
uses: thollander/actions-comment-pull-request@v2
with:
pr_number: ${{ env.PR_NUMBER }}
comment_tag: build_status_report
filePath: artifacts_merged.md
6 changes: 2 additions & 4 deletions .github/workflows/publish_latest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ jobs:
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Checkout Cyclus
Expand All @@ -53,10 +53,10 @@ jobs:
uses: docker/setup-buildx-action@v3

- name: Build, Test, and Push Cyclus
id: build-cyclus
uses: docker/build-push-action@v5
with:
cache-from: type=registry,ref=ghcr.io/cyclus/cyclus_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cyclus:ci-layer-cache
cache-to: type=registry,ref=ghcr.io/cyclus/cyclus_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cyclus:ci-layer-cache,mode=max
file: docker/Dockerfile
push: true
tags: ghcr.io/cyclus/cyclus_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cyclus:${{ env.tag }}
Expand All @@ -78,7 +78,6 @@ jobs:
context: ${{ github.workspace }}/cycamore
file: ${{ github.workspace }}/cycamore/docker/Dockerfile
cache-from: type=registry,ref=ghcr.io/cyclus/cycamore_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cycamore:ci-layer-cache
cache-to: type=registry,ref=ghcr.io/cyclus/cycamore_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cycamore:ci-layer-cache,mode=max
tags: ghcr.io/cyclus/cycamore_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cycamore:${{ env.tag }}
push: true
build-args: |
Expand All @@ -99,7 +98,6 @@ jobs:
context: ${{ github.workspace }}/cymetric
file: ${{ github.workspace }}/cymetric/docker/Dockerfile
cache-from: type=registry,ref=ghcr.io/cyclus/cymetric_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cymetric:ci-layer-cache
cache-to: type=registry,ref=ghcr.io/cyclus/cymetric_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cymetric:ci-layer-cache,mode=max
tags: ghcr.io/cyclus/cymetric_${{ matrix.ubuntu_versions }}_${{ matrix.pkg_mgr }}/cymetric:${{ env.tag }}
push: true
build-args: |
Expand Down
2 changes: 1 addition & 1 deletion CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Since last release
will fail unless update-alternatives has been used to point python at the
correct python3 version (#1558)
* build and test are now fown on githubAction in place or CircleCI (#1569)
* Have separate workflows for testing, publishing dependency images, and publishing release images (#1597, #1602, #1606, #1609, #1629, #1633, #1637)
* Have separate workflows for testing, publishing dependency images, and publishing release images (#1597, #1602, #1606, #1609, #1629, #1633, #1637, #1668)
* Add Ubuntu 20.04 to the list of supported platforms (#1605, #1608)
* Add random number generator (Mersenne Twister 19937, from boost) and the ability to set the seed in the simulation control block (#1599)
* Added code coverage reporting to GitHub workflows (#1616)
Expand Down