CI #13
Workflow file for this run
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # DO NOT EDIT - GENERATED | |
| # This workflow is intended to be called by workflows in our various pathogen | |
| # build repos. See workflow-templates/pathogen-repo-ci.yaml (a "starter" | |
| # workflow) in this repo for an example of what the caller workflow looks like. | |
| name: CI | |
| defaults: | |
| run: | |
| # This is the same as GitHub Action's `bash` keyword as of 20 June 2023: | |
| # https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsshell | |
| # | |
| # Completely spelling it out here so that GitHub can't change it out from under us | |
| # and we don't have to refer to the docs to know the expected behavior. | |
| shell: bash --noprofile --norc -eo pipefail {0} | |
| on: | |
| workflow_call: | |
| inputs: | |
| repo: | |
| description: >- | |
| Repository name with owner (e.g. nextstrain/zika). Defaults to the repository of the caller workflow. | |
| type: string | |
| default: ${{ github.repository }} | |
| required: false | |
| ref: | |
| type: string | |
| default: "" | |
| required: false | |
| env: | |
| description: >- | |
| Additional environment variables to set before the build, as a string containing YAML. This is easily produced, for example, by pretending you're writing normal nested YAML within a literal multi-line block scalar (introduced by "|"): | |
| with: | |
| env: | | |
| FOO: bar | |
| I_CANT_BELIEVE: "it's not YAML" | |
| would_you_believe: | | |
| it's | |
| not | |
| yaml | |
| Do not use for secrets! Instead, pass them via GitHub Action's dedicated secrets mechanism. | |
| type: string | |
| default: "" | |
| required: false | |
| runtimes: | |
| description: >- | |
| List of Nextstrain runtimes under which to run the build, as a string containing YAML. This is easily produced, for example, by pretending you're writing normal nested YAML within a literal multi-line block scalar (introduced by "|"): | |
| with: | |
| runtimes: | | |
| - docker | |
| - conda | |
| Defaults to "docker" and "conda". One job per runtime will be run. | |
| type: string | |
| default: | | |
| - docker | |
| - conda | |
| required: false | |
| artifact-name: | |
| description: >- | |
| A base name to use for the uploaded artifacts from the build. This will be concatenated with the runtime name used to invoke the build to generate the full artifact file name. | |
| Defaults to `ci-outputs`. | |
| N.b., you almost certainly don't need to change this, unless you're running multiple invocations of this action in a single workflow. Then you should provide unique values to ensure you can access the artifacts from all the builds you're doing. | |
| type: string | |
| default: ci-outputs | |
| required: false | |
| continue-on-error: | |
| description: >- | |
| Pass thru for <https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idcontinue-on-error>. | |
| type: boolean | |
| default: false | |
| required: false | |
| workflow_dispatch: | |
| inputs: | |
| ref: | |
| type: string | |
| default: "" | |
| required: false | |
| env: | |
| description: >- | |
| Additional environment variables to set before the build, as a string containing YAML. This is easily produced, for example, by pretending you're writing normal nested YAML within a literal multi-line block scalar (introduced by "|"): | |
| with: | |
| env: | | |
| FOO: bar | |
| I_CANT_BELIEVE: "it's not YAML" | |
| would_you_believe: | | |
| it's | |
| not | |
| yaml | |
| Do not use for secrets! Instead, pass them via GitHub Action's dedicated secrets mechanism. | |
| type: string | |
| default: "" | |
| required: false | |
| runtimes: | |
| description: >- | |
| List of Nextstrain runtimes under which to run the build, as a string containing YAML. This is easily produced, for example, by pretending you're writing normal nested YAML within a literal multi-line block scalar (introduced by "|"): | |
| with: | |
| runtimes: | | |
| - docker | |
| - conda | |
| Defaults to "docker" and "conda". One job per runtime will be run. | |
| type: string | |
| default: | | |
| - docker | |
| - conda | |
| required: false | |
| artifact-name: | |
| description: >- | |
| A base name to use for the uploaded artifacts from the build. This will be concatenated with the runtime name used to invoke the build to generate the full artifact file name. | |
| Defaults to `ci-outputs`. | |
| N.b., you almost certainly don't need to change this, unless you're running multiple invocations of this action in a single workflow. Then you should provide unique values to ensure you can access the artifacts from all the builds you're doing. | |
| type: string | |
| default: ci-outputs | |
| required: false | |
| continue-on-error: | |
| description: >- | |
| Pass thru for <https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idcontinue-on-error>. | |
| type: boolean | |
| default: false | |
| required: false | |
| repo: | |
| description: >- | |
| Repository name with owner (e.g. nextstrain/zika). | |
| type: string | |
| default: "" | |
| required: true | |
| permissions: | |
| contents: read | |
| id-token: write | |
| packages: read | |
| env: | |
| NEXTSTRAIN_GITHUB_DIR: .git/nextstrain/.github | |
| jobs: | |
| configuration: | |
| runs-on: ubuntu-latest | |
| steps: | |
| - id: inputs | |
| env: | |
| runtimes: ${{ inputs.runtimes }} | |
| shell: bash | |
| run: | | |
| runtimes="$(yq --output-format=json --indent=0 . <<<"$runtimes")" | |
| echo runtimes="$runtimes" | tee -a "$GITHUB_OUTPUT" | |
| outputs: | |
| runtimes: ${{ steps.inputs.outputs.runtimes }} | |
| workflow-context: | |
| runs-on: ubuntu-latest | |
| steps: | |
| - id: workflow-context | |
| uses: nextstrain/.github/actions/workflow-context@master | |
| outputs: | |
| repository: ${{ steps.workflow-context.outputs.repository }} | |
| sha: ${{ steps.workflow-context.outputs.sha }} | |
| nextstrain-build: | |
| needs: [configuration, workflow-context] | |
| strategy: | |
| fail-fast: false | |
| matrix: | |
| runtime: ${{ fromJSON(needs.configuration.outputs.runtimes) }} | |
| name: build (${{ matrix.runtime }}) | |
| runs-on: ubuntu-latest | |
| continue-on-error: ${{ inputs.continue-on-error }} | |
| steps: | |
| # Log in, if possible, to docker.io (Docker Hub), since authenticated | |
| # requests get higher rate limits (e.g. for image pulls). Our org-level | |
| # secret DOCKER_TOKEN_PUBLIC_READ_ONLY is available to all our public | |
| # repos on GitHub but only available here to this reusable workflow when | |
| # called with "secrets: inherit". On Docker Hub, the token is granted | |
| # "public read-only" access. | |
| # | |
| # The secrets context is not allowed in "if:" conditions, so we must | |
| # launder it thru env. | |
| - if: env.token-available == 'true' | |
| env: | |
| token-available: ${{ secrets.DOCKER_TOKEN_PUBLIC_READ_ONLY != '' }} | |
| name: Log in to docker.io | |
| uses: docker/login-action@v3 | |
| with: | |
| registry: docker.io | |
| username: nextstrainbot | |
| password: ${{ secrets.DOCKER_TOKEN_PUBLIC_READ_ONLY }} | |
| continue-on-error: true | |
| # Log in, if possible, to ghcr.io which we use for staging images in | |
| # nextstrain/docker-base. The automatic GITHUB_TOKEN is restricted to | |
| # read-only access by the "permissions:" block above. | |
| - name: Log in to ghcr.io | |
| uses: docker/login-action@v3 | |
| with: | |
| registry: ghcr.io | |
| username: ${{ github.actor }} | |
| password: ${{ secrets.GITHUB_TOKEN }} | |
| continue-on-error: true | |
| # Transforms the inputs.env *string* containing YAML like this: | |
| # | |
| # FOO: bar | |
| # I_CANT_BELIEVE: "it's not YAML" | |
| # would_you_believe: | | |
| # it's | |
| # not | |
| # yaml | |
| # | |
| # first into the equivalent JSON (with yq) and then into text (with jq) | |
| # like this: | |
| # | |
| # FOO=<<__EOF__ | |
| # bar | |
| # __EOF__ | |
| # I_CANT_BELIEVE<<__EOF__ | |
| # it's not YAML | |
| # __EOF__ | |
| # would_you_believe<<__EOF__ | |
| # it's | |
| # not | |
| # yaml | |
| # __EOF__ | |
| # | |
| # which is suitable for appending to the $GITHUB_ENV file in order to set | |
| # the environment variables for subsequent steps. | |
| # | |
| # See the GitHub docs for more info on this heredoc-like syntax¹, which I | |
| # use here to avoid quoting issues in arbitrary env var values. | |
| # | |
| # By doing this slightly-convoluted conversion here, callers can use the | |
| # familiar env: block syntax almost without change and avoid paying much | |
| # in accidental complexity. We box it up here and let callers focus on | |
| # their essential complexity. | |
| # -trs, 23 May 2022 | |
| # | |
| # ¹ https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#multiline-strings | |
| # | |
| - if: inputs.env | |
| name: Set environment variables | |
| env: | |
| env: ${{ inputs.env }} | |
| run: > | |
| # shellcheck disable=SC2154 | |
| echo "$env" | yq --output-format json . | jq --raw-output ' | |
| to_entries | |
| | map("\(.key)<<__EOF__\n\(.value)\n__EOF__") | |
| | join("\n") | |
| ' | tee -a "$GITHUB_ENV" | |
| - name: Checkout ${{ inputs.repo }} | |
| uses: actions/checkout@v4 | |
| with: | |
| repository: ${{ inputs.repo }} | |
| ref: ${{ inputs.ref }} | |
| # Need to run this after the build repo is cloned so that cloning the | |
| # build repo does not overwrite the .git dir and remove the extra support files | |
| # that we need from nextstrain/.github repo | |
| - name: Checkout ${{ needs.workflow-context.outputs.repository }} (sha ${{ needs.workflow-context.outputs.sha }}) | |
| uses: actions/checkout@v4 | |
| with: | |
| repository: ${{ needs.workflow-context.outputs.repository }} | |
| ref: ${{ needs.workflow-context.outputs.sha }} | |
| path: ${{ env.NEXTSTRAIN_GITHUB_DIR }} | |
| - name: Verify nextstrain-pathogen.yaml file | |
| run: > | |
| if [[ ! -f nextstrain-pathogen.yaml ]]; then | |
| echo "To use this workflow, there must be a 'nextstrain-pathogen.yaml' file present in the repository root"; | |
| exit 1; | |
| fi | |
| - name: Set up Nextstrain runtime ${{ matrix.runtime }} | |
| uses: ./.git/nextstrain/.github/actions/setup-nextstrain-cli | |
| with: | |
| cli-version: ">=8.3.0" | |
| runtime: ${{ matrix.runtime }} | |
| - name: Run ingest | |
| if: hashFiles('ingest/Snakefile') && hashFiles('ingest/build-configs/ci/config.yaml') | |
| id: ingest | |
| run: nextstrain build ingest --configfile build-configs/ci/config.yaml | |
| - name: Run phylogenetic | |
| if: hashFiles('phylogenetic/Snakefile') && hashFiles('phylogenetic/build-configs/ci/config.yaml') && !cancelled() | |
| id: phylogenetic | |
| run: nextstrain build phylogenetic --configfile build-configs/ci/config.yaml | |
| - name: Run nextclade | |
| if: hashFiles('nextclade/Snakefile') && hashFiles('nextclade/build-configs/ci/config.yaml') && !cancelled() | |
| id: nextclade | |
| run: nextstrain build nextclade --configfile build-configs/ci/config.yaml | |
| - if: always() | |
| uses: actions/upload-artifact@v4 | |
| with: | |
| name: ${{ inputs.artifact-name }}-${{ matrix.runtime }} | |
| if-no-files-found: ignore | |
| # @actions/glob has no support for brace expansion. Hrumph. | |
| path: | | |
| ingest/.snakemake/log/ | |
| ingest/auspice/ | |
| ingest/benchmarks/ | |
| ingest/logs/ | |
| ingest/results/ | |
| phylogenetic/.snakemake/log/ | |
| phylogenetic/auspice/ | |
| phylogenetic/benchmarks/ | |
| phylogenetic/logs/ | |
| phylogenetic/results/ | |
| nextclade/.snakemake/log/ | |
| nextclade/auspice/ | |
| nextclade/benchmarks/ | |
| nextclade/logs/ | |
| nextclade/results/ | |
| - if: always() | |
| name: Verify a workflow ran | |
| env: | |
| # "outcome" is success/failure/cancelled/skipped _before_ | |
| # "continue-on-error" is applied to calculate "conclusion"; we no | |
| # longer use continue-on-error for these steps, but even so, | |
| # conceptually here what we want is outcome not conclusion. | |
| ingest: ${{ steps.ingest.outcome }} | |
| phylogenetic: ${{ steps.phylogenetic.outcome }} | |
| nextclade: ${{ steps.nextclade.outcome }} | |
| run: | | |
| # Show step outcomes in job logs… | |
| echo "ingest $ingest" | |
| echo "phylogenetic $phylogenetic" | |
| echo "nextclade $nextclade" | |
| # …and also in the workflow summary. | |
| "$NEXTSTRAIN_GITHUB_DIR"/bin/interpolate-env < "$NEXTSTRAIN_GITHUB_DIR"/text-templates/pathogen-repo-ci.md > "$GITHUB_STEP_SUMMARY" | |
| # Assert status; we're good if we see at least one success and the | |
| # rest are success or skipped. | |
| [[ | |
| ($ingest == success || $phylogenetic == success || $nextclade == success) | |
| && ($ingest == success || $ingest == skipped) | |
| && ($phylogenetic == success || $phylogenetic == skipped) | |
| && ($nextclade == success || $nextclade == skipped) | |
| ]] |