Skip to content

Commit

Permalink
pathogen-repo-build: Move build.log outside of working directory
Browse files Browse the repository at this point in the history
The entire working directory gets zipped and uploaded to AWS Batch when
using the `aws-batch` runtime, which includes an early `build.log` file.
When the build completes and downloads the build outputs, the early
`build.log` file overwrites the local `build.log` of the GH Action.
This results in a truncated file that does not include the
AWS_BATCH_JOB_ID and prevents the generation of the AWS Batch summary.

This commit moves the hard-coded `build.log` file to outside of the
current working directory to prevent it from being uploaded with the
pathogen repo build.

Using "~/build.log" because "../build.log" is not allowed as a pattern
for the `upload-artifact`` action.¹

¹ actions/upload-artifact#176
  • Loading branch information
joverlee521 committed Mar 15, 2024
1 parent 5cd0ba1 commit 0c2d300
Show file tree
Hide file tree
Showing 2 changed files with 17 additions and 17 deletions.
18 changes: 9 additions & 9 deletions .github/workflows/pathogen-repo-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -98,14 +98,14 @@ on:
The default paths included in the artifact are:
build.log
~/build.log
auspice/
results/
benchmarks/
logs/
.snakemake/log/
The "build.log" contains log messages from the `nextstrain build` command. The other paths are common output paths for Nextstrain builds. If a path does not exist in your build, then the action will still succeed and will print out a warning for the non-existent file(s). Use an exclude pattern for any of the default paths that you would like to exclude from the artifact (e.g. !build.log).
The "~/build.log" contains log messages from the `nextstrain build` command. The other paths are common output paths for Nextstrain builds. If a path does not exist in your build, then the action will still succeed and will print out a warning for the non-existent file(s). Use an exclude pattern for any of the default paths that you would like to exclude from the artifact (e.g. !~/build.log).
This is not supported for builds on AWS Batch because the workflow detaches from the build. Please use the `nextstrain build` command locally to reattach to AWS Batch builds to download outputs.
type: string
Expand Down Expand Up @@ -189,14 +189,14 @@ on:
The default paths included in the artifact are:
build.log
~/build.log
auspice/
results/
benchmarks/
logs/
.snakemake/log/
The "build.log" contains log messages from the `nextstrain build` command. The other paths are common output paths for Nextstrain builds. If a path does not exist in your build, then the action will still succeed and will print out a warning for the non-existent file(s). Use an exclude pattern for any of the default paths that you would like to exclude from the artifact (e.g. !build.log).
The "~/build.log" contains log messages from the `nextstrain build` command. The other paths are common output paths for Nextstrain builds. If a path does not exist in your build, then the action will still succeed and will print out a warning for the non-existent file(s). Use an exclude pattern for any of the default paths that you would like to exclude from the artifact (e.g. !~/build.log).
This is not supported for builds on AWS Batch because the workflow detaches from the build. Please use the `nextstrain build` command locally to reattach to AWS Batch builds to download outputs.
type: string
Expand Down Expand Up @@ -274,14 +274,14 @@ jobs:
# shellcheck disable=SC2154
set -x
eval "$NEXTSTRAIN_BUILD_COMMAND" |& tee build.log
eval "$NEXTSTRAIN_BUILD_COMMAND" |& tee ~/build.log
# Attempt to get the AWS Batch ID even if the run build command failed
# as long as the runtime is `aws-batch` and the `build.log` file exists
- if: ${{ always() && inputs.runtime == 'aws-batch' && hashFiles('build.log') != '' }}
# as long as the runtime is `aws-batch` and the `~/build.log` file exists
- if: ${{ always() && inputs.runtime == 'aws-batch' && hashFiles('~/build.log') != '' }}
name: Get AWS Batch job id
id: aws-batch
run: |
echo "AWS_BATCH_JOB_ID=$(sed -nE 's/.+AWS Batch Job ID\:.+ ([-a-f0-9]+)$/\1/p' < build.log)" | tee -a "$GITHUB_ENV"
echo "AWS_BATCH_JOB_ID=$(sed -nE 's/.+AWS Batch Job ID\:.+ ([-a-f0-9]+)$/\1/p' < ~/build.log)" | tee -a "$GITHUB_ENV"
- if: ${{ always() && env.AWS_BATCH_JOB_ID }}
name: Generate AWS Batch summary
run: |
Expand All @@ -293,7 +293,7 @@ jobs:
if-no-files-found: warn
name: ${{ inputs.artifact-name }}
path: |
build.log
~/build.log
auspice/
results/
benchmarks/
Expand Down
16 changes: 8 additions & 8 deletions .github/workflows/pathogen-repo-build.yaml.in
Original file line number Diff line number Diff line change
Expand Up @@ -128,19 +128,19 @@ on:

The default paths included in the artifact are:

build.log
~/build.log
auspice/
results/
benchmarks/
logs/
.snakemake/log/

The "build.log" contains log messages from the `nextstrain build` command.
The "~/build.log" contains log messages from the `nextstrain build` command.
The other paths are common output paths for Nextstrain builds.
If a path does not exist in your build, then the action will still
succeed and will print out a warning for the non-existent file(s).
Use an exclude pattern for any of the default paths that you would like to
exclude from the artifact (e.g. !build.log).
exclude from the artifact (e.g. !~/build.log).

This is not supported for builds on AWS Batch because the workflow
detaches from the build. Please use the `nextstrain build` command
Expand Down Expand Up @@ -240,15 +240,15 @@ jobs:
# shellcheck disable=SC2154
set -x

eval "$NEXTSTRAIN_BUILD_COMMAND" |& tee build.log
eval "$NEXTSTRAIN_BUILD_COMMAND" |& tee ~/build.log

# Attempt to get the AWS Batch ID even if the run build command failed
# as long as the runtime is `aws-batch` and the `build.log` file exists
- if: ${{ always() && inputs.runtime == 'aws-batch' && hashFiles('build.log') != '' }}
# as long as the runtime is `aws-batch` and the `~/build.log` file exists
- if: ${{ always() && inputs.runtime == 'aws-batch' && hashFiles('~/build.log') != '' }}
name: Get AWS Batch job id
id: aws-batch
run: |
echo "AWS_BATCH_JOB_ID=$(sed -nE 's/.+AWS Batch Job ID\:.+ ([-a-f0-9]+)$/\1/p' < build.log)" | tee -a "$GITHUB_ENV"
echo "AWS_BATCH_JOB_ID=$(sed -nE 's/.+AWS Batch Job ID\:.+ ([-a-f0-9]+)$/\1/p' < ~/build.log)" | tee -a "$GITHUB_ENV"

- if: ${{ always() && env.AWS_BATCH_JOB_ID }}
name: Generate AWS Batch summary
Expand All @@ -262,7 +262,7 @@ jobs:
if-no-files-found: warn
name: ${{ inputs.artifact-name }}
path: |
build.log
~/build.log
auspice/
results/
benchmarks/
Expand Down

0 comments on commit 0c2d300

Please sign in to comment.