Skip to content
This repository has been archived by the owner on Dec 14, 2023. It is now read-only.

[RFR] Print test failures in GH Actions #765

Open
wants to merge 26 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ jobs:
- name: Print tests that are going to be run in this chunk
run: cat tests_chunk_${{ matrix.chunk_number }}

# Run a selected chunk of tests in parallel, keep a log; on error, print out said log
# Run a selected chunk of tests in parallel, keep a log; on error, print out said log.
- name: Run tests
run: >-
cat tests_chunk_${{ matrix.chunk_number }}
Expand All @@ -189,3 +189,7 @@ jobs:
MC_TWITTER_ACCESS_TOKEN_SECRET: ${{ secrets.MC_TWITTER_ACCESS_TOKEN_SECRET }}
MC_TWITTER_CONSUMER_KEY: ${{ secrets.MC_TWITTER_CONSUMER_KEY }}
MC_TWITTER_CONSUMER_SECRET: ${{ secrets.MC_TWITTER_CONSUMER_SECRET }}

- name: List test failures and errors
run: echo -e 'Test failures:\n' && awk '/FAILURES/,/short/' /home/runner/runners/*/_diag/pages/*.log && echo -e '\n\nErrors:\n' && awk '/non-zero/,/status 1/' /home/runner/runners/*/_diag/pages/*.log
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's in /home/runner/runners/*/_diag/pages/*.log, i.e. how are you finding those errors? Do you perhaps have a sample log file from that directory that we could look at (without various private credentials that might be in this log)?

Grepping (awking?) out failures from a log file is a bit wonky because any test might print a string FAILURES at any point for a legitimate reason and still succeed; plus, tests don't have to fail with status 1 specifically as they can have any non-zero exit status.

if: always()
1 change: 1 addition & 0 deletions apps/munin-node/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ FROM gcr.io/mcback/base:latest
RUN \
#
# Install plugin dependencies
apt-get -y update && \
apt-get -y --no-install-recommends install \
libdbd-pg-perl \
libdbix-simple-perl \
Expand Down
4 changes: 4 additions & 0 deletions doc/dev_test_environment.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,7 @@ To import stories into Solr from within the test file:
```

If you use `MediaWords::Test::Solr::setup_test_index()` helper to import test stories to Solr, it will do the importing for you.

## Debugging GitHub Actions

We use GitHub Actions for our build/test/deploy flow, with the configuration stored in [`.github/workflows/build.yml`](https://github.com/mediacloud/backend/blob/master/.github/workflows/build.yml). Should you need to debug this process (say, if you're attempting to add a new step and it's not working as expected), you can temporarily make use of the tmate tool (as is done in [this commit](https://github.com/mediacloud/backend/blob/2f6e85ba536a4938fb10a8b83353977a4f43dda0/.github/workflows/build.yml#L192-L196)). Said tool will expose a URL in the build log at which you can `ssh` to the GitHub Actions container. Don't forget to **remove the tmate step** when your debugging is complete. Note that you need to be a collaborator on the project for this to work.