Skip to content

Commit

Permalink
docs(overview): update OVERVIEW.md
Browse files Browse the repository at this point in the history
Update the OVERVIEW.md file to provide more details about pass/fail
status propagation, busted running mode, static mode and other
configuration options
  • Loading branch information
samugi authored and hanshuebner committed Feb 6, 2024
1 parent 69f0c2a commit 9addcdf
Showing 1 changed file with 30 additions and 14 deletions.
44 changes: 30 additions & 14 deletions OVERVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,24 +25,40 @@ runner.
![Build & Test Workflow](img/build_and_test.png)

The "Build & Test" workflow triggers the "Schedule busted tests to
run" job, distributing tests among a user-configurable set of
runners. This job first fetches the predicted runtimes file from a
dedicated storage repository and then executes the "Schedule"
Action. The action assembles a test list from a definition file in the
Gateway repository, and using both this list and the runtime
prediction file, it creates distinct files for each runner, specifying
the tests to run. These files are subsequently uploaded to a GitHub
artifact, facilitating retrieval by the runner jobs.
run" job, distributing tests among a set of runners.
This job first fetches the predicted runtimes file from a dedicated
storage repository and then executes the `schedule` Action. The action
assembles a test list from a definition file in the Gateway repository,
and using both this list and the runtime prediction file, it creates
distinct files for each runner, specifying the tests to run. These
files are subsequently uploaded to a GitHub artifact, facilitating
retrieval by the runner jobs.
The number of runners can also be configured in this action using the
`runner-count` input parameter.
The `schedule` action needs to access the storage repository that holds
the runtimes file in order to balance tests based on their execution
times. To enhance robustness and minimize dependencies on external
systems, the action provides a `static-mode` input parameter. Enabling
this allows the system to operate autonomously without relying on
external dependencies, conducting a more basic scheduling process that
does not rely on test runtime information.
`static-mode` gets enabled in "Build & Test" when the "Schedule busted
tests to run" job is retried.

The "Busted test runner" job utilizes its ordinal number to choose the
relevant control file from the GitHub artifact. Subsequently, it
executes tests sequentially, recording the execution time for each
test file in a "runtime log" file. In case of test failure, the
specification is documented in a "failed" file. Upon completion of all
tests, both the "runtime log" and "failed" files are uploaded to
GitHub artifacts. When a runner is rerun owing to test failures, the
"failed" file serves as the control file, ensuring only the failed
tests are re-executed.
test file in a "runtime log" file.
Test failures are detected by capturing the events generated by Busted,
in case of `failure` or `error` events, these are propagated with an
exit status of 1, causing the job to fail. Busted is executed without
the `--no-keep-going` flag, so the tests execution does not stop when
there are failures.
In case of test failure, the specification is documented in a "failed"
file. Upon completion of all tests, both the "runtime log" and "failed"
files are uploaded to GitHub artifacts. When a runner is rerun owing to
test failures, the "failed" file serves as the control file, ensuring
only the failed tests are re-executed.

## The "Predict Runtimes" Workflow

Expand Down

0 comments on commit 9addcdf

Please sign in to comment.