Skip to content

[feat] Make the asynchronous execution policy default #1127

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jan 24, 2020
Merged

[feat] Make the asynchronous execution policy default #1127

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
ee214fe
Make the async execution policy default
Jan 21, 2020
cc8e9f3
Merge branch 'master' of github.com:eth-cscs/reframe into feature/def…
Jan 23, 2020
ba1ca1c
Address PR comments
Jan 23, 2020
9544028
Address PR comments (ver 2)
Jan 24, 2020
69c8abd
Merge branch 'master' into feature/default_async_policy
Jan 24, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions docs/running.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1059,8 +1059,8 @@ Asynchronous Execution of Regression Checks
-------------------------------------------

From version `2.4 <https://github.com/eth-cscs/reframe/releases/tag/v2.4>`__, ReFrame supports asynchronous execution of regression tests.
This execution policy can be enabled by passing the option ``--exec-policy=async`` to the command line.
The default execution policy is ``serial`` which enforces a sequential execution of the selected regression tests.
This execution policy is the default one.
To enforce a sequential execution of the regression tests the ``serial`` execution policy can be enabled by passing the option ``--exec-policy=serial`` to the command line.
The asynchronous execution policy parallelizes only the `running phase <pipeline.html#the-run-phase>`__ of the tests.
The rest of the phases remain sequential.

Expand Down Expand Up @@ -1152,6 +1152,11 @@ Here is an example output of ReFrame using asynchronous execution policy:
The asynchronous execution policy may provide significant overall performance benefits for run-only regression tests.
For compile-only and normal tests that require a compilation, the execution time will be bound by the total compilation time of the test.

.. note::
.. versionchanged:: 3.0

The asynchronous execution policy has become the default.


Manipulating modules
--------------------
Expand Down
92 changes: 47 additions & 45 deletions docs/tutorial.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ For the configuration of the system, we provide a minimal configuration file for
The site configuration that we used for this tutorial is the following:

.. literalinclude:: ../tutorial/config/settings.py
:lines: 12-75
:lines: 11-79
:dedent: 4

You can find the full ``settings.py`` file ready to be used by ReFrame in ``<reframe-install-prefix>/tutorial/config/settings.py``.
Expand Down Expand Up @@ -172,50 +172,52 @@ If everything is configured correctly for your system, you should get an output

.. code-block:: none

Command line: ./bin/reframe -C tutorial/config/settings.py -c tutorial/example1.py -r
Reframe version: 2.13-dev0
Launched by user: XXX
Launched on host: daint104
Reframe paths
=============
Check prefix :
Check search path : 'tutorial/example1.py'
Stage dir prefix : /current/working/dir/stage/
Output dir prefix : /current/working/dir/output/
Logging dir : /current/working/dir/logs
[==========] Running 1 check(s)
[==========] Started on Fri May 18 13:19:12 2018

[----------] started processing Example1Test (Simple matrix-vector multiplication example)
[ RUN ] Example1Test on daint:login using PrgEnv-cray
[ OK ] Example1Test on daint:login using PrgEnv-cray
[ RUN ] Example1Test on daint:login using PrgEnv-gnu
[ OK ] Example1Test on daint:login using PrgEnv-gnu
[ RUN ] Example1Test on daint:login using PrgEnv-intel
[ OK ] Example1Test on daint:login using PrgEnv-intel
[ RUN ] Example1Test on daint:login using PrgEnv-pgi
[ OK ] Example1Test on daint:login using PrgEnv-pgi
[ RUN ] Example1Test on daint:gpu using PrgEnv-cray
[ OK ] Example1Test on daint:gpu using PrgEnv-cray
[ RUN ] Example1Test on daint:gpu using PrgEnv-gnu
[ OK ] Example1Test on daint:gpu using PrgEnv-gnu
[ RUN ] Example1Test on daint:gpu using PrgEnv-intel
[ OK ] Example1Test on daint:gpu using PrgEnv-intel
[ RUN ] Example1Test on daint:gpu using PrgEnv-pgi
[ OK ] Example1Test on daint:gpu using PrgEnv-pgi
[ RUN ] Example1Test on daint:mc using PrgEnv-cray
[ OK ] Example1Test on daint:mc using PrgEnv-cray
[ RUN ] Example1Test on daint:mc using PrgEnv-gnu
[ OK ] Example1Test on daint:mc using PrgEnv-gnu
[ RUN ] Example1Test on daint:mc using PrgEnv-intel
[ OK ] Example1Test on daint:mc using PrgEnv-intel
[ RUN ] Example1Test on daint:mc using PrgEnv-pgi
[ OK ] Example1Test on daint:mc using PrgEnv-pgi
[----------] finished processing Example1Test (Simple matrix-vector multiplication example)

[ PASSED ] Ran 12 test case(s) from 1 check(s) (0 failure(s))
[==========] Finished on Fri May 18 13:20:17 2018

Command line: ./bin/reframe -C tutorial/config/settings.py -c tutorial/example1.py -r
Reframe version: 3.0-dev0
Launched by user: XXX
Launched on host: daint103
Reframe paths
=============
Check prefix :
Check search path : 'tutorial/example1.py'
Stage dir prefix : /current/working/dir/stage/
Output dir prefix : /current/working/dir/output/
Perf. logging prefix : /current/working/dir/perflogs/
[==========] Running 1 check(s)
[==========] Started on Thu Jan 23 14:14:12 2020

[----------] started processing Example1Test (Simple matrix-vector multiplication example)
[ RUN ] Example1Test on daint:login using PrgEnv-cray
[ RUN ] Example1Test on daint:login using PrgEnv-gnu
[ RUN ] Example1Test on daint:login using PrgEnv-intel
[ RUN ] Example1Test on daint:login using PrgEnv-pgi
[ RUN ] Example1Test on daint:gpu using PrgEnv-cray
[ RUN ] Example1Test on daint:gpu using PrgEnv-gnu
[ RUN ] Example1Test on daint:gpu using PrgEnv-intel
[ RUN ] Example1Test on daint:gpu using PrgEnv-pgi
[ RUN ] Example1Test on daint:mc using PrgEnv-cray
[ RUN ] Example1Test on daint:mc using PrgEnv-gnu
[ RUN ] Example1Test on daint:mc using PrgEnv-intel
[ RUN ] Example1Test on daint:mc using PrgEnv-pgi
[----------] finished processing Example1Test (Simple matrix-vector multiplication example)

[----------] waiting for spawned checks to finish
[ OK ] Example1Test on daint:login using PrgEnv-intel
[ OK ] Example1Test on daint:login using PrgEnv-cray
[ OK ] Example1Test on daint:login using PrgEnv-gnu
[ OK ] Example1Test on daint:login using PrgEnv-pgi
[ OK ] Example1Test on daint:mc using PrgEnv-pgi
[ OK ] Example1Test on daint:mc using PrgEnv-gnu
[ OK ] Example1Test on daint:gpu using PrgEnv-intel
[ OK ] Example1Test on daint:gpu using PrgEnv-cray
[ OK ] Example1Test on daint:mc using PrgEnv-cray
[ OK ] Example1Test on daint:gpu using PrgEnv-gnu
[ OK ] Example1Test on daint:gpu using PrgEnv-pgi
[ OK ] Example1Test on daint:mc using PrgEnv-intel
[----------] all spawned checks have finished

[ PASSED ] Ran 12 test case(s) from 1 check(s) (0 failure(s))
[==========] Finished on Thu Jan 23 14:16:25 2020

Notice how our regression test is run on every partition of the configured system and for every programming environment.

Expand Down
4 changes: 2 additions & 2 deletions reframe/frontend/cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,9 +173,9 @@ def main():
help='Skip prog. environment check')
run_options.add_argument(
'--exec-policy', metavar='POLICY', action='store',
choices=['serial', 'async'], default='serial',
choices=['async', 'serial'], default='async',
help='Specify the execution policy for running the regression tests. '
'Available policies: "serial" (default), "async"')
'Available policies: "async" (default), "serial"')
run_options.add_argument(
'--mode', action='store', help='Execution mode to use')
run_options.add_argument(
Expand Down