Skip to content

Conversation

@bryevdv
Copy link
Contributor

@bryevdv bryevdv commented Jun 28, 2022

Requested by @fduguet-nv This PR adds support for a -vv option to test.py. When -vv is sprecified, the pytest -s option is added to the individual test file invocations that the test.py test runner launches. This will cause the stdout output, which is normally suppressed on passing tests, to be displayed unconditionally.

It is recommended that this option be used in conjunction with the --files argument, in order to obtain more manageable output. Here is an example of such usage:

dev38 ❯ ./test.py --use=cuda,cpus,eager -vv --files tests/integration/test_binary_op_broadcast.py                         

############################################################
### 
### Test Suite Configuration
### 
### * Feature stages       : cpus, cuda, eager
### * Test files per stage : 1
### 
############################################################

############################################################
### Entering stage: CPU (with 1 worker)
############################################################

[PASS] (CPU) tests/integration/test_binary_op_broadcast.py
   ============================= test session starts ==============================
   platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0 -- /home/bryan/anaconda3/envs/dev38/bin/python3
   cachedir: .pytest_cache
   rootdir: /home/bryan/work/cunumeric, configfile: pyproject.toml
   plugins: cov-3.0.0, mock-3.8.1
   collecting ... collected 4 items
   
   tests/integration/test_binary_op_broadcast.py::test_random[2-(20, 21, 22)]   (20, 21) x (21,)
     (20, 21) x (1, 21)
     (20, 21) x (20, 1)
   PASSED
   tests/integration/test_binary_op_broadcast.py::test_random[2-(1, 21, 22)]   (1, 21) x (21,)
     (1, 21) x (1, 21)
     (1, 21) x (1, 1)
   PASSED
   tests/integration/test_binary_op_broadcast.py::test_random[3-(20, 21, 22)]   (20, 21, 22) x (22,)
     (20, 21, 22) x (21, 22)
     (20, 21, 22) x (1, 21, 22)
     (20, 21, 22) x (20, 1, 22)
     (20, 21, 22) x (20, 21, 1)
     (20, 21, 22) x (20, 1, 1)
     (20, 21, 22) x (1, 21, 1)
     (20, 21, 22) x (1, 1, 22)
   PASSED
   tests/integration/test_binary_op_broadcast.py::test_random[3-(1, 21, 22)]   (1, 21, 22) x (22,)
     (1, 21, 22) x (21, 22)
     (1, 21, 22) x (1, 21, 22)
     (1, 21, 22) x (1, 1, 22)
     (1, 21, 22) x (1, 21, 1)
     (1, 21, 22) x (1, 1, 1)
     (1, 21, 22) x (1, 21, 1)
     (1, 21, 22) x (1, 1, 22)
   PASSED
   
   ============================== 4 passed in 0.43s ===============================
   [0 - 7fd3894ec000]    0.000084 {4}{threads}: reservation ('CPU proc 1d00000000000004') cannot be satisfied
   
                           CPU: Passed 1 of 1 tests (100.0%)

############################################################
### 
### Exiting state: CPU
### 
### * Results      : 1 / 1 files passed (100.0%)
### * Elapsed time : 0:00:01.831223
### 
############################################################

############################################################
### Entering stage: GPU (with 7 workers)
############################################################

[PASS] (GPU) tests/integration/test_binary_op_broadcast.py
   ============================= test session starts ==============================
   platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0 -- /home/bryan/anaconda3/envs/dev38/bin/python3
   cachedir: .pytest_cache
   rootdir: /home/bryan/work/cunumeric, configfile: pyproject.toml
   plugins: cov-3.0.0, mock-3.8.1
   collecting ... collected 4 items
   
   tests/integration/test_binary_op_broadcast.py::test_random[2-(20, 21, 22)]   (20, 21) x (21,)
     (20, 21) x (1, 21)
     (20, 21) x (20, 1)
   PASSED
   tests/integration/test_binary_op_broadcast.py::test_random[2-(1, 21, 22)]   (1, 21) x (21,)
     (1, 21) x (1, 21)
     (1, 21) x (1, 1)
   PASSED
   tests/integration/test_binary_op_broadcast.py::test_random[3-(20, 21, 22)]   (20, 21, 22) x (22,)
     (20, 21, 22) x (21, 22)
     (20, 21, 22) x (1, 21, 22)
     (20, 21, 22) x (20, 1, 22)
     (20, 21, 22) x (20, 21, 1)
     (20, 21, 22) x (20, 1, 1)
     (20, 21, 22) x (1, 21, 1)
     (20, 21, 22) x (1, 1, 22)
   PASSED
   tests/integration/test_binary_op_broadcast.py::test_random[3-(1, 21, 22)]   (1, 21, 22) x (22,)
     (1, 21, 22) x (21, 22)
     (1, 21, 22) x (1, 21, 22)
     (1, 21, 22) x (1, 1, 22)
     (1, 21, 22) x (1, 21, 1)
     (1, 21, 22) x (1, 1, 1)
     (1, 21, 22) x (1, 21, 1)
     (1, 21, 22) x (1, 1, 22)
   PASSED
   
   ============================== 4 passed in 0.67s ===============================
   [0 - 7f941e08d000]    0.000098 {4}{threads}: reservation ('CPU proc 1d00000000000003') cannot be satisfied
   
                           GPU: Passed 1 of 1 tests (100.0%)

############################################################
### 
### Exiting state: GPU
### 
### * Results      : 1 / 1 files passed (100.0%)
### * Elapsed time : 0:00:02.297075
### 
############################################################

############################################################
### Entering stage: Eager (with 1 worker)
############################################################

[PASS] (Eager) tests/integration/test_binary_op_broadcast.py
   ============================= test session starts ==============================
   platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0 -- /home/bryan/anaconda3/envs/dev38/bin/python3
   cachedir: .pytest_cache
   rootdir: /home/bryan/work/cunumeric, configfile: pyproject.toml
   plugins: cov-3.0.0, mock-3.8.1
   collecting ... collected 4 items
   
   tests/integration/test_binary_op_broadcast.py::test_random[2-(20, 21, 22)]   (20, 21) x (21,)
     (20, 21) x (1, 21)
     (20, 21) x (20, 1)
   PASSED
   tests/integration/test_binary_op_broadcast.py::test_random[2-(1, 21, 22)]   (1, 21) x (21,)
     (1, 21) x (1, 21)
     (1, 21) x (1, 1)
   PASSED
   tests/integration/test_binary_op_broadcast.py::test_random[3-(20, 21, 22)]   (20, 21, 22) x (22,)
     (20, 21, 22) x (21, 22)
     (20, 21, 22) x (1, 21, 22)
     (20, 21, 22) x (20, 1, 22)
     (20, 21, 22) x (20, 21, 1)
     (20, 21, 22) x (20, 1, 1)
     (20, 21, 22) x (1, 21, 1)
     (20, 21, 22) x (1, 1, 22)
   PASSED
   tests/integration/test_binary_op_broadcast.py::test_random[3-(1, 21, 22)]   (1, 21, 22) x (22,)
     (1, 21, 22) x (21, 22)
     (1, 21, 22) x (1, 21, 22)
     (1, 21, 22) x (1, 1, 22)
     (1, 21, 22) x (1, 21, 1)
     (1, 21, 22) x (1, 1, 1)
     (1, 21, 22) x (1, 21, 1)
     (1, 21, 22) x (1, 1, 22)
   PASSED
   
   ============================== 4 passed in 0.04s ===============================
   [0 - 7fb6a8ab5000]    0.000068 {4}{threads}: reservation ('CPU proc 1d00000000000002') cannot be satisfied
   
                         Eager: Passed 1 of 1 tests (100.0%)

############################################################
### 
### Exiting state: Eager
### 
### * Results      : 1 / 1 files passed (100.0%)
### * Elapsed time : 0:00:01.318096
### 
############################################################

    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                     All tests: Passed 3 of 3 tests (100.0%)

@bryevdv bryevdv requested review from fduguet-nv and magnatelee June 28, 2022 23:24
@bryevdv bryevdv merged commit 8e2f3a3 into nv-legate:branch-22.07 Jun 29, 2022
@bryevdv bryevdv deleted the bryanv/more_verbose_testpy_option branch June 29, 2022 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants