From d0f5ed862039937182c164067ae6bade9001e418 Mon Sep 17 00:00:00 2001 From: Kenichi Maehashi Date: Thu, 19 Oct 2017 17:40:32 +0900 Subject: [PATCH 1/4] fix contribution guide for test framework change --- docs/source/contribution.rst | 30 ++++++++++++++++-------------- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/docs/source/contribution.rst b/docs/source/contribution.rst index 87074b2efc49..76a68e88dbe9 100644 --- a/docs/source/contribution.rst +++ b/docs/source/contribution.rst @@ -286,40 +286,42 @@ Unit Testing Testing is one of the most important part of your code. You must write test cases and verify your implementation by following our testing guide. -Note that we are using nose and mock package for testing, so install them before writing your code:: +Note that we are using pytest and mock package for testing, so install them before writing your code:: - $ pip install nose mock + $ pip install pytest mock How to Run Tests ~~~~~~~~~~~~~~~~ -You can run unit tests simply by running ``nosetests`` command at the repository root:: +You can run unit tests simply by running ``python -m pytest`` command at the repository root:: - $ nosetests + $ python -m pytest or specify the test script that you want to run:: - $ nosetests path/to/your/test.py + $ python -m pytest path/to/your/test.py You can also run all unit tests under a specified directory:: - $ nosetests tests/chainer_tests/ + $ python -m pytest tests/chainer_tests/ -It requires CUDA by default. -In order to run unit tests that do not require CUDA, pass ``--attr='!gpu'`` option to the ``nosetests`` command:: +It requires CUDA and cuDNN by default. +In order to run unit tests that do not require CUDA and cuDNN, use ``CHAINER_TESTING_GPU_LIMIT=0`` environment variable and ``-m='not cudnn'`` option:: - $ nosetests path/to/your/test.py --attr='!gpu' + $ export CHAINER_TESITNG_GPU_LIMIT=0 + $ python -m pytest path/to/your/test.py -m='not cudnn' Some GPU tests involve multiple GPUs. -If you want to run GPU tests with insufficient number of GPUs, specify the number of available GPUs by ``--eval-attr='gpu Date: Fri, 27 Oct 2017 14:04:12 +0900 Subject: [PATCH 2/4] update environment variable name --- docs/source/contribution.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/source/contribution.rst b/docs/source/contribution.rst index 76a68e88dbe9..51c0e2f17f3d 100644 --- a/docs/source/contribution.rst +++ b/docs/source/contribution.rst @@ -306,16 +306,16 @@ You can also run all unit tests under a specified directory:: $ python -m pytest tests/chainer_tests/ It requires CUDA and cuDNN by default. -In order to run unit tests that do not require CUDA and cuDNN, use ``CHAINER_TESTING_GPU_LIMIT=0`` environment variable and ``-m='not cudnn'`` option:: +In order to run unit tests that do not require CUDA and cuDNN, use ``CHAINER_TEST_GPU_LIMIT=0`` environment variable and ``-m='not cudnn'`` option:: $ export CHAINER_TESITNG_GPU_LIMIT=0 $ python -m pytest path/to/your/test.py -m='not cudnn' Some GPU tests involve multiple GPUs. -If you want to run GPU tests with insufficient number of GPUs, specify the number of available GPUs to ``CHAINER_TESTING_GPU_LIMIT``. +If you want to run GPU tests with insufficient number of GPUs, specify the number of available GPUs to ``CHAINER_TEST_GPU_LIMIT``. For example, if you have only one GPU, launch ``pytest`` by the following command to skip multi-GPU tests:: - $ export CHAINER_TESTING_GPU_LIMIT=1 + $ export CHAINER_TEST_GPU_LIMIT=1 $ python -m pytest path/to/gpu/test.py Some tests spend too much time. From 6d15a964b56375c89ff9c9cdaba0d988a6ac2fdd Mon Sep 17 00:00:00 2001 From: Kenichi Maehashi Date: Fri, 27 Oct 2017 14:11:00 +0900 Subject: [PATCH 3/4] update how to write test section --- docs/source/contribution.rst | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/docs/source/contribution.rst b/docs/source/contribution.rst index 51c0e2f17f3d..03e60d3534dd 100644 --- a/docs/source/contribution.rst +++ b/docs/source/contribution.rst @@ -356,8 +356,9 @@ Test functions that require CUDA must be tagged by ``chainer.testing.attr.gpu`` def test_my_gpu_func(self): ... -The functions tagged by the ``gpu`` decorator are skipped if ``--attr='!gpu'`` is given. -We also have the ``chainer.testing.attr.cudnn`` decorator to let ``nosetests`` know that the test depends on cuDNN. +The functions tagged by the ``gpu`` decorator are skipped if ``CHAINER_TEST_GPU_LIMIT=0`` environment variable is set. +We also have the ``chainer.testing.attr.cudnn`` decorator to let ``pytest`` know that the test depends on cuDNN. +The test functions decorated by ``cudnn`` are skipped if ``-m='not cudnn'`` is given. The test functions decorated by ``gpu`` must not depend on multiple GPUs. In order to write tests for multiple GPUs, use ``chainer.testing.attr.multi_gpu()`` decorator instead:: @@ -373,7 +374,7 @@ In order to write tests for multiple GPUs, use ``chainer.testing.attr.multi_gpu( ... If your test requires too much time, add ``chainer.testing.attr.slow`` decorator. -The test functions decorated by ``slow`` are skipped if ``--attr='!slow'`` is given:: +The test functions decorated by ``slow`` are skipped if ``-m='not slow'`` is given:: import unittest from chainer.testing import attr @@ -386,8 +387,8 @@ The test functions decorated by ``slow`` are skipped if ``--attr='!slow'`` is gi ... .. note:: - If you want to specify more than two attributes, separate them with a comma such as ``--attr='!gpu,!slow'``. - See detail in `the document of nose `_. + If you want to specify more than two attributes, use ``and`` operator like ``-m='not cudnn and not slow'``. + See detail in `the document of pytest `_. Once you send a pull request, your code is automatically tested by `Travis-CI `_ **with --attr='!gpu,!slow' option**. Since Travis-CI does not support CUDA, we cannot check your CUDA-related code automatically. From 40eebc47e225ab3c674e068ad3b69ed5d2c98ff3 Mon Sep 17 00:00:00 2001 From: Kenichi Maehashi Date: Fri, 27 Oct 2017 15:06:07 +0900 Subject: [PATCH 4/4] fix travis test run condition --- docs/source/contribution.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/contribution.rst b/docs/source/contribution.rst index 03e60d3534dd..7562b2c14030 100644 --- a/docs/source/contribution.rst +++ b/docs/source/contribution.rst @@ -390,7 +390,7 @@ The test functions decorated by ``slow`` are skipped if ``-m='not slow'`` is giv If you want to specify more than two attributes, use ``and`` operator like ``-m='not cudnn and not slow'``. See detail in `the document of pytest `_. -Once you send a pull request, your code is automatically tested by `Travis-CI `_ **with --attr='!gpu,!slow' option**. +Once you send a pull request, your code is automatically tested by `Travis-CI `_ **except for tests annotated with ``gpu``, ``multi_gpu`` and ``slow``**. Since Travis-CI does not support CUDA, we cannot check your CUDA-related code automatically. The reviewing process starts after the test passes. Note that reviewers will test your code without the option to check CUDA-related code.