New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unit-testing for the server API #70

Merged
merged 24 commits into from Oct 17, 2016

Conversation

Projects
None yet
2 participants
@vxgmichel
Copy link
Contributor

vxgmichel commented Sep 30, 2016

This PR is meant to be a discussion about unit-testing pytango.

As a starting point, 3 tests have already been implemented: test_empty_device, test_set_state and test_set_status. Running pytest gives the following result (works with python 2 and python 3):

.../pytango $ pytest -v
================================================ test session starts ================================================
platform linux2 -- Python 2.7.6, pytest-3.0.3, py-1.4.31, pluggy-0.4.0 -- ~/.virtualenvs/tango9/bin/python
cachedir: .cache
rootdir: /home/vinmic/pytango, inifile: 
collected 3 items 

test/test_server.py::test_empty_device PASSED
test/test_server.py::test_set_state PASSED
test/test_server.py::test_set_status PASSED

============================================= 3 passed in 10.20 seconds =============================================

As you can see the tests are really slow. It turns out the first test lasts a few milliseconds whereas the following tests last 5 seconds. After investigation, I realized the device server takes exactly 5 seconds to start for the last two tests. Does anyone have an explanation for that?

@vxgmichel

This comment has been minimized.

Copy link
Contributor Author

vxgmichel commented Oct 10, 2016

@vxgmichel

This comment has been minimized.

Copy link
Contributor Author

vxgmichel commented Oct 10, 2016

I'm now using the --boxed option of pytest-xdist to run each test in its own subprocess. The 16 tests are running in 0.58 s:

(tango9-python3) ~/pytango $ python setup.py test --addopts -v
running pytest
running egg_info
writing dependency_links to pytango.egg-info/dependency_links.txt
writing requirements to pytango.egg-info/requires.txt
writing pytango.egg-info/PKG-INFO
writing top-level names to pytango.egg-info/top_level.txt
reading manifest file 'pytango.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
running build_ext
copying build/lib.linux-x86_64-3.4/tango/_tango.cpython-34m.so -> tango
================================================ test session starts ================================================
platform linux -- Python 3.4.3, pytest-3.0.3, py-1.4.31, pluggy-0.4.0 -- ~/.virtualenvs/tango9-python3/bin/python
cachedir: .cache
rootdir: ~/pytango, inifile: pytest.ini
plugins: xdist-1.15.0
collected 16 items 

test/test_server.py::test_empty_device PASSED
test/test_server.py::test_set_state[MOVING] PASSED
test/test_server.py::test_set_state[ON] PASSED
test/test_server.py::test_set_state[FAULT] PASSED
test/test_server.py::test_set_state[DISABLE] PASSED
test/test_server.py::test_set_state[INSERT] PASSED
test/test_server.py::test_set_state[EXTRACT] PASSED
test/test_server.py::test_set_state[OFF] PASSED
test/test_server.py::test_set_state[UNKNOWN] PASSED
test/test_server.py::test_set_state[ALARM] PASSED
test/test_server.py::test_set_state[RUNNING] PASSED
test/test_server.py::test_set_state[STANDBY] PASSED
test/test_server.py::test_set_state[CLOSE] PASSED
test/test_server.py::test_set_state[INIT] PASSED
test/test_server.py::test_set_state[OPEN] PASSED
test/test_server.py::test_set_status PASSED

============================================= 16 passed in 0.58 seconds =============================================

@vxgmichel vxgmichel force-pushed the unit-test branch 2 times, most recently from 48f6b30 to a767505 Oct 11, 2016

@vxgmichel vxgmichel force-pushed the unit-test branch from a767505 to 4fb9edf Oct 11, 2016

@vxgmichel vxgmichel force-pushed the unit-test branch 4 times, most recently from 9ab57eb to 33e3dd5 Oct 11, 2016

@vxgmichel vxgmichel force-pushed the unit-test branch 6 times, most recently from 0048038 to beb7290 Oct 11, 2016

@vxgmichel vxgmichel force-pushed the unit-test branch 2 times, most recently from 1aefc0e to 3c9b24b Oct 11, 2016

@vxgmichel vxgmichel force-pushed the unit-test branch from 3c9b24b to 20a8bbb Oct 11, 2016

@johanfforsberg

This comment has been minimized.

Copy link

johanfforsberg commented Oct 12, 2016

Excellent stuff!

@johanfforsberg

This comment has been minimized.

Copy link

johanfforsberg commented Oct 12, 2016

What is the reason to put the utils in the tango package but keep the tests outside?

I am not sure of the names "test_utils.py" and "test_context.py", they suggest that the files contain tests and might be picked up by test runners as such, right?

@vxgmichel

This comment has been minimized.

Copy link
Contributor Author

vxgmichel commented Oct 12, 2016

@johanfforsberg Well, I used the test structure of asyncio as a reference. I think it makes sense to keep the actual tests out of the package, but I know it is something that is often debated. About the names, it should be fine since we restrict the pytest discovery to the test/ directory, but I don't mind changing them if you have something nicer.

vxgmichel added some commits Oct 12, 2016

@johanfforsberg
Copy link

johanfforsberg left a comment

👍


# Pytest fixtures
from tango.test_utils import state, typed_values, server_green_mode
state, typed_values, server_green_mode

This comment has been minimized.

@johanfforsberg

This comment has been minimized.

@vxgmichel

vxgmichel Oct 14, 2016

Author Contributor

I didn't want Flake-8 to complain about unused imports, but there's probably a better way.

@johanfforsberg johanfforsberg merged commit e5ec7c3 into develop Oct 17, 2016

1 check passed

continuous-integration/travis-ci/push The Travis CI build passed
Details

@vxgmichel vxgmichel deleted the unit-test branch Dec 20, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment