forked from CiscoSystems/neutron
/
TESTING
100 lines (67 loc) · 3.5 KB
/
TESTING
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
Testing Neutron
=============================================================
Overview
The unit tests are meant to cover as much code as possible and should
be executed without the service running. They are designed to test
the various pieces of the neutron tree to make sure any new changes
don't break existing functionality.
The functional tests are intended to validate actual system
interaction. Mocks should be used sparingly, if at all. Care
should be taken to ensure that existing system resources are not
modified and that resources created in tests are properly cleaned
up.
Running tests
There are two mechanisms for running tests: run_tests.sh and tox.
Before submitting a patch for review you should always ensure all
test pass; a tox run is triggered by the jenkins gate executed on
gerrit for each patch pushed for review.
With both mechanisms you can either run the tests in the standard
environment or create a virtual environment to run them in.
By default after running all of the tests, any pep8 errors
found in the tree will be reported.
Running individual tests
For running individual test modules or cases, you just need to pass
the dot-separated path to the module you want as an argument to it.
For executing a specific test case, specify the name of the test case
class separating it from the module path with a colon.
For example, the following would run only the JSONV2TestCase tests from
neutron/tests/unit/test_api_v2.py:
$ ./run_tests.sh neutron.tests.unit.test_api_v2:JSONV2TestCase
or
$ ./tox neutron.tests.unit.test_api_v2:JSONV2TestCase
Adding more tests
Neutron has a fast growing code base and there is plenty of areas that
need to be covered by unit and functional tests.
To get a grasp of the areas where tests are needed, you can check
current coverage by running:
$ ./run_tests.sh -c
Development process
It is expected that any new changes that are proposed for merge
come with tests for that feature or code area. Ideally any bugs
fixes that are submitted also have tests to prove that they stay
fixed! In addition, before proposing for merge, all of the
current tests should be passing.
Debugging
By default, calls to pdb.set_trace() will be ignored when tests
are run. For pdb statements to work, invoke run_tests as follows:
$ ./run_tests.sh -d [test module path]
It's possible to debug tests in a tox environment:
$ tox -e venv -- python -m testtools.run [test module path]
Tox-created virtual environments (venv's) can also be activated
after a tox run and reused for debugging:
$ tox -e venv
$ . .tox/venv/bin/activate
$ python -m testtools.run [test module path]
Tox packages and installs the neutron source tree in a given venv
on every invocation, but if modifications need to be made between
invocation (e.g. adding more pdb statements), it is recommended
that the source tree be installed in the venv in editable mode:
# run this only after activating the venv
$ pip install --editable .
Editable mode ensures that changes made to the source tree are
automatically reflected in the venv, and that such changes are not
overwritten during the next tox run.
Post-mortem debugging
Setting OS_POST_MORTEM_DEBUG=1 in the shell environment will ensure
that pdb.post_mortem() will be invoked on test failure:
$ OS_POST_MORTEM_DEBUG=1 ./run_tests.sh -d [test module path]