forked from edgewall/trac
/
TESTING-README
126 lines (101 loc) · 5.22 KB
/
TESTING-README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
= Testing Readme =
So, you want to see what's broken? Eeeexcellent.
If you are running python < 2.4.4, please see the troubleshooting section.
== Quick Start ==
First thing to do is run the tests.
If you have genshi and twill installed on your system, you should be able to run the tests like this:
{{{
PYTHONPATH=. ./trac/test.py
}}}
== Slow Start ==
If you want to test against specific versions of genshi, twill, pygments, etc., you can set those up like this:
{{{
myworktree/trac
/pygments-0.8
/twill-0.9
/genshi-0.4.4
}}}
Run `python setup egg_info` in those subdirectories as needed.
Then you can run:
{{{
PYTHONPATH=.:../twill-0.9:../genshi-0.4.4:../pygments-0.8 ./trac/test.py
}}}
If you want to run just the functional tests, you can do that by running
{{{
PYTHONPATH=.:../twill-0.9:../genshi-0.4.4:../pygments-0.8 ./trac/tests/functional/testcases.py
}}}
And to run everything except the functional tests,
{{{
PYTHONPATH=. ./trac/test.py --skip-functional-tests
}}}
NOTE: Unlike most unittests, the functional tests share a test fixture across tests. This means that you can't(*) run just one of the tests by itself.
But you can run a sub-set of the functional tests:
{{{
PYTHONPATH=.:../twill-0.9:../genshi-0.4.4:../pygments-0.8 ./trac/tests/functional/__init__.py
PYTHONPATH=.:../twill-0.9:../genshi-0.4.4:../pygments-0.8 ./trac/ticket/tests/functional.py
PYTHONPATH=.:../twill-0.9:../genshi-0.4.4:../pygments-0.8 ./trac/versioncontrol/tests/functional.py
}}}
Note that that there is a set of functional tests that are run regardless of what subset you choose; these tests setup the shared fixture.
(*) Unless you modify the code to comment out the other functional tests. The test fixture is setup and torn down by `FunctionalTestSuite`, and it runs the tests added to it with `_tester` and `_testenv` set in the testcase objects.
The functional tests require subversion, and use a random local port 8000-8999 for the test web server.
== Testing output and byproducts ==
There is some logging done:
- testing.log
output from trac environment creation, tracd, and some svn commands
- functional-testing.log
output from twill
The test fixture is left behind in 'testenv' so you can inspect it when debugging a problem.
{{{
testenv/htpasswd -- the password/authentication for the test fixture. password = username
/repo -- the Subversion repository
/trac -- the Trac environment
}}}
(Note that running the tests again will automatically delete this test environment and create a new one. If you want to save a test environment, you will need to rename this directory before running the tests again.)
The command to serve the test environment is:
{{{
PYTHONPATH=.:../twill-0.9:../genshi-0.4.4:../pygments-0.8 \
./trac/web/standalone.py --basic-auth=trac,testenv/htpasswd, -s \
--port=8888 testenv/trac
}}}
This is particularly useful when a test fails and you want to explore the environment to debug the problem.
== Test Coverage ==
You can now determine statement coverage of unittests and functional tests. But you'll have to run them separately for now.
`figleaf` will need to be on your `PATH`. Set the `FIGLEAF` environment variable to the figleaf command.
Create a figleaf-exclude file with filename patterns to exclude. For example:
{{{
/usr/lib/python.*/.*.py
.*.html
}}}
Then run the tests something like this:
{{{
export FIGLEAF=figleaf
figleaf ./trac/test.py -v --skip-functional-tests
mv .figleaf .figleaf.unittests
python trac/tests/functional/testcases.py -v
mv .figleaf .figleaf.functional
figleaf2html --exclude-patterns=../figleaf-exclude .figleaf.functional .figleaf.unittests
}}}
Also, this is very slow; on a decent machine, 10 minutes for the functional tests is normal.
To run without figleaf, be sure to unset `FIGLEAF`.
--------------------------------------------------------------------------------
== TROUBLESHOOTING: ==
1. trac-admin is failing on initenv with this exception:
{{{
raise Exception('Failed with exitcode %s running trac-admin with %r' % (retval, args))
Exception: Failed with exitcode 1 running trac-admin with ('initenv', 'testenv', 'sqlite:db/trac.db', 'svn', '..../testenv/repo')
}}}
This can be caused by not having run `python setup.py egg_info` in the genshi tree.
2. Windows needs an implementation of crypt or fcrypt. Carey Evans' pure
python version works, but prints warnings on Python 2.3 (they can be
ignored). See http://carey.geek.nz/code/python-fcrypt/
3. Python versions compatibility notes
* If using Python >=2.4.0, <2.4.4, you need to backport `unittest` from
2.4.4. If you do not do this, you will get an `AttributeError` regarding
`_fixture` being unset on every test.
* If using Python >=2.3.0, <2.3.5, you need to backport a few modules for
`setuptools` to work properly (`httplib2`, `cookielib`, `_*CookieJar`) and
remove the tuple imports to replace with explicit line continuations.
* If using Python >=2.3.0, <2.4.0, you need `subprocess`, `unittest`, and
`traceback` from 2.4.4. On Windows you also need to modify `subprocess.py`
to use `pywin32` instead of `_subprocess`. Twill includes a `subprocess`
module that will not work in this situation because it is not modified.