Pytest target #1493

joshmoore · 2013-09-13T11:17:15Z

Enable all pytests (OmeroPy/test) including gatewaytests via the "integration" target. This means all of the following should now run Python:

 ome.git$ ./build.py test-integration
 ome.git$ cd components/tools/OmeroPy
 OmeroPy$ ../../../build.py integration
 OmeroPy$ ../../../build.py python-integration

bpindelski · 2013-09-13T14:22:53Z

@joshmoore If you think this PR fixes http://trac.openmicroscopy.org.uk/ome/ticket/11431, could the ticket be closed? If there is any more work needed, I'm happy to contribute - please add to the ticket. Thanks.

joshmoore · 2013-09-13T15:09:54Z

Agreed. Closed the ticket. A looming question: how do we select unit tests? /cc @ximenesuk

ximenesuk · 2013-09-14T07:54:42Z

The simplest way would be to include unit tests in a specific directory under OmeroPy/test. It could also be done using naming or markers but that is probably more complex.

ximenesuk · 2013-09-16T14:30:53Z

Testing this:

ome.git$ ./build.py test-integration

results in the following usage message

jrs-macbookpro-25031:ome cblackburn$ ./build.py test-integration
Buildfile: /Users/cblackburn/Work/repos/ome/build.xml

check-ivy:
:: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ ::
:: loading settings :: file = /Users/cblackburn/Work/repos/ome/etc/ivysettings.xml

test-integration:
:: loading settings :: file = /Users/cblackburn/Work/repos/ome/etc/ivysettings.xml
Entering /Users/cblackburn/Work/repos/ome/components/xsd-fu...
Entering /Users/cblackburn/Work/repos/ome/components/tools/OmeroCpp...
Entering /Users/cblackburn/Work/repos/ome/components/tools/OmeroFS...

python-integration:
WARNING: '' not a valid package name; please use only.-separated package names in setup.py
WARNING: '' not a valid package name; please use only.-separated package names in setup.py
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help
error: option --junitxml not recognized
Result: 1
Entering /Users/cblackburn/Work/repos/ome/components/tools/OmeroPy...

the tests then proceed to run.

 ome.git$ cd components/tools/OmeroPy
 OmeroPy$ ../../../build.py integration
 OmeroPy$ ../../../build.py python-integration

both run the tests without the usage message. I've not yet run any integration tests to conclusion to confirm whether a junitxml file is produced in ether case.

OmeroFS and all other python-based tools components will need the PyTest class as a command in their setup.py.

joshmoore · 2013-09-16T21:09:32Z

With the latest two commits, most of this is fixed. However, OmeroFS is going to require more work, that may or may not be doable here.

Move unit tests

components/tools/OmeroPy$ ./setup.py test -s test/unit/
...
======================================================= test session starts =======================================================
platform linux2 -- Python 2.7.4 -- pytest-2.3.5
collected 89 items 

test/unit/test_clients.py .......
test/unit/test_config.py ...........
test/unit/test_ext.py .
test/unit/test_model.py .......................
test/unit/test_parameters.py ......................
test/unit/test_permissions.py ...
test/unit/test_rtypes.py ...........
test/unit/test_tempfiles.py ...........

==================================================== 89 passed in 0.36 seconds ====================================================
(ome1)jamoore@blue:/opt/ome2/components/tools/OmeroPy$

Refactoring PyTest

...components/tools/OmeroFS$ ../../../build.py python-integration

============================= test session starts ==============================
platform linux2 -- Python 2.7.4 -- pytest-2.3.5 -- /opt/venv/ome1/bin/python
collecting ... collected 0 items
 generated xml file: /opt/ome2/components/tools/OmeroFS/target/reports/junit-results.xml 
===============================  in 0.00 seconds ===============================

joshmoore · 2013-09-17T08:26:51Z

Moved final test setups to PyTest. Only remaining files that won't be run by PyTest are:

components/tools/OmeroPy$ git ls-files test | grep -v test_ | grep -v __init__
test/gatewaytest/conftest.py
test/gatewaytest/library.py
test/integration/client_ctors.cfg
test/integration/helpers.py
test/integration/library.py
test/integration/scriptsharness/definition.cfg
test/integration/scriptsharness/definition.py
test/integration/scriptsharness/harness.py
test/integration/scriptsharness/simple_script.cfg
test/integration/scriptsharness/simple_script.py
test/integration/tablestest/service-reference-dev_4_4_5.h5.bz2
test/unit/clitest/mocks.py
test/unit/tablestest/library.py

Of these, only scriptsharness should need further work.

joshmoore · 2013-09-17T09:54:08Z

With the last commit, unit tests are now run properly by build.py. Both with and without -DTEST=somepath:

/opt/ome2/components/tools/OmeroPy$ ../../../build.py python-test -DTEST=test/unit/clitest/test_import.py 
Buildfile: /opt/ome2/components/tools/OmeroPy/build.xml
Entering /opt/ome2/components/tools/OmeroPy...

python-test:
Entering /opt/ome2/components/tools/OmeroPy...

python-test-single:
...
============================= test session starts ==============================
platform linux2 -- Python 2.7.4 -- pytest-2.3.5 -- /opt/venv/ome1/bin/python
collecting ... collected 1 items
test/unit/clitest/test_import.py:31: TestImport.testDropBoxArgs PASSED
 generated xml file: /opt/ome2/components/tools/OmeroPy/target/reports/junit-results.xml 
=========================== 1 passed in 0.15 seconds ===========================
Entering /opt/ome2/components/tools/OmeroPy...

BUILD SUCCESSFUL
Total time: 0 seconds

joshmoore · 2013-09-17T09:56:57Z

Unless there's something very grievous here, I'm assuming that's it for me. Happy to have this merged and then pass off to @ximenesuk for either fixing or marking the failing unit tests.

joshmoore · 2013-09-17T09:58:22Z

Note: I also didn't move around the OmeroFS test files which will need to be done to not have them missed by setup.py test -s test/

ximenesuk · 2013-09-17T12:33:21Z

I noticed in testing this that:

../../../build.py python-integration

seemed to also run the tests under OmeroPy/test/unit

joshmoore · 2013-09-17T13:48:28Z

That's always been my preference so that the complete suite on integration. Open to suggestions.

ximenesuk · 2013-09-17T13:55:15Z

I assume you mean, "... so that the complete suite is run on integration." If so that's fine. I just found the name a little confusing. I guess I would have expected something like python-test, python-unit and python-integration where the first is a union of the second two. But as long as the targets are (eventually) documented somewhere I'm easy.

joshmoore · 2013-09-17T14:17:18Z

Agreed. (Along with the naming of the junit results file discussed today with @bpindelski)

joshmoore · 2013-09-18T07:59:36Z

Jabber-based 👍 from @ximenesuk

Pytest target

ximenesuk · 2013-09-18T16:08:44Z

--rebased-to #1524

joshmoore and others added 3 commits September 13, 2013 13:17

Enable pytest via "integration" target

8b8cfe7

Add further py.test arguments to setup.py

c684f6c

Enable junitxml under testreports.dir

60941c0

joshmoore added 2 commits September 16, 2013 23:07

Move unit tests to OmeroPy/test/unit

6973df5

Refactor PyTest out to tools/test_setup.py

17b18f2

OmeroFS and all other python-based tools components will need the PyTest class as a command in their setup.py.

joshmoore added 7 commits September 17, 2013 09:41

Migrate clitests to PyTest

2075b69

Migrate cmdtest to PyTest

85548cd

Migrate tablestest/setup.py to PyTest

5c3706e

Migrate tablestest.integration_suite to PyTest

e277ed6

Migrate scriptstest/suite.py to PyTest

5405090

Migrate scriptstest/integration_suite.py to PyTest

64d2ece

Migrate PythonOnly to PyTest

9bcb4bf

joshmoore added 2 commits September 17, 2013 10:27

Remove integration_suite.py

cf2652c

Fix unit test execution

3b3ab31

joshmoore added a commit that referenced this pull request Sep 18, 2013

Merge pull request #1493 from joshmoore/pytest-target

f08f73d

Pytest target

joshmoore merged commit f08f73d into ome:dev_4_4 Sep 18, 2013

This was referenced Sep 18, 2013

Pytest target (rebased onto develop) #1523

Closed

Python integration test fixes, pytest and target (rebased onto develop) #1524

Merged

joshmoore deleted the pytest-target branch January 23, 2014 19:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pytest target #1493

Pytest target #1493

joshmoore commented Sep 13, 2013

bpindelski commented Sep 13, 2013

joshmoore commented Sep 13, 2013

ximenesuk commented Sep 14, 2013

ximenesuk commented Sep 16, 2013

joshmoore commented Sep 16, 2013

joshmoore commented Sep 17, 2013

joshmoore commented Sep 17, 2013

joshmoore commented Sep 17, 2013

joshmoore commented Sep 17, 2013

ximenesuk commented Sep 17, 2013

joshmoore commented Sep 17, 2013

ximenesuk commented Sep 17, 2013

joshmoore commented Sep 17, 2013

joshmoore commented Sep 18, 2013

ximenesuk commented Sep 18, 2013

Pytest target #1493

Pytest target #1493

Conversation

joshmoore commented Sep 13, 2013

bpindelski commented Sep 13, 2013

joshmoore commented Sep 13, 2013

ximenesuk commented Sep 14, 2013

ximenesuk commented Sep 16, 2013

joshmoore commented Sep 16, 2013

Move unit tests

Refactoring PyTest

joshmoore commented Sep 17, 2013

joshmoore commented Sep 17, 2013

joshmoore commented Sep 17, 2013

joshmoore commented Sep 17, 2013

ximenesuk commented Sep 17, 2013

joshmoore commented Sep 17, 2013

ximenesuk commented Sep 17, 2013

joshmoore commented Sep 17, 2013

joshmoore commented Sep 18, 2013

ximenesuk commented Sep 18, 2013