Add Py.test testing framework support #6730

Kojoley · Jul 12, 2016

This PR is part of preparation for migration to pytest

Kojoley · Jul 12, 2016

Missed several other places, will update PR soon.

story645 · Jul 20, 2016

Is this a continuation of #5325?

tacaswell · Jul 20, 2016

Also, please be aware of https://github.com/matplotlib/pytest-mpl

Kojoley · Aug 21, 2016

Pytest 3.0 released with this change:

yield-based tests are considered deprecated and will be removed in pytest-4.0.

😕

Kojoley · Aug 21, 2016

Coverage problem is because of if [[ $NOSE_ARGS="--with-coverage" ]]; then in .travis.yml. The expression should be either [ a=b ] or [[ a==b ]].

Kojoley · Aug 21, 2016

PR is now ready for review and merge.

Key notes:

Collection phase is long due to image_comparison decorator
(pytest runs setup_class during it)
matplotlib.test is not aware of pytest yet
pytest-xdist is disabled due to #6899
--collect-filter is blacklist by default (#6902 (comment))
--no-pep8 is implemented, but --pep8 is not

tacaswell · Aug 23, 2016

It looks like pytest is finding 5 more tests than nose so that is probably a victory.

It looks like pep8 is being run by pytest on travis, but not on appveyor, but that is ok for now.

There is a big CI hit time wise for this and we will have to (soon) pick some of the nose builds to drop.

Kojoley · Aug 23, 2016

It looks like pep8 is being run by pytest on travis, but not on appveyor, but that is ok for now.

I made this intentionally because we have separated pep8 only build on travis so there is no need to run them on all appveyor builds. Or I was wrong?

tacaswell · Aug 23, 2016

If anything pep8 should be skipped by travis pytest build. According to the handy output, it is only 45s which is not a big deal on top of our total runtime.

jenshnielsen · Aug 24, 2016

As far as I remember the main reason for splitting pep8 out in a separate build is not so much runtime but making it simpler to spot failures that are due to pep8 only

QuLogic · Nov 7, 2016

Should we backport this commit?

This only was crucial for pytest, while nose somehow is fine without the fix.

mdboom added the needs_review label Jul 12, 2016

Kojoley referenced this pull request Jul 12, 2016
Open
Migration to Py.test testing framework #6731
28 of 39 tasks complete

tacaswell added this to the 2.1 (next point release) milestone Jul 12, 2016

story645 added the Testing label Jul 20, 2016

Kojoley changed the title from Move out from nose specific things in tests to Add Py.test testing framework support Jul 23, 2016

Kojoley referenced this pull request Aug 3, 2016
Merged
support for updating axis ticks for categorical data #6889

tacaswell removed the needs_review label Aug 23, 2016

Kojoley	Pytest compatible `knownfailif` and `image_comparison` decorators	`e5ea9ee`
Kojoley	Do not rely on outer scope variables in `ImageComparisonTest.test.do_… …test` `do_test` function variables are changed by loop where function was creates. It will work as you expect only if you call yielded function immediately.	`903d118`
Kojoley	`Add support of nose setup/teardown to pytest collector`	`af4afe2`
Kojoley	Filter tests leaked from imported pylab in `test_basic`	`922d3ac`
Kojoley	`Implemented test filtering for pytest`	`6c94b47`
Kojoley	Added call to `_init_tests` in `pytest_configure`	`abd769d`
Kojoley	Added `--no-pep8` key for pytest	`dadeac9`

Kojoley	`Added travis and appveyor pytest builds`	Some checks were not successful 2 failing and 1 successful checks continuous-integration/appveyor/pr — AppVeyor build failed Details continuous-integration/travis-ci/pr — The Travis CI build failed Details coverage/coveralls — Coverage remained the same at 70.222% Details	`f20cc01`
Kojoley	A better way to deal with scope pollution in `test_basic` test	Some checks were not successful 2 failing and 1 successful checks continuous-integration/appveyor/pr — AppVeyor build failed Details continuous-integration/travis-ci/pr — The Travis CI build failed Details coverage/coveralls — Coverage increased (+0.002%) to 70.224% Details	`1f0792b`
Kojoley	Pytest `skipif` does not work on classes with generator methods The bug will not be fixed in pytest because generator functions are deprecated since pytest 3.0 release.	Some checks were not successful 1 failing and 2 successful checks coverage/coveralls — Coverage decreased (-9.2%) to 61.033% Details continuous-integration/appveyor/pr — AppVeyor build succeeded Details continuous-integration/travis-ci/pr — The Travis CI build passed Details	`a66d599`

matplotlib/matplotlib

Add Py.test testing framework support #6730

Kojoley commented Jul 12, 2016

mdboom added the needs_review label Jul 12, 2016

Kojoley referenced this pull request Jul 12, 2016

Migration to Py.test testing framework #6731

Kojoley commented Jul 12, 2016

tacaswell added this to the 2.1 (next point release) milestone Jul 12, 2016

story645 added the Testing label Jul 20, 2016

story645 commented Jul 20, 2016

tacaswell commented Jul 20, 2016

Kojoley changed the title from Move out from nose specific things in tests to Add Py.test testing framework support Jul 23, 2016

Kojoley referenced this pull request Aug 3, 2016

support for updating axis ticks for categorical data #6889

Kojoley added some commits Jul 10, 2016

Kojoley commented Aug 21, 2016

Kojoley added some commits Jul 25, 2016

Kojoley commented Aug 21, 2016

Kojoley commented Aug 21, 2016

tacaswell commented Aug 23, 2016

tacaswell merged commit `84f471e` into matplotlib:master Aug 23, 2016
2 of 3 checks passed

2 of 3 checks passed

tacaswell removed the needs_review label Aug 23, 2016

Kojoley commented Aug 23, 2016

tacaswell commented Aug 23, 2016

jenshnielsen commented Aug 24, 2016

QuLogic commented on `903d118` Nov 7, 2016

Kojoley replied Nov 8, 2016

matplotlib/matplotlib

Join GitHub today

Add Py.test testing framework support #6730

Conversation

Kojoley commented Jul 12, 2016

mdboom added the needs_review label Jul 12, 2016

Kojoley referenced this pull request Jul 12, 2016

Migration to Py.test testing framework #6731

Kojoley commented Jul 12, 2016

tacaswell added this to the 2.1 (next point release) milestone Jul 12, 2016

story645 added the Testing label Jul 20, 2016

story645 commented Jul 20, 2016

tacaswell commented Jul 20, 2016

Kojoley changed the title from Move out from nose specific things in tests to Add Py.test testing framework support Jul 23, 2016

Kojoley referenced this pull request Aug 3, 2016

support for updating axis ticks for categorical data #6889

Kojoley added some commits Jul 10, 2016

Kojoley commented Aug 21, 2016

Kojoley added some commits Jul 25, 2016

Some checks were not successful

Some checks were not successful

Some checks were not successful

Kojoley commented Aug 21, 2016

Kojoley commented Aug 21, 2016

tacaswell commented Aug 23, 2016

Hide details View details tacaswell merged commit 84f471e into matplotlib:master Aug 23, 2016 2 of 3 checks passed

2 of 3 checks passed

tacaswell removed the needs_review label Aug 23, 2016

Kojoley commented Aug 23, 2016

tacaswell commented Aug 23, 2016

jenshnielsen commented Aug 24, 2016

QuLogic commented on 903d118 Nov 7, 2016

Kojoley replied Nov 8, 2016

tacaswell merged commit `84f471e` into matplotlib:master Aug 23, 2016
2 of 3 checks passed

QuLogic commented on `903d118` Nov 7, 2016