Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

nosetests returns zero exit code while there are many test failures (Py3.x, Ubuntu, Travis CI) #984

Open
vasily-v-ryabov opened this issue Feb 19, 2016 · 0 comments
Open

nosetests returns zero exit code while there are many test failures (Py3.x, Ubuntu, Travis CI) #984

vasily-v-ryabov opened this issue Feb 19, 2016 · 0 comments

Comments

@vasily-v-ryabov
Copy link

Hi team! I'm trying to understand what's happening in a Travis CI build while running nosetests on Python 3.x (not sure it's py3 specific issue because py2 tests pass 100%).

Python 3.x tests are failing but the job is indicated as green and I see the following message:

The command "nosetests --with-xunit --with-coverage --cover-html --cover-html-dir=Coverage_report --verbosity=3 test/testall.py" exited with 0.

I've already tried to reduce the number of arguments up to nosetests test/testall.py with no success. One more strange thing that is not reproduced on local Ubuntu VM (with py2.7) is second message about strange 1 test that is OK. I don't know from where this magic test has been appeared as a separate test.

Ran 415 tests in 0.263s
FAILED (failures=26, errors=225)
ok

Ran 1 test in 0.467s  # ?????
OK

This magic test might be a root cause. I have no ideas how to fix that. The GitHub project is here.
openstack-gerrit pushed a commit to openstack/horizon that referenced this issue Jun 8, 2018
@e0ne
Use default Django test runner instead of nose
1f80d94 
Nose has been in maintenance mode for the past several years. It has
issue with exit code [1] which leads to false positive results for our
seleniun-headless job.

This patch changes test runner for Horizon tests and does the following
things:

* Django test runner  executes test in a different order than Nose does.
  That's why we've got an issue with side-effect in
  horizon.tests.unit.tables.test_tables.MyToggleAction class. This patch
  adds workaround to it.
* Rename filename of test files to names starting with 'test_'
  so that the django test runner can find tests expectedly.
* '--with-html-output' option is temporary dropped and will be added in
  a following patch.
* Integraion tests is marked via django.test.tag mechanism which is
  introduced in Django 1.10
* 'selenium-headless' is broken now because we don't have geckodriver on
  gates, this patch makes it non-voting.
* 'tox -e cover' is fixed
* Remove @Memorized decorator from
  dashboards.project.images.images.tables.filter_tenant_ids function.

[1] nose-devs/nose#984

Depends-On: https://review.openstack.org/572095
Depends-On: https://review.openstack.org/572124
Depends-On: https://review.openstack.org/572390
Depends-On: https://review.openstack.org/572391

Related blueprint: improve-horizon-testing
Change-Id: I7fb2fd7dd40f301ea822154b9809a9a07610c507
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@vasily-v-ryabov