Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test command does not return failure code #891

Closed
cedk opened this issue Dec 16, 2016 · 5 comments
Closed

test command does not return failure code #891

cedk opened this issue Dec 16, 2016 · 5 comments

Comments

@cedk
Copy link
Contributor

@cedk cedk commented Dec 16, 2016

Since version 30.2.1, when running the command "python setup.py test" if the tests fails, the exit code is always 0.
I tracked the change of behavior to 53b47e1
We found the issue because our CI is no more reporting failure because we run the test with this command.
About #850, I think it is an expected behavior that if a command in a list fails that the following are not executed. So I think the revision should be reverted.

@cedk
Copy link
Contributor Author

@cedk cedk commented Dec 16, 2016

Also I think the new behaviour is in contradiction of the documentation:

running automated builds that need testing before they are deployed

from https://setuptools.readthedocs.io/en/latest/setuptools.html#test-build-package-and-run-a-unittest-suite

@jaraco
Copy link
Member

@jaraco jaraco commented Dec 16, 2016

About #850, I think it is an expected behavior that if a command in a list fails that the following are not executed. So I think the revision should be reverted.

But the point of #850 was that if a command doesn't fail, it should allow other commands to continue. Reverting that revision would lose that goal.

Probably what should happen is a failure should terminate the command processing, but a success should allow additional commands to proceed.

This also relates to #869 where it seems reasonable that a failed upload should have similar characteristics - short-circuit and exit with a non-zero status code on failure, but continue on success.

@jaraco
Copy link
Member

@jaraco jaraco commented Dec 16, 2016

I welcome someone to put together a pull request.

@pokoli
Copy link

@pokoli pokoli commented Dec 16, 2016

About #850 with the current behaviour running

python setup.py test build

will still build the packages with the test failing. The previous behaviour the packages where not build as they first step failed.

For me the previous behaviour is better as it will avoid the build of broken packages.

Don't know if the right solution is to return the exit code correctly for the test command and keep the current behaviour for others. But that's one idea.

@cedk
Copy link
Contributor Author

@cedk cedk commented Dec 16, 2016

OK, I will work on a PR that will stop only on failure.

cedk added a commit to cedk/setuptools that referenced this issue Dec 16, 2016
When test fails, it should not continue to run other commands.
Fixes pypa#891
@jaraco jaraco closed this in #892 Dec 16, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants