test command does not return failure code

[cedk]

Since version 30.2.1, when running the command "python setup.py test" if the tests fails, the exit code is always 0.
I tracked the change of behavior to 53b47e1
We found the issue because our CI is no more reporting failure because we run the test with this command.
About #850, I think it is an expected behavior that if a command in a list fails that the following are not executed. So I think the revision should be reverted.

[cedk]

Also I think the new behaviour is in contradiction of the documentation:

running automated builds that need testing before they are deployed

from https://setuptools.readthedocs.io/en/latest/setuptools.html#test-build-package-and-run-a-unittest-suite

[jaraco]

About #850, I think it is an expected behavior that if a command in a list fails that the following are not executed. So I think the revision should be reverted.

But the point of #850 was that if a command doesn't fail, it should allow other commands to continue. Reverting that revision would lose that goal.

Probably what should happen is a failure should terminate the command processing, but a success should allow additional commands to proceed.

This also relates to #869 where it seems reasonable that a failed upload should have similar characteristics - short-circuit and exit with a non-zero status code on failure, but continue on success.

[jaraco]

I welcome someone to put together a pull request.

[pokoli]

About #850 with the current behaviour running

python setup.py test build

will still build the packages with the test failing. The previous behaviour the packages where not build as they first step failed.

For me the previous behaviour is better as it will avoid the build of broken packages.

Don't know if the right solution is to return the exit code correctly for the test command and keep the current behaviour for others. But that's one idea.

[cedk]

OK, I will work on a PR that will stop only on failure.