-
-
Notifications
You must be signed in to change notification settings - Fork 31.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
unittest discover order is filesystem specific - hard to reproduce #60913
Comments
Openstack recently switched from nose to using discover. discover walks the filesystem using os.listdir(), and that is just a thin layer over readdir. On ext3/ext4 filesystems, readdir is in an arbitrary order dependent on file insertion into the directory if dir_index is enabled (which is the default). This means that files are loaded in an order that isn't reproducable by other developers, so bad tests that have isolation issues can be very tricky to track down. Just wrapping the os.listdir() in sorted() would be sufficient to make this robust and repeatable, and avoid the headache. |
+1. It also makes test output nicer. |
Seems reasonable. |
Should this be considered a new feature or should it be applied to older versions as well? |
It smells like a feature to me (it isn't a direct "bug" fix anyway). It can be applied to earlier versions of Python through a new unittest2 release. |
Not sure if there was anything more to it than this, but please find an attempt to add this attached. |
The patch looks good to me. |
As we're specifying this behaviour in the documentation it would be nice to test it. |
I've added tests for this behavior by un-sorting the test inputs for test_find_tests, and adding comments that the results should be sorted for reliable test execution. Attaching an updated patch. |
Fixed a spelling error. |
Fixed a spelling error. (Part deux) |
New changeset fe18f16dc2a6 by Michael Foord in branch 'default': |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: