Release 4.5.2 broke our tests vs 4.5.1

[Safihre]

We suddenly have various failures after our CI pipline started using 4.5.2 instead of 4.5.1.
I don't even know where to start to find the cause. It doesn't seem to be OS-dependent, since some Linux build fail, but not all. The error on Windows is different than on Linux and macOS succeeds.
No code was changed..

Yesterday: https://github.com/sabnzbd/sabnzbd/actions/runs/1431017533
Today: https://github.com/sabnzbd/sabnzbd/actions/runs/1434684518

(cc @jcfp)

[mrbean-bremen]

Thank you for the report. I just found the problem - it is not actually a bug, but is due to a behavior change of os.listdir in pyfakefs. From the release notes:

os.listdir, os.scandir and pathlib.Path.listdir now return the directory list in a random order (see #638)

This has been requested as the order is not fixed in the real file system, but may lead to failed tests, if they relied on thi behavior. I just disabled this change in a pyfakefs branch and directed a build of your repo (e.g. a fork) to this branch. As you can see, the builds pass in this case.

I now realize that such a behavior change should not have been in a bug fix release, and probably should be a opt-in configuration option, at least for some time.

[Safihre]

Thanks a lot for looking into this in detail, I was a bit lost where to start!
Of course we can sort our listdir results to fix our tests with the new code. But indeed, our assumption of sorted listdir is only used in the tests, not in the functions being tested.

[mrbean-bremen]

Shall be fixed now in version 4.5.3, please check. It makes sense to sort the results in your tests anyway, as in some future version the unsorted output will be the default (as it matches the real fs).

[Safihre]

Thank you, it works 👍 👍