Sort tests

[mvidner]

https://bugzilla.opensuse.org/show_bug.cgi?id=990267

[mvidner]

An alternative is to use a random but deterministic order, with rspec --order random ... which will print out the random seed used and allow reproducing the problem.

But I think such randomization should only be used on a noncritical build path, similarly to what we do with -Werror for C/++ packages.

[mvidner]

An example of a fix of order-dependent test bug is yast/yast-network#421

[ancorgs]

Should we really do this or let the tests fail and fix them one by one? They are not supposed to be order-dependent.

[ancorgs]

In other words, I would vote for the alternative proposed in @mvidner's comment.

[mvidner]

Now I realize that it should have been debuggable even before, because rake prints out how it calls rspec.

The main problem with the randomized tests is that they tend to break at inopportune moments, and it will put the team in a situation where you are fixing a bug that a colleague made but you can't even tell them "c'mon you should've ran the tests" because they did. And the benefit is IMHO small.

[jreidinger]

@mvidner I disagree, as order dependent tests also break when you work locally with given test. And when bootloader failed, I also rerun test with exact order written in travis and see if it can be reproduced. so also 👎 for this change.

[jreidinger]

@mvidner @ancorgs so as it is already written out how exactly it is ran, I think alternative is already implemented, so are you OK with closing this PR?

[mvidner]

I still disagree and think this PR is valid, but let's close it now and argue when/if the problem appears again.