Add Collections#shuffle(List) to forbidden APIs

[jasontedor]

Several tests are using Collections#shuffle(List) which uses a default source of randomness rather than using Collections#shuffle(List, Random) which enables a reproducible source of randomness. This prevents reproducible build failures. One example is NodeJoinControllerTests but there are many others.

The method Collections#shuffle(List) should be added to forbidden APIs.

[javanna]

+1

[nik9000]

The method Collections#shuffle(List) should be added to forbidden APIs.

The list for tests or the list for all classes?

[jasontedor]

The list for tests or the list for all classes?

I intend to completely forbid it lest there still remain non-reproducible sources of randomness. I discussed this with @rmuir and we have a path forward.

[rmuir]

its the only way. if something is non-test and uses random behavior, give it an optional setting like initialSeed and set it in the test framework.

[nik9000]

its the only way. if something is non-test and uses random behavior, give it an optional setting like initialSeed and set it in the test framework.

I figured as much but its nice to have it confirmed. IIRC its used in a few places.

[jasontedor]

This method is also used inside ESTestCase#randomSubsetOf greatly increasing the surface area of non-reproducible tests.