Use a System wide property to enable PRS

[noblepaul]

This is to clean up the JUnit tests. We want all tests to use PRS randomly

[dsmiley]

I believe this has caused NullPointerExceptions in OverseerStatusTest -- createcollection is null. Quite a number of test failures for this since this was merged.

[dsmiley]

BTW it's nice to see PRS used/tested more :-)

[dsmiley]

http://fucit.org/solr-jenkins-reports/failure-report.html 8% is considerable. Not sure if the other failures are related. Is this a tough one to figure out @noblepaul ?

[dsmiley]

TestLocalFSCloudBackupRestore failures seems plausibly related as its failures begin at the same time:
https://ge.apache.org/scans/tests?search.relativeStartTime=P90D&search.rootProjectNames=solr-root&search.timeZoneId=America%2FNew_York&tests.container=*TestLocalFSCloudBackupRestore Maybe only 2-3% fail

[dsmiley]

And LocalFSCloudIncrementalBackupTest
https://ge.apache.org/scans/tests?search.relativeStartTime=P90D&search.rootProjectNames=solr-root&search.timeZoneId=America%2FNew_York&tests.container=org.apache.solr.cloud.api.collections.LocalFSCloudIncrementalBackupTest

Gradle Enterprise does a nice job of visualizing recent failure trends.

[dsmiley]

@noblepaul OverseerStatusTest still fails occasionally :-( A number of people have bumped into it and maybe wasted some time trying to figure out what happened. Something has to be done.

[gerlowskija]

+1 I spent a bunch of time on this this morning after running into OverseerStatusTest failures on main, before stumbling on this discussion. Any chance you're able to investigate further @noblepaul ?

[gus-asf]

Neither this PR nor the commit mention a jira so I'll drop this reproducing failure here: https://issues.apache.org/jira/browse/SOLR-17265

[HoustonPutman]

Ok created a PR to fix this, I'm not sure what the logic here was supposed to be, but I've changed PRS to be a much more normal per-test-class randomized option.

[HoustonPutman]

Why isn't any of this PRS testing logic backported to 9x @noblepaul ?

[dsmiley]

The approach seen here is perfect -- bake something risky in main then bring to branch_9x.

[HoustonPutman]

Fair enough. We will let it bake then backport both.