libregrtest should always set random.seed

[sobolevn]

Feature or enhancement

While working on #110160 I've noticed that it is rather hard to reproduce random test failures. So, I want to propose a new feature / fix for that.

First of all, there's existing prior work of @vstinner who added --randseed flag.
Right now it is used together with -r to randomize test order and seeding random.

I propose to:

• Split --randseed into two options: test order randomization and random.seed usage
• Let's keep -r as-is
• Let's add --no-use-randseed flag to disable setting random.seed, let's always seed random by default
• Always print information about current random seed to be able to reuse it in the next runs

Example:

» ./python.exe -m test test_regrtest
Using random seed 65906482
0:00:00 load avg: 1.55 Run 1 test sequentially
0:00:00 load avg: 1.55 [1/1] test_regrtest

== Tests result: SUCCESS ==

1 test OK.

Total duration: 12.7 sec
Total tests: run=102 skipped=2
Total test files: run=1/1
Result: SUCCESS

Basically, this is how https://github.com/pytest-dev/pytest-randomly works.

I have a PR ready.

Linked PRs

• gh-110171: libregrtest always sets random.seed unless --no-use-randseed is provided #110172

[terryjreedy]

I think that the seed should be random but recorded so that a run can be reproduced.

[sobolevn]

Yes 👍
@terryjreedy please, check my PR.

[vstinner]

I had a similar idea.

I would prefer that --randomize keeps its current behavior: randomize the list of tests. It has a significant impact when tests are run sequentially.

Let's add --no-use-randseed flag to disable setting random.seed, let's always seed random by default

As an user, it's hard to understand this option. I don't understand the use case neither. Is it to increase the entropy of the initial seed? Recently, I increased the rand seed range to make it 10x larger :-)

I suggest to not add such confusing option. Just always set a random seed and display it.

[sobolevn]

I've checked all other Random() instances in Lib/test none of them need extra seeding.
Most of them are in test_random, which handles seeds correctly!

[vstinner]

I've checked all other Random() instances in Lib/test none of them need extra seeding.

Thanks for checking.