document random reproducibility policy #33350
Conversation
stevengj
added
domain:docs
|
Thanks for writing this up. Do you think we can/should make a commitment to reproducibility across OSs/architectures? The native Julia code should have this property (though we should be careful about endianness if the issue arises), but I don't know if the dSMFT code can guarantee this. |
|
I would hope that the MT code would guarantee portability, since it is supposed to be following a specific number-theoretic sequence... |
|
Should be good to merge? CI failure ( |
|
cc @staticfloat for the win failure |
Yes, but perhaps higher-level rand functions might be architecture-specific. Would it be better to explicitly state policy w.r.t architecture? |
|
|
||
| Software tests that rely on *specific* "random" data should also generally save the data or embed it into the test code. On the other hand, tests that should pass for *most* random data (e.g. testing `A \ (A*x) ≈ x` for a random matrix `A = randn(n,n)`) can use an RNG with a fixed seed to ensure that simply running the test many times does not encounter a failure due to very improbable data (e.g. an extremely ill-conditioned matrix). | ||
|
|
||
| The statistical *distribution* from which random samples are drawn *is* guaranteed to be the same across any minor Julia releases. |
There was a problem hiding this comment.
Would this rule out the possibility to change the distribution of rand(::Type{<:AbstractFloat}) from uniform in [0,1) into uniform in (0,1) which is currently discussed? or is this a "minor" enough change?
There was a problem hiding this comment.
In measure theoretic terms, those are technically the same distribution 😬
See e.g. discussion in #30494 and on discourse.
See also https://numpy.org/neps/nep-0019-rng-policy.html for a similar discussion by the NumPy developers.