Test suite for std.random

[dlangBugzillaToGithub]

joseph.wakeling (@WebDrake) reported this on 2013-07-08T06:42:39Z

Transfered from https://issues.dlang.org/show_bug.cgi?id=10572

CC List

• bugzilla (@WalterBright)

Description

Random number generation in Phobos needs an effective battery of tests to ensure that users are being delivered a solution of sufficient statistical quality.

Part of this can be done in unittests, e.g. testing PRNGs' output against sequences from reference implementations.  However, more rigorous tests of randomness require extensive computation and would be too time- and resource-intensive for inclusion in the standard Phobos test suite.

So, I propose the creation of an independent test suite specifically for use with std.random.  To the greatest possible degree this suite should re-use existing established tools rather than re-implementing known tests.  Some examples include:

  * TestU01 <https://en.wikipedia.org/wiki/TestU01>, an extensive suite of tests which can be applied in varying degrees of intensity (from SmallCrush, 10 tests which take a total of about 2 mins, to BigCrush, 106 tests which take a total of about 12 hours).

  * Dieharder <http://www.phy.duke.edu/~rgb/General/dieharder.php>, a successor to Marsiglia's Diehard battery of tests which is readily available in Linux distros.

The suite could also include a number of simpler "eye" tests which allow developers to check that their work does not introduce "obvious" departures from statistical correctness.

Its purpose should be doubly to allow Phobos contributors to make fast working checks on the statistical validity of their code while developing or debugging, and to allow Phobos maintainers to enforce rigorous quality standards when accepting changes to std.random.

[dlangBugzillaToGithub]

joseph.wakeling commented on 2013-07-08T06:51:08Z

As part of the work of building this suite, I suggest we take the opportunity to engage in some community-building.  We should aim to engage a network of experts who are sympathetic to our efforts to construct a suite of high-quality random number tools.  In turn we should aim to serve these experts via the D language -- it would be good to see D being adopted as the computational lingua franca in these areas of research work.