Pass DieHarder statistical/randomness tests #185

moodmosaic · 2018-04-25T00:29:08Z

Currently, most DieHarder tests are failing. This resolves #125 by passing all DieHarder tests 🎉 🎈

/cc @jystic @markhibberd @nhibberd @thumphries, not necessarily in this order!

moodmosaic · 2018-04-25T05:08:08Z

~~Closing this for now as there's some more work to be done, as seen from the failing check on Travis.~~

moodmosaic · 2018-04-25T18:56:17Z

Hmm... build currently fails on Travis CI, in hedgehog/test/Test/Hedgehog/Text.hs, on line 32:

Building test suite 'test' for hedgehog-0.5.3..
Running 1 test suites...
Test suite test: RUNNING...
━━━ Test.Hedgehog.Text ━━━
  ✗ 1/4 complete: 1 failed (running)
test: Enum.fromEnum{Word64}:
  value (18446744073709551615) is outside of Int's bounds
  (-9223372036854775808,9223372036854775807)

I can easily reproduce this in GHCi:

λ Gen.print (Gen.enumBounded :: Gen Word64)
=== Outcome ===
*** Exception: Enum.fromEnum{Word64}:
  value (18446744073709551615) is outside of Int's bounds
  (-9223372036854775808,9223372036854775807)

I'll investigate what might be causing this.

markhibberd · 2018-04-25T23:19:33Z

From the docs on max bounded:

Convert to an Int. It is implementation-dependent what fromEnum returns when applied to a value that is too large to fit in an Int.

For Word64 it isn't total, so it it looks like it is doing the right thing even if it is crap. enumBounded just isn't safe for Word64.

markhibberd · 2018-04-25T23:19:49Z

Thanks for the bit-masks in hex as well ;)

See also - http://gee.cs.oswego.edu/dl/papers/oopsla14.pdf - https://github.com/janestreet/splittable_random/blob/2a21d48eeb4eb4c92d8591ef0ba19c75dcd0885d/src/splittable_random.ml#L33-L34 - https://github.com/phadej/splitmix/blob/2b3945095ce9d907b802ecad63b803abf2d040db/src/System/Random/SplitMix.hs#L57-L58 - https://github.com/fscheck/FsCheck/blob/0a1d7c7e122a0b77346d150b66587bc34fc4054c/src/FsCheck/Random.fs#L30-L33 - http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8u40-b25/java/util/SplittableRandom.java Given the following program import Data.ByteString.Builder (Builder) import qualified Data.ByteString.Builder as Builder import System.IO (stdout) import Hedgehog.Internal.Seed (Seed(..), nextWord64, random) main :: IO () main = Builder.hPutBuilder stdout . go =<< random where go :: Seed -> Builder go s0 = case nextWord64 s0 of ~(v0, s1) -> Builder.word64LE v0 `mappend` go s1 and adding a test-suite in hedgehog.cabal test-suite dieharder type: exitcode-stdio-1.0 main-is: Dieharder.hs ghc-options: -Wall -threaded -O2 hs-source-dirs: test build-depends: hedgehog , base >= 3 && < 5 , bytestring >=0.10.4.0 && < 0.11 Running dieharder passes all tests, as shown in http://manpages.ubuntu.com/manpages/xenial/man1/dieharder.1.html#contenttoc6 #=============================================================================# # dieharder version 3.31.1 Copyright 2003 Robert G. Brown # #=============================================================================# rng_name |rands/second| Seed | stdin_input_raw| 3.73e+06 |3112223130| #=============================================================================# test_name |ntup| tsamples |psamples| p-value |Assessment #=============================================================================# diehard_birthdays| 0| 100| 100|0.34563424| PASSED diehard_operm5| 0| 1000000| 100|0.91055807| PASSED diehard_rank_32x32| 0| 40000| 100|0.54739259| PASSED diehard_rank_6x8| 0| 100000| 100|0.26505805| PASSED diehard_bitstream| 0| 2097152| 100|0.08284214| PASSED diehard_opso| 0| 2097152| 100|0.00752931| PASSED diehard_oqso| 0| 2097152| 100|0.99546974| WEAK diehard_dna| 0| 2097152| 100|0.17733131| PASSED diehard_count_1s_str| 0| 256000| 100|0.57339628| PASSED diehard_count_1s_byt| 0| 256000| 100|0.64992966| PASSED diehard_parking_lot| 0| 12000| 100|0.16680695| PASSED diehard_2dsphere| 2| 8000| 100|0.68902795| PASSED diehard_3dsphere| 3| 4000| 100|0.99992722| WEAK diehard_squeeze| 0| 100000| 100|0.94137755| PASSED diehard_sums| 0| 100| 100|0.50267983| PASSED diehard_runs| 0| 100000| 100|0.79817449| PASSED diehard_runs| 0| 100000| 100|0.21850798| PASSED diehard_craps| 0| 200000| 100|0.92958876| PASSED diehard_craps| 0| 200000| 100|0.62629655| PASSED marsaglia_tsang_gcd| 0| 10000000| 100|0.47671094| PASSED marsaglia_tsang_gcd| 0| 10000000| 100|0.42735871| PASSED sts_monobit| 1| 100000| 100|0.92317911| PASSED sts_runs| 2| 100000| 100|0.78842199| PASSED sts_serial| 1| 100000| 100|0.77489764| PASSED sts_serial| 2| 100000| 100|0.96104267| PASSED sts_serial| 3| 100000| 100|0.30179030| PASSED sts_serial| 3| 100000| 100|0.12929165| PASSED sts_serial| 4| 100000| 100|0.95686266| PASSED sts_serial| 4| 100000| 100|0.46981192| PASSED sts_serial| 5| 100000| 100|0.68755789| PASSED sts_serial| 5| 100000| 100|0.51597633| PASSED sts_serial| 6| 100000| 100|0.80927343| PASSED sts_serial| 6| 100000| 100|0.99992079| WEAK sts_serial| 7| 100000| 100|0.94245243| PASSED sts_serial| 7| 100000| 100|0.94625255| PASSED sts_serial| 8| 100000| 100|0.81975829| PASSED sts_serial| 8| 100000| 100|0.59703661| PASSED sts_serial| 9| 100000| 100|0.96646876| PASSED sts_serial| 9| 100000| 100|0.72616335| PASSED sts_serial| 10| 100000| 100|0.94215083| PASSED sts_serial| 10| 100000| 100|0.64812061| PASSED sts_serial| 11| 100000| 100|0.33255018| PASSED sts_serial| 11| 100000| 100|0.34827077| PASSED sts_serial| 12| 100000| 100|0.56818982| PASSED sts_serial| 12| 100000| 100|0.51835009| PASSED sts_serial| 13| 100000| 100|0.47037623| PASSED sts_serial| 13| 100000| 100|0.85810264| PASSED sts_serial| 14| 100000| 100|0.28607292| PASSED sts_serial| 14| 100000| 100|0.03414349| PASSED sts_serial| 15| 100000| 100|0.67423768| PASSED sts_serial| 15| 100000| 100|0.56303644| PASSED sts_serial| 16| 100000| 100|0.31462994| PASSED sts_serial| 16| 100000| 100|0.90478391| PASSED rgb_bitdist| 1| 100000| 100|0.95021276| PASSED rgb_bitdist| 2| 100000| 100|0.63340855| PASSED rgb_bitdist| 3| 100000| 100|0.94626759| PASSED rgb_bitdist| 4| 100000| 100|0.05343869| PASSED rgb_bitdist| 5| 100000| 100|0.05472197| PASSED rgb_bitdist| 6| 100000| 100|0.79748521| PASSED rgb_bitdist| 7| 100000| 100|0.53616218| PASSED rgb_bitdist| 8| 100000| 100|0.20352776| PASSED rgb_bitdist| 9| 100000| 100|0.46809570| PASSED rgb_bitdist| 10| 100000| 100|0.71940373| PASSED rgb_bitdist| 11| 100000| 100|0.05294303| PASSED rgb_bitdist| 12| 100000| 100|0.89183483| PASSED rgb_minimum_distance| 2| 10000| 1000|0.82703854| PASSED rgb_minimum_distance| 3| 10000| 1000|0.50017724| PASSED rgb_minimum_distance| 4| 10000| 1000|0.04900225| PASSED rgb_minimum_distance| 5| 10000| 1000|0.46461538| PASSED rgb_permutations| 2| 100000| 100|0.76572585| PASSED rgb_permutations| 3| 100000| 100|0.55150618| PASSED rgb_permutations| 4| 100000| 100|0.69989462| PASSED rgb_permutations| 5| 100000| 100|0.12805292| PASSED rgb_lagged_sum| 0| 1000000| 100|0.56055497| PASSED rgb_lagged_sum| 1| 1000000| 100|0.52548113| PASSED rgb_lagged_sum| 2| 1000000| 100|0.97868238| PASSED rgb_lagged_sum| 3| 1000000| 100|0.64319424| PASSED rgb_lagged_sum| 4| 1000000| 100|0.81184518| PASSED rgb_lagged_sum| 5| 1000000| 100|0.91426183| PASSED rgb_lagged_sum| 6| 1000000| 100|0.43130599| PASSED rgb_lagged_sum| 7| 1000000| 100|0.20481828| PASSED rgb_lagged_sum| 8| 1000000| 100|0.48736601| PASSED rgb_lagged_sum| 9| 1000000| 100|0.93720003| PASSED rgb_lagged_sum| 10| 1000000| 100|0.16670530| PASSED rgb_lagged_sum| 11| 1000000| 100|0.88148050| PASSED rgb_lagged_sum| 12| 1000000| 100|0.67551637| PASSED rgb_lagged_sum| 13| 1000000| 100|0.77455480| PASSED rgb_lagged_sum| 14| 1000000| 100|0.36246394| PASSED rgb_lagged_sum| 15| 1000000| 100|0.12116804| PASSED rgb_lagged_sum| 16| 1000000| 100|0.88373307| PASSED rgb_lagged_sum| 17| 1000000| 100|0.02209034| PASSED rgb_lagged_sum| 18| 1000000| 100|0.92576558| PASSED rgb_lagged_sum| 19| 1000000| 100|0.54040518| PASSED rgb_lagged_sum| 20| 1000000| 100|0.29900201| PASSED rgb_lagged_sum| 21| 1000000| 100|0.23348961| PASSED rgb_lagged_sum| 22| 1000000| 100|0.49675291| PASSED rgb_lagged_sum| 23| 1000000| 100|0.06393575| PASSED rgb_lagged_sum| 24| 1000000| 100|0.60661627| PASSED rgb_lagged_sum| 25| 1000000| 100|0.17661266| PASSED rgb_lagged_sum| 26| 1000000| 100|0.30398633| PASSED rgb_lagged_sum| 27| 1000000| 100|0.85102164| PASSED rgb_lagged_sum| 28| 1000000| 100|0.82477733| PASSED rgb_lagged_sum| 29| 1000000| 100|0.90980482| PASSED rgb_lagged_sum| 30| 1000000| 100|0.66624618| PASSED rgb_lagged_sum| 31| 1000000| 100|0.62473778| PASSED rgb_lagged_sum| 32| 1000000| 100|0.43643499| PASSED rgb_kstest_test| 0| 10000| 1000|0.53860519| PASSED dab_bytedistrib| 0| 51200000| 1|0.24021340| PASSED dab_dct| 256| 50000| 1|0.51809345| PASSED Preparing to run test 207. ntuple = 0 dab_filltree| 32| 15000000| 1|0.00818897| PASSED dab_filltree| 32| 15000000| 1|0.06469865| PASSED Preparing to run test 208. ntuple = 0 dab_filltree2| 0| 5000000| 1|0.81755887| PASSED dab_filltree2| 1| 5000000| 1|0.01399998| PASSED Preparing to run test 209. ntuple = 0 dab_monobit2| 12| 65000000| 1|0.92987365| PASSED

moodmosaic · 2018-04-26T07:53:26Z

@markhibberd, thanks! I force-pushed 2abbe3f, Travis CI build is now green.

moodmosaic · 2018-04-26T10:05:21Z

hedgehog/hedgehog.cabal

@@ -134,3 +134,18 @@ test-suite test
    , semigroups                      >= 0.16       && < 0.19
    , text                            >= 1.1        && < 1.3
    , transformers                    >= 0.3        && < 0.6
+
+executable dieharder-input


I've added this as executable because, as a test-suite, IIRC, Travis CI tries to run it and eventually it'll timeout as it takes ~25 minutes to finish.

This way, we can run the DieHarder tests on demand, as executable, via

dieharder-input | dieharder -a -g 200

(About those -a -g 200 command line arguments, see DieHarder manpage.)

@markhibberd

See https://github.com/moodmosaic/SplitMix/blob/ab042749b8338aff7e1cbcde97ce22bf121a0ca8/SplitMix.fsx This should keep @markhibberd happy :) hedgehogqa#124 (comment)

This is a stripped-down version of https://github.com/phadej/splitmix/blob/90265357da5c8660f1d488018743c9ef78a057ca/tests/Dieharder.hs Resolves hedgehogqa#125

moodmosaic · 2018-04-27T12:31:10Z

hedgehog-dieharder/hedgehog-dieharder.cabal

+  , GHC == 8.2.2
+  , GHC == 8.4.1
+
+executable dieharder-input


Can be run as

dieharder-input | dieharder -a -g 200

(About those -a -g 200 command line arguments, see DieHarder manpage.)

See hedgehogqa/haskell-hedgehog#185

moodmosaic closed this Apr 25, 2018

moodmosaic reopened this Apr 25, 2018

moodmosaic force-pushed the topic/seed branch 2 times, most recently from c83859c to 5e05736 Compare April 25, 2018 16:24

moodmosaic force-pushed the topic/seed branch from 5e05736 to 1e3022c Compare April 26, 2018 06:43

moodmosaic commented Apr 26, 2018

View reviewed changes

Don't write bit masks in decimal

84f75ec

See https://github.com/moodmosaic/SplitMix/blob/ab042749b8338aff7e1cbcde97ce22bf121a0ca8/SplitMix.fsx This should keep @markhibberd happy :) hedgehogqa#124 (comment)

moodmosaic force-pushed the topic/seed branch 2 times, most recently from 5b63b82 to e1d097b Compare April 27, 2018 10:08

Add hedgehog-dieharder package

aff0daa

This is a stripped-down version of https://github.com/phadej/splitmix/blob/90265357da5c8660f1d488018743c9ef78a057ca/tests/Dieharder.hs Resolves hedgehogqa#125

moodmosaic force-pushed the topic/seed branch from e1d097b to aff0daa Compare April 27, 2018 11:20

moodmosaic commented Apr 27, 2018

View reviewed changes

thumphries merged commit d97e846 into hedgehogqa:master May 1, 2018

moodmosaic deleted the topic/seed branch May 2, 2018 07:38

moodmosaic added a commit to moodmosaic/splitmix that referenced this pull request May 16, 2018

Sync with hedgehogqa/haskell-hedgehog#185

a5e53f7

moodmosaic added a commit to moodmosaic/fsharp-hedgehog that referenced this pull request May 28, 2018

Use 64-bit unsigned integers (UInt64) in Seed

d3d8967

See hedgehogqa/haskell-hedgehog#185

moodmosaic mentioned this pull request May 28, 2018

Sync F# version of Seed with Haskell hedgehogqa/fsharp-hedgehog#160

Merged

moodmosaic mentioned this pull request Jul 9, 2018

Enum.fromEnum{Int64} fails on 32-bit #132

Open

ghost mentioned this pull request Sep 20, 2021

Fix a few things in our Seed implementation hedgehogqa/fsharp-hedgehog#362

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pass DieHarder statistical/randomness tests #185

Pass DieHarder statistical/randomness tests #185

moodmosaic commented Apr 25, 2018

moodmosaic commented Apr 25, 2018 •

edited

Loading

moodmosaic commented Apr 25, 2018

markhibberd commented Apr 25, 2018

markhibberd commented Apr 25, 2018

moodmosaic commented Apr 26, 2018

moodmosaic Apr 26, 2018

moodmosaic Apr 27, 2018

Pass DieHarder statistical/randomness tests #185

Pass DieHarder statistical/randomness tests #185

Conversation

moodmosaic commented Apr 25, 2018

moodmosaic commented Apr 25, 2018 • edited Loading

moodmosaic commented Apr 25, 2018

markhibberd commented Apr 25, 2018

markhibberd commented Apr 25, 2018

moodmosaic commented Apr 26, 2018

moodmosaic Apr 26, 2018

Choose a reason for hiding this comment

moodmosaic Apr 27, 2018

Choose a reason for hiding this comment

moodmosaic commented Apr 25, 2018 •

edited

Loading