math/rand: remove statistical correlation in ziggurat algorithms

[zephyrtronium]

A statistical correlation is present in the algorithms used to generate normally and
exponentially distributed values due to the reuse of random bits for both the
x-coordinate and the segment index. This reuse could be avoided by generating a 63-bit
number, taking the low 32 bits for the x-coordinate, the next 7 or 8 bits for the
segment number, and on the normal distribution, one more bit to determine the sign of
values generated from the tail.

For example: http://play.golang.org/p/xeCjwEFUSs

[ianlancetaylor]

Comment 1:

Thanks for the report.  Please send patches using the process described at
http://golang.org/doc/contribute.html.  Thanks.

Labels changed: added repo-main, release-go1.5.

[zephyrtronium]

Comment 2:

Alas, a complication: after making these changes and running all.bash, I see that there
are tests to make sure that the outputs of Rand.NormFloat64() and Rand.ExpFloat64() stay
consistent: a contract which this improvement necessarily violates.

[josharian]

Based on the discussion in #8013, I don't think we can't change this.

@ianlancetaylor what do you think?

[ianlancetaylor]

We deliberately postponed the decision in #8013. There were various arguments, but no final decision.

If the numbers that we get from the generators are statistically correlated, that sounds to me like a bug. I personally would be OK with changing the generated sequence to fix a bug.

This probably should be discussed on golang-dev, if somebody has a patch.

[ianlancetaylor]

For Go 2 I think we are likely to move in the direction of #21835, which seems to be a superior random number generator in any case. I think that is a better place to discuss the future of math/rand. Closing this issue.