x/pkgsite: experiment rollout imprecise and unstable

[abursavich]

What is the URL of the page with the issue?

N/A

What is your user agent?

N/A

Screenshot

N/A

What did you do?

Read the experiments code.

What did you expect to see?

Experiments that rollout with precision and per user stability.

What did you see instead?

The chosen arithmetic uint(h.Sum32())%(100/e.Rollout) == 0 has two problems.

• The integer division of 100/e.Rollout greatly reduces the precision available (consider that e.Rollout = 51 actually results in 100% rollout).
• Varying the modulus divisor with the rollout threshold shuffles users in and out of an experiment rather than making a user enter and remain in the experiment once at a given threshold.

[gopherbot]

Change https://golang.org/cl/243659 mentions this issue: internal/middleware: fix experiment assignment precision and stability