Skip to content

[BUG]: Random key generator does not produce keys according to input parameters #532

@sleeepyjack

Description

@sleeepyjack

Is this a duplicate?

Type of Bug

Silent Failure

Describe the bug

This snippet generates N uniform keys given a target key multiplicity M for each iteration and produces the following output:

N=10
  M_EXPECTED=1
  M_MEASURED=10
  CONSECUTIVE_SAMPLES=[ 3 3 3 3 3 3 3 3 3 3 ]

N=10
  M_EXPECTED=5
  M_MEASURED=10
  CONSECUTIVE_SAMPLES=[ 2 2 2 2 2 2 2 2 2 2 ]

N=10
  M_EXPECTED=10
  M_MEASURED=10
  CONSECUTIVE_SAMPLES=[ 1 1 1 1 1 1 1 1 1 1 ]

N=100
  M_EXPECTED=1
  M_MEASURED=100
  CONSECUTIVE_SAMPLES=[ 44 44 44 44 44 44 44 44 44 44 ]

N=100
  M_EXPECTED=5
  M_MEASURED=100
  CONSECUTIVE_SAMPLES=[ 13 13 13 13 13 13 13 13 13 13 ]

N=100
  M_EXPECTED=10
  M_MEASURED=100
  CONSECUTIVE_SAMPLES=[ 8 8 8 8 8 8 8 8 8 8 ]

N=1000
  M_EXPECTED=1
  M_MEASURED=41.6667
  CONSECUTIVE_SAMPLES=[ 244 244 244 244 244 244 244 244 244 244 ]

N=1000
  M_EXPECTED=5
  M_MEASURED=166.667
  CONSECUTIVE_SAMPLES=[ 194 194 194 194 194 194 194 194 194 194 ]

N=1000
  M_EXPECTED=10
  M_MEASURED=250
  CONSECUTIVE_SAMPLES=[ 21 21 21 21 21 21 21 21 21 21 ]

N=10000
  M_EXPECTED=1
  M_MEASURED=4.44642
  CONSECUTIVE_SAMPLES=[ 9668 9668 9669 9669 9669 9669 9670 9670 9670 9670 ]

N=10000
  M_EXPECTED=5
  M_MEASURED=22.1729
  CONSECUTIVE_SAMPLES=[ 1801 1801 1801 1801 1802 1802 1802 1802 1802 1802 ]

N=10000
  M_EXPECTED=10
  M_MEASURED=44.4444
  CONSECUTIVE_SAMPLES=[ 248 248 248 248 248 248 248 248 248 248 ]

N=100000
  M_EXPECTED=1
  M_MEASURED=1.87691
  CONSECUTIVE_SAMPLES=[ 62772 62774 62776 62779 62781 62783 62785 62788 62790 62792 ]

N=100000
  M_EXPECTED=5
  M_MEASURED=5
  CONSECUTIVE_SAMPLES=[ 8175 8176 8176 8177 8177 8178 8178 8179 8179 8180 ]

N=100000
  M_EXPECTED=10
  M_MEASURED=10
  CONSECUTIVE_SAMPLES=[ 6935 6935 6935 6935 6936 6936 6936 6936 6936 6937 ]

N=1000000
  M_EXPECTED=1
  M_MEASURED=1.34564
  CONSECUTIVE_SAMPLES=[ 854682 854705 854727 854750 854772 854795 854817 854840 854862 854885 ]

N=1000000
  M_EXPECTED=5
  M_MEASURED=5
  CONSECUTIVE_SAMPLES=[ 68414 68419 68423 68428 68432 68437 68441 68446 68450 68455 ]

N=1000000
  M_EXPECTED=10
  M_MEASURED=10
  CONSECUTIVE_SAMPLES=[ 98136 98138 98140 98142 98145 98147 98149 98151 98154 98156 ]

Which surface multiple problems with the generated keys:

  • When N<100,000 then measured key multiplicity is way off the expected multiplicity value
  • Even for large N and M_EXPECTED=1 the M_MEASURED is still somewhat far off
  • The sequence of output keys seems to be at least partially sorted (see CONSECUTIVE_SAMPLES) which is undesireable since it leads to caching effects that could skew our benchmarks

How to Reproduce

See above section

Expected behavior

  • Fix multiplicity value for small N
  • Find a way to ensure M_EXPECTED=1 -> M_MEASURED=1+epsilon (maybe just dispatch to a unique key sequence?)
  • Eliminate consecutive runs of the same key in the output (either by shuffling the generated keys one more time or shuflle/randomize the seeds)

Reproduction link

https://godbolt.org/z/cqeWG8af1

Operating System

No response

nvidia-smi output

No response

NVCC version

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions