MinHasher32 test is flakey

[johnynek]

[info] MinHasher32
[info] - should measure 0.5 similarity in 1024 bytes with < 0.1 error
[info] - should measure 0.8 similarity in 1024 bytes with < 0.05 error *** FAILED ***
[info]   0.05833333333333324 was not less than 0.05 (MinHasherTest.scala:30)
[info] - should measure 1.0 similarity in 1024 bytes with < 0.01 error

This is quite common. Need to weaken the bound or figure out if there is really some bug (doubt it).

[Gabriella439]

Another instance of this occurring:

https://travis-ci.org/twitter/algebird/jobs/111090636

[info] - should measure 0.8 similarity in 1024 bytes with < 0.05 error *** FAILED ***
[info]   0.07916666666666672 was not less than 0.05 (MinHasherTest.scala:30)

[johnynek]

we have that approach that @sid-kap added. @Gabriel439 want to take a stab at porting this test to that framework?

[Gabriella439]

Sure, I will give it a stab

[Gabriella439]

So I think this is a case of the error bounds being incorrect. The test that occasionally fails is this one:

    "measure 0.8 similarity in 1024 bytes with < 0.05 error" in {
      test(new MinHasher32(0.8, 1024), 0.8, 0.05)
    }

... and if you follow the code for that specific MinHasher32 constructor initialization it initializes numHashes to 247. The expected error for the Min Hash algorithm is 1 / sqrt numHashes which is in this case evaluates to approximately 0.064. This explains why the test occasionally fails because the test requires an error less 0.05 which is below the expected error. We should probably bump the expected error up to at least 0.1.

[johnynek]

sounds fine to me.