Something work wrong for large sets

[DKurilo]

Maybe I just didn't get idea good enough, but it looks like bug for me:

let fzr132 = (fromList . take 132 . repeat) (pack "John Smith")
getWithMinScore 0.72 (fzr132 `add` (pack "Joseph Dombrowski")) (pack "Joe Dombrowski")
-- [(0.8235294117647058,"Joseph Dombrowski")]
let fzr133 = (fromList . take 133 . repeat) (pack "John Smith")
getWithMinScore 0.72 (fzr133 `add` (pack "Joseph Dombrowski")) (pack "Joe Dombrowski")
-- []

[DKurilo]

More information:

let fzr133 = (FZ.fromList . take 133 . repeat) (DT.pack "Nebraska")
FZ.getWithMinScore 0.72 (fzr133 `FZ.add` (DT.pack "Nevada")) (DT.pack "Devada")

works.

let fzr133 = (FZ.fromList . take 133 . repeat) (DT.pack "Nebraska")
FZ.getWithMinScore 0.72 (fzr133 `FZ.add` (DT.pack "Nevada")) (DT.pack "Nevoda")

doesn't work.

[laserpants]

Thanks for the report. I haven't touched this library for a while, but it definitely looks like a bug. I'll look into it.

[DKurilo]

Thank you!
I belive it's somewhere here https://github.com/laserpants/fuzzyset-haskell/blob/master/src/Data/FuzzySet/Internal.hs#L29 , but I don't know algorithm good enough to be sure.
(sorry, it's not true)
But will try to understand how this magic is supposed to work. :)

[laserpants]

The algorithm is adopted from here, and indirectly here.

But looking at this now, I'd like to clean up the code and get rid of the Unicode syntax. The only part I am happy about are the docblocks.

[laserpants]

@DKurilo I am working on a refactor branch that I plan to merge shortly. You can have a look at it, if you're still interested. It is an effort to clean up the code and improve overall readability. I've refactored most of the Data.FuzzySet.Internal module, and as far as I can see, this bug is no longer present. I added your code above as a test case. (The tests are still a mess, but the rest of the code should hopefully be a lot easier to follow now.)

[DKurilo]

@laserpants , great! Thank you.
Will try refactored version soon.

[DKurilo]

@laserpants , I tried to use refactored version. It works fine.
Thank you.

[laserpants]

Thanks, I'll merge into master and push this to Hackage as version 0.2.0 once I've updated the docs.