discussion for why modulo hash / scaled signatures are awesome

[ctb]

From private conversations with @luizirber @bluegenes @halexand recently -- scaled signatures are different from MinHash because:

• intended for containment (e.g. vs mash screen, regular minhashes)
• streaming compatible in the following way: at any point in a stream, scaled signatures are always proper subsets of the scaled signature of the whole data set, unlike MinHashes. specifically, no hash is ever removed from a scaled signature as more data is received. This means that for searches of a database using streamed-in data, all prior matches remain valid (although their significance may change as more data is received). This allows us to place algorithmic guarantees on containment searches.
• scaled signatures can be "abundance trimmed" while regular minhashes cannot be.
• in many cases scaled signatures are compatible with regular minhashes and can be converted (see [MRG] add 'sourmash signature' signature manipulation utilities. #587 downsample and discussion in API docs in [MRG] Update the API docs #596), and both could maybe be built simultaneously (see Improvement: allow both num and scaled to be set #538)
• I strongly suspect that scaled signatures deal much better with sets of different sizes
• scaled signatures can be downsampled to larger scaled numbers (similar to downsampling MinHash, so not a huge advantage but could be mentioned)

These properties need to be clearly laid out, discussed, evaluated empirically, and (ideally) described theoretically. cc @dkoslicki

[ctb]

(of course, this all needs to be balanced against the point that they can grow indefinitely :)

[ctb]

note Richard Durbin's modimizer, which uses similar concepts! https://github.com/richarddurbin/modimizer - the README is informative for this issue.

[ctb]

a more succinct way of putting the containment guarantees above are "Containment never decreases as you get more data" (which is nice for streaming esp.)

[ctb]

note also that you can subtract and add scaled signatures, and filter them on abundance, and other things, without fear.

[ctb]

preprinted and available! see link in #823.