[MRG] Debug and fix SBT search

[ctb]

We have a persistent problem in that sourmash sbt search does not find everything it should. Containment analysis (sourmash gather) seems to work fine.

First task: figure out how to reproduce it quickly & easily.

• add utils/check-tree.py to systematically search trees for their leaves

Second task: scour the codebase with flame and acid.

• fix via a1210a1

Ref #244 and #454.

• Is it mergeable?
• make test Did it pass the tests?
• make coverage Is the new code covered?
• Did it change the command-line interface? Only additions are allowed
  without a major version increment. Changing file formats also requires a
  major version number increment.
• Was a spellchecker run on the source code and documentation after
  changes were made?

[codecov-io]

Codecov Report

Merging #484 into master will decrease coverage by 0.03%.
The diff coverage is 92.15%.

@@            Coverage Diff            @@
##           master    #484      +/-   ##
=========================================
- Coverage   90.54%   90.5%   -0.04%     
=========================================
  Files          33      33              
  Lines        4906    4931      +25     
  Branches       36      36              
=========================================
+ Hits         4442    4463      +21     
- Misses        463     467       +4     
  Partials        1       1

Impacted Files	Coverage Δ
sourmash/sbtmh.py	86.25% <85.71%> (-0.11%)	⬇️
sourmash/sbt.py	83.29% <94.59%> (+0.07%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c5791cc...c4fd90d. Read the comment docs.

[ctb]

It looks to my naive eye like the problem is due to use of max instead of min to calculate the highest possible Jaccard similarity below an internal tree node. If so, this is a really embarrassing brain fart on my part.

[ctb]

@luizirber your thoughts welcome ... might be ready for merging.

[luizirber]

I aggree with the changes, but trying to think how to support them on 2.0.0a6 without breaking existing DBs (or sourmash installation): probably need to keep max_n_below as a synonym for min_n_below in the metadata?

[ctb]

On Mon, Jun 04, 2018 at 01:26:34PM -0700, Luiz Irber wrote: I aggree with the changes, but trying to think how to support them on `2.0.0a6` without breaking existing DBs (or sourmash installation): probably need to keep `max_n_below` as a synonym for `min_n_below` in the metadata?

I think it _should_ break existing DBs. You can't use existing DBs for search without getting this parameter set. No?

[luizirber]

I bumped the version to 2.0.0a7.

On the DB side, two things:

• Create a new "old DBs" section on the docs, to keep track of where they are Create a new "old DBs" section on the docs, to keep track of where they are #487
• Create a migrate command (ou use convert?), and only load the latest DB version available (right now it tries to be clever and load any older version, which might take a long time depending on what needs to be set) Create migrations from older index versions #488

[ctb]

we should also make available a reasonably sized database (I can use the podar set) constructed using each important version. I think there's an issue for this already.