Version 0.4.0

Release Notes

Added

• Reader-side small-genome screening, now the default. profile --screen-small-genomes N (default 50, sylph 1.0's --min-sparse-kmers default; 0 disables) finds the genomes of a .syl2db that have fewer than N stored stage-1 screen k-mers when the database is opened, reads back each one's N smallest dense k-mers from its dense block, and screens the sample against those directly instead of through the pooled index. db-convert/db-add gain --small-genome-screen none|profile|loosen, defaulting to profile (none builds the same file): no per-genome floor is stored and the pooled index stays at the nominal --screen-c, rather than being widened to the densest rate any small genome needed (loosen, the previous behaviour and upstream sylph's, still available). The reader screens exactly the k-mer set a loosen build stores — a genome's sparse set is always a value-ordered prefix of its dense hashes — so the two give the same per-genome counts, matches and stage-1 survivors (tested), and enabling the reader on a database built with a floor changes nothing. It also makes databases built elsewhere (sylph) benefit. Cost moves from every sample k-mer's index lookup to O(retained small-genome k-mers) per sample, a win while small genomes are a minority of the reference; in the sylph benchmark (2,000 GTDB genomes + 100,000 vOTUs) it had 100% recall, a 3x smaller index and an 8-11x faster stage-1 screen than loosen. Stage-1 -M scaling now uses the densest rate anything in the database is screened at (TwoStageDb::effective_screen_c). --screen-dump rows are now sorted by genome so two runs' dumps are comparable. Behaviour change: .syl2db files built with the default options are now smaller and no longer carry a floor; pass --small-genome-screen loosen to get the old files.
• .syl2db databases written by upstream sylph are now read directly (profile, query, db-add, inspect). The two formats are the same layout apart from the 8-byte whole-file checksum weebill inserts after the version byte, which shifts the two header offsets; everything after the header — dense blocks, screen index, footer — carries the same meaning, and with the sparse floor below the two tools' output of the same .syldb now differs only in those 8 bytes plus the per-genome dense offsets they shift. A sylph database (e.g. a hosted one) therefore needs no conversion; only the on-demand integrity check is unavailable for it, which inspect reports as checksum: absent rather than claiming ok. Sylph 1.0 does not read weebill's version 3 in the other direction; that needs ~20 lines in its parse_header (patch available).
• db-convert/db-add --min-sparse-kmers (default 50; applies with --small-genome-screen loosen, see above) puts a floor on each genome's stage-1 screen k-mers: a genome whose --screen-c subsample falls short of it is selected at a denser, genome-specific rate instead, and the rate recorded in the file is loosened just enough to keep those k-mers matchable. Without it a small genome — plasmid, phage, short contig — can have a handful of screen k-mers or none at all (a 50 kbp plasmid has ~17 at --screen-c 3000), so its screen ANI is noise and two-stage profiling silently loses it where single-stage finds it. Matches upstream sylph's option and default, so both tools build equivalent screen indexes. A genome's sparse set is always the smallest n of its dense hashes, and n is already in the screen index, so db-add reproduces a densified genome's screen entry exactly rather than re-subsampling it at the database's rate. db-add also selects the genomes it adds at the nominal --screen-c the database was built with rather than the (possibly much denser) rate recorded in it, which one small genome can drag down: the nominal rate is not stored anywhere — the footer layout is shared with sylph — but the copied genomes bound it, since each of them keeps at least every dense hash below the nominal threshold. Without that, one plasmid in a database would make every later db-add behave as if --screen-c were the dense -c, ballooning the stage-1 index for good.
• -d/--databases on profile/query names database inputs explicitly, alongside or instead of positional input, matching upstream sylph. --threads is now accepted as the long form of -t on every subcommand, and db-convert also answers to sylph's name for it, convert-db-two-screen.
• weebill sketch now writes a genome database (.syldb/.syldbc) with its genomes in input order. They were written in the order threads happened to finish them, so the same genomes sketched with -t 2 or more gave a different file each time (one thread was already stable). The sketch of each genome was unaffected, but the order is what ref-build tie-breaks on, so it made reference builds irreproducible; ref-build --genome-order below still pins it independently of the database.
• weebill ref-build --genome-order <file> fixes the order genomes are fed to the owner-assignment pass, which is what decides the owner of a k-mer contested by fewer than --pool-min-genomes same-tier genomes. Without it that order is whatever order the sketches sit in their .syldb, and weebill sketch writes genomes in parallel-completion order, so two builds from the same FASTAs can produce references that differ — and a .sylspr compressed against one then refuses to decode against the other. The flag makes a build reproducible, and lets a lost reference be rebuilt exactly. The order file must cover the input exactly; a missing or duplicated genome is an error rather than a silently shifted tie-break. Builds without the flag are unchanged: the ownership tally now keeps the lowest id seen rather than the first encountered, which is the same thing wherever ids arrive in increasing order.
• WEEBILL_SIMD=auto|avx512|avx2|scalar overrides the k-mer extraction kernel, for benchmarking one against another in a real run (cargo run --release --example bench_seeding benchmarks them in isolation).

Changed

• AVX-512 k-mer extraction is now used only for read sequences of at least 2 kbp, and never for genome sketching. Both kernels split the sequence into lanes and prime each with k-1 bases before its rolling hash yields anything, so AVX-512 pays twice the priming for half as many inner-loop iterations. Measured on an AMD EPYC 9684X (Zen 4) at k=31, c=200: on the read path it is 1.08-1.10x faster than AVX2 on long sequences but 0.46-0.72x on 100-250 bp reads, crossing over at ~1 kbp — end to end that made single-threaded paired read sketching 30% slower (median 4.91s vs 3.82s for 4M pairs of 150 bp). On the genome path (the _positions kernels, which emit position tuples and so cannot compact survivors with vpcompressq) it is 0.88-0.90x AVX2 even over a 5 Mbp contig, ~7% slower end to end (1.06s vs 0.98s for 102 E. coli genomes). What is left is long-read sketching, worth ~1.08x. The two kernels emit the same multiset of hashes, so which one runs never changes a sketch.
• -M/--min-number-kmers now defaults to 10 rather than 50, with a new --min-contain (default 7) requiring that many contained k-mers for a hit. The old floor scaled with genome size, so at c=200 it silently excluded anything under ~10 kbp of sampled k-mers (small viruses, plasmids); the new pair keeps precision through an absolute evidence floor instead. Matches upstream sylph's defaults.
• profile no longer inflates its samples-in-flight count to at least threads/3 + 1; it uses query's min(num_samples, threads), counted over every sample rather than only the raw read files. Counting all of them is what makes the floor redundant — it could then only exceed min(num_samples, threads) when it also exceeded the number of samples, which get_chunks caps at anyway — and it fixes runs over pre-sketched samples, which the old count ignored entirely: profile db *.sylsp was held to threads/3 + 1 samples in flight however many samples and threads it was given, and query db *.sylsp to one at a time. Upstream dropped the floor because it starved their per-file sketch pipeline.

Fixed

• Bootstrap confidence intervals are reproducible again. bootstrap_interval seeded fastrand's thread-local global generator, but genomes are finalized concurrently through rayon, so genomes sharing a worker thread reset and consumed each other's stream and their CIs depended on the interleaving. It now uses a private, seeded generator per call, giving every genome the resamples a single-threaded run would give it, independent of -t. (Same fix as upstream sylph's.)

• weebill db-add adds new genomes to an existing two-stage database (.syl2db) without rebuilding it from its source .syldb. The new genomes come from pre-sketched databases (*.syldb/*.syldbc) and/or raw FASTAs given with -g, which are sketched here at the existing database's own -c/-k/--min-spacing so they are directly comparable. Because every genome's dense block is independently Golomb-Rice coded and self-delimiting, the existing blocks are copied through byte for byte and the new ones appended; only the stage-1 screen index is rebuilt, since its pooled MPHF cannot absorb new keys. The result is exactly what a db-convert of all the genomes together would produce — same metadata, same dense blocks, same screen results — verified against it in the tests. Updates are in place by default (written to a sibling temp file and renamed, so an interrupted run leaves the original intact) or to a separate -o output. Adding a genome whose file name is already in the database is an error, since a .syl2db identifies genomes by that name alone; --skip-existing ignores such genomes instead, making a repeated add a no-op. Mismatched -c/-k, genomes sketched with --disable-profiling, and a database that fails its checksum (unless --no-verify) are all refused before anything is written.

Install weebill 0.4.0

Install prebuilt binaries via shell script

curl --proto '=https' --tlsv1.2 -LsSf https://github.com/wwood/weebill/releases/download/v0.4.0/weebill-installer.sh | sh

Download weebill 0.4.0

File	Platform	Checksum
weebill-x86_64-unknown-linux-gnu.tar.xz	x64 Linux	checksum
weebill-x86_64-unknown-linux-musl.tar.xz	x64 MUSL Linux	checksum