load-into-counting.py: ValueError: bigcount is not supported for this storage

[sjackman]

load-into-counting.py with --small-count yields ValueError: bigcount is not supported for this storage.. Adding --no-bigcount doesn't seem to help.

❯❯❯ load-into-counting.py --small-count -x 1e7 khmer2 foo.fq.gz
PARAMETERS:
 - kmer size =     32 		(-k)
 - n tables =      4 		(-N)
 - max tablesize = 1e+07 	(-x)
Estimated memory usage is 0.0 Gb (4e+07 bytes = 4 bytes x 1e+07 entries / 1 entries per byte)
--------
Saving k-mer countgraph to khmer2
Loading kmers from sequences in ['foo.fq.gz']
making countgraph
Traceback (most recent call last):
  File "/Users/sjackman/.homebrew/bin/load-into-counting.py", line 221, in <module>
    main()
  File "/Users/sjackman/.homebrew/bin/load-into-counting.py", line 139, in main
    countgraph.set_use_bigcount(args.bigcount)
ValueError: bigcount is not supported for this storage.
❯❯❯ load-into-counting.py --version
khmer 2.1.1

[betatim]

I will take a look at this.

[betatim]

Thanks for catching this!

[sjackman]

No worries! Is there a workaround for version 2.1.1? load-into-counting.py --small-count --no-bigcount -x 1e7 khmer2 foo.fq.gz didn't work for me.

[sjackman]

Or can you suggest a different command that I can use to test the memory usage of --small-count vs the default?

[standage]

I'm porting this to the v2.1 maintenance branch now. Will cut a bugfix release once the JOSS review is complete.

[sjackman]

For the item Performance: Have any performance claims of the software been confirmed? I'm trying to confirm the reduced memory usage of --small-count vs the default. Can you suggest a command?

[standage]

Once #1726 is merged, you should be able to run the following commands on branch maint/2.1 (from which release version 2.1.2 will be cut when the JOSS review is complete).

Same graph size, same FPR, less memory

scripts/load-into-counting.py -x 2e6 default.count tests/test-data/test-reads.fq.gz
scripts/load-into-counting.py -x 2e6 small.count --small-count tests/test-data/test-reads.fq.gz

The reduced memory usage should be reflected in the configuration dump printed to the screen, as well as the count graph file sizes (default.cg vs small.cg).

Same amount of memory, larger graph size, smaller FPR

load-into-counting.py -M 4M default.cg tests/test-data/test-reads.fq.gz
load-into-counting.py -M 4M small.cg --small-count tests/test-data/test-reads.fq.gz

The output file sizes should be the same, but the reported table size for the --small-count should be larger, and the reported FPR smaller.

[sjackman]

Thanks. I've subscribed to PR #1726.

[sjackman]

Your suggested commands worked on the maint/2.1 branch. Thanks. I confirmed that the memory usage with --small-count is half that of the default.

[standage]

Excellent.