splitHaplotype is using too much memory

[brianwalenz]

Two reports.

On a 3tb node:

p    prefixes       bits gigabytes (allowed: 1 GB)
-- -------------- ---------------- ---------
21    2097152   76313443474   8.884
22    4194304   75010317320   8.732
23    8388608   73841408894   8.596
24    16777216   72940935924   8.491
25    33554432   72577333866   8.449 (smallest)
26    67108864   73287473632   8.532
27   134217728   76145097046   8.864
28   268435456   83297687756   9.697
29   536870912   99040213058  11.530
30   1073741824   131962607544  15.362
-- -------------- ---------------- ---------
Not enough memory to load 1437343882 distinct 21-kmers.
Need at least 8.449 GB memory.

On 192 gb node, it also fails.

With -memory 500 it succeeds.

-- Loading haplotype data, using up to 250 GB memory for each.
-- p    prefixes       bits gigabytes (allowed: 250 GB)
-- -------------- ---------------- ---------
21    2097152   76313443474   8.884
22    4194304   75010317320   8.732
23    8388608   73841408894   8.596
24    16777216   72940935924   8.491
25    33554432   72577333866   8.449 (smallest)
26    67108864   73287473632   8.532
27   134217728   76145097046   8.864
28   268435456   83297687756   9.697
29   536870912   99040213058  11.530
30   1073741824   131962607544  15.362
31   2147483648   199244740398  23.195
32   4294967296   335246349988  39.028
33   8589934592   608686913050  70.860
34  17179869184  1157005383056  134.693 (used)
35  34359738368  2255079666950  262.526
36  68719476736  4452665578620  518.358
37  137438953472  8849274745842 1030.191
38  274877906944  17643930424168 2054.024
-- -------------- ---------------- ---------
For 1437343882 distinct 21-mers (with 34 bits used for indexing and 8 bits for tags):
 134.693 GB memory
 128.000 GB memory for index (17179869184 elements 64 bits wide)
  1.339 GB memory for tags (1437343882 elements 8 bits wide)
  5.355 GB memory for data (1437343882 elements 32 bits wide)