Discrepancy between code and documentation for genotype_matrix()

[alxsimon]

Hi,
While trying to reduce my RAM usage, I found a discrepancy for the type returned by genotype_matrix between the documentation

tskit/python/tskit/trees.py

Line 5304 in 1919abe

:rtype: numpy.ndarray (dtype=np.int8)

and the code

tskit/python/tskit/trees.py

Line 5325 in 1919abe

ret = np.zeros(shape=(self.num_sites, num_samples), dtype=np.int32)

[benjeffery]

Thanks @alxsimon! This was a recent change, and we missed changing the docstring. Will fix, unless you want to submit a PR?

[jeromekelleher]

Would being able to specify the dtype be useful to you @alxsimon?

[alxsimon]

@benjeffery I didn't want to submit a PR as I didn't know which of the code or documentation was the decided dtype, but I can.

@jeromekelleher using int32 was memory expensive in my case, especially as I was only storing 0s and 1s.
I don't know if it's worth adding another parameter as I'm not sure exporting the complete genotype matrix is often used.
Is there a specific reason int32 was decided? It feels like most of the time int8 would be enough, with some edge cases of int16 maybe.

[jeromekelleher]

We chose int32 here for simplicity, as this is what the dtype of the Variant.genotypes is. However, this is massive overkill most of the time, as you say.

For people using genotype matrix "in anger", adding a dtype argument would allow them to cut a large amount of ram off. Would it help you to be able to store them as int8, or whatever?

We could also do something like figure out what the maximum number of alleles at a site is by looking at the mutation table. If there's less than 127 mutations at any site, then I guess we can't possibly have more than 127 alleles?

Anyway, if this is a bottleneck for you in your code we can definitely improve it.

[benjeffery]

@alxsimon You can see discussion of this at #2108 and #463

[alxsimon]

Thanks a lot for taking the time to explain!
You're right exporting the genotype matrix is rage coding 😉.
I've spent a bit more time going around the issue and not export the complete matrix, and I could go further into improvements if needed.
I don't think it's necessary to change anything, as I might have said, I'm only doing unusual stuff.