Maximum number of objects

[jboulanger91]

I am having problems with saving the output of Cellpose:
OverflowError: cannot serialize a string larger than 4GiB
It seems that the _seg.npy file is limited to 4Gb. It is something that could be fixed in the future?

Also what is the maximum number of objects that can be predicted? It seems the 2^16 is the max.
This would be nice to increase this for large data-sets.

Thanks a lot,
Jon

[kevinjohncutler]

Oh that's interesting. I'm guessing you'd need to go through the code and change all the uint16 datatypes to uint32. If you're lucky, the network itself won't care about the datatype, and it is actually just the mask reconstruction in dynamics.py that you need to change (I assume you aren't training on images with more than 2^16 manually created labels!).

Of course, a workaround would be to subdivide your images, but it is tricky to do that without sacrificing cells to the newly imposed boundaries.

[papalagirauscher]

Hello,
I would like to join in on this conversation. Thank you for the great project and comment provided by @kevinjohncutler
If I am not completely wrong, dynamics.py does not contain uint16 entrys. The only py file containing them is io.py.
If I change them in this file to uint32, I however get an error message:

cv2.imwrite(filename, arr)
TypeError: Expected Ptrcv::UMat for argument 'img'

Any ideas are appreciated, cellpose proves to be great, but sadly I need to process large images.

Thank you,
Jan

[kevinjohncutler]

@papalagirauscher You are right, the flows array is float32, which I think is already compatible with uint32 masks. As to your error, I think this is due to trying to save as a PNG, which limits the bit depth. Try changing this to a TIF, for example by editing the defaults in the save_masks function of io.py:
save_masks(images, masks, flows, file_names, png=False, tif=True):