A novice's request for advice on loading very large tractograms (.tck)

[MarkNelson86]

Hi, I am trying to load VERY large tractograms (40M streamlines, ~50GB) and dipy.io.streamline / nibabel.streamlines seem to be completely unable to handle them (> 48 hours proc time). When I finally terminate the code, I get:

File "~/nibabel/streamlines/array_sequence.py", line 30, in _safe_resize
    a.resize(shape)
**ValueError: cannot resize an array that references or is referenced by another array in this way.**
Use the np.resize function or refcheck=False

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File /dipy/io/streamline.py, line 144, in load_tractogram
    tractogram_obj = nib.streamlines.load(filename).tractogram
  File "/nibabel/streamlines/__init__.py", line 95, in load
    return tractogram_file.load(fileobj, lazy_load=lazy_load)
  File "/nibabel/streamlines/tck.py", line 151, in load
    streamlines = ArraySequence(tck_reader)
  File "/nibabel/streamlines/array_sequence.py", line 135, in __init__
    self.extend(iterable)
  File "/nibabel/streamlines/array_sequence.py", line 316, in extend
    self.append(e, cache_build=True)
  File "/nibabel/streamlines/array_sequence.py", line 238, in append
    self._resize_data_to(req_rows, build_cache)
  File "/nibabel/streamlines/array_sequence.py", line 274, in _resize_data_to
    self._data = _safe_resize(self._data, new_shape)
  File "/nibabel/streamlines/array_sequence.py", line 33, in _safe_resize
    a.resize(shape, refcheck=False)
KeyboardInterrupt

I included reference=dwi_b0.nii.gz (~6MB) in the function call.

I am a relative novice at working with tractograms in Python and could use some advice. Do you think these tractograms can be loaded with dipy.io.streamline? Or would it be better to load directly with nibabel.streamlines? Then, I could make use of the lazy_load option (my subsequent processing will be linearized) and set refcheck=False to avoid the issue above. Until now, I have avoided this option as there is a greater chance I make a mistake without the added tractogram loading support offered by dipy. Any advice is appreciated. Thanks.
Mark