numpy.core._exceptions._ArrayMemoryError: Unable to allocate 2.86 TiB for an array with shape (886662, 886662) and data type float32

[lfdelzam]

Hi, thanks for a great binning program. I have been using semibin2 successfully until yesterday. It crashed (HPC, 1TB RAM), printing out the following message:

INFO Start binning.
Traceback (most recent call last):
File "miniconda3/envs/semibin2_env/bin/SemiBin2", line 10, in
sys.exit(main2())
^^^^^^^
File "miniconda3/envs/semibin2_env/lib/python3.12/site-packages/SemiBin/main.py", line 1481, in main2
single_easy_binning(
File "miniconda3/envs/semibin2_env/lib/python3.12/site-packages/SemiBin/main.py", line 1192, in single_easy_binning
binning(**binning_kwargs)
File "miniconda3/envs/semibin2_env/lib/python3.12/site-packages/SemiBin/main.py", line 1100, in binning
cluster(
File "miniconda3/envs/semibin2_env/lib/python3.12/site-packages/SemiBin/cluster.py", line 280, in cluster
embedding, contig_labels = run_embed_infomap(logger, model, data,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "miniconda3/envs/semibin2_env/lib/python3.12/site-packages/SemiBin/cluster.py", line 155, in run_embed_infomap
kl = cal_kl(depth[:,2k], depth[:, 2k + 1])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "miniconda3/envs/semibin2_env/lib/python3.12/site-packages/SemiBin/cluster.py", line 54, in cal_kl
res = ne.evaluate(
^^^^^^^^^^^^
File "miniconda3/envs/semibin2_env/lib/python3.12/site-packages/numexpr/necompiler.py", line 975, in evaluate
return re_evaluate(local_dict=local_dict, global_dict=global_dict, _frame_depth=_frame_depth)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "miniconda3/envs/semibin2_env/lib/python3.12/site-packages/numexpr/necompiler.py", line 1007, in re_evaluate
return compiled_ex(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 2.86 TiB for an array with shape (886662, 886662) and data type float32

Any tip about how to solve it?
Thanks in advance.