Classification + Analysis step Inefficiencies

[JuiceSackey]

Hello,

It seems like classification and analysis step of the program take entirely too long for my dataset (~1 day for each step).
We are analyzing whole brain cfos expression, so there are over 700k cells, which is probably contributing to this issue. I raised an issue on the forums and I was told it may be due to inefficiencies in the coding for these steps and to raise the issue here.
The data is all stored/written to an internal HDD.

This is a log file for just analysis ran:
cellfinder_2021-09-05_09-30-11_log.txt

This is a log file with the classification+analysis step run:
cellfinder_2021-09-07_13-31-22_log.txt

As you can imagine, this is a pretty big bottleneck in processing our data.
Please let me know if you need any more information. Thanks!

My specs are as below:

• OS: Ubuntu
• Version 18.04.5 LTS
• Hardware specs: 64GB RAM, AMD® Ryzen 9 3900x 12-core processor × 24 CPU, GeForce RTX 2080 Ti/PCIe/SSE2 GPU

[adamltyson]

I've removed the bug label, as technically everything is working correctly, just very slow on big datasets, but this does need to be improved.

I don't know when I'll get time to look at this, but I would hope it shouldn't be too complex.