CL2D producing less classes than requested

[dallakyan]

Thank you to XMIPP developers for your contributions. We are running CL2D (xmipp_mpi_classify_CL2D) with ~10k particles and requesting 40 classes (--nref 40). However, it produces only 4 classes. When we run it on a subset of ~5k particles, with the same parameters, it is able to produce all 40 classes. This seems counterintuitive, and when I dig deeper in the code, I see that it stops when number of assignment changes is less than 0.5% of total particles.

I was wondering if anyone else has seen this issue and can comment on this. One workaround to get more classes is to reduce the number of iteration, but I'm not sure if this is am optimal solution.
Thanks.

[DStrelak]

Hello @dallakyan,

thanks for reporting this issue!
To summarize the problem:

1. you have a set of 10k particles S
2. your run xmipp_mpi_classify_CL2D --nref 40 and get only 4 classes
3. you create a subset of S, say s, with 5k particles
4. you run xmipp_mpi_classify_CL2D --nref 40 and get 40 classes

Is that correct?
Can you send us (dstrelak@cnb.csic.es) Scipion logs from both protocols?
Thanks.

[ajimoreno]

Hello @dallakyan,

thanks for your report.
I was taking a look at the code. You're right, the code stops when the number of assignment changes is less than 0.5%. Maybe, with the whole input set, the program gets stuck and it's unable to evolve from the beginning classes and is stopping in this condition. On the other hand, with the subset of the input particles, it seems that the program is able to generate initial classes with more differences among them and it's able to evolve from these.
I was thinking about how to manage this problem. You could try to run a couple of classifications using subsets of the input particles, then you can concatenate the obtained classes. Another option is to use the "Number of initial classes" field in the protocol, if you put a higher number of random initial classes (e.g. 8), maybe the program will find enough differences among the initial classes to evolve and generate the desired number of output classes.
I hope that any of these recommendations can help you. Let me know if the problem persists.

Thanks.

[dallakyan]

Thank you @DStrelak and @ajimoreno.