[pyclustering.nnet.sync] Multi-core implementation

[annoviko]

Multi-core implementation should be introduced. Prototype has been successfully tested on 24-core machine and shows advantages.

[jesusareyesv]

Hello, I'm testing your implementation of X-means clustering algorithm in my research work, and the performance with some datasets that contains 4 or 5 million points is not exactly what i've expected. The future multicore version of this package will improve the performance of all the algorithms in the package (including X-means) or only some of them?

[annoviko]

Hello, @TemetNosce95,

Thank you for your attention to the library.
Have you tried to use ccore flag to increase performance of the X-means? There is an example how to do that:

xmeans_instance = xmeans(sample, start_centers, 20, ccore = True);

In this case (when ccore flag is true) C++ implementation of the algorithm will be used instead of python implementation and should improve. Please, let me know if you have got troubles with it (there were several compatibility issues that are already fixed on 'master' and '0.7.dev' branches).

This multi-core implementation is related to Sync oscillatory network, but not for X-Means algorithm.

Still, I will investigate the performance issue related to X-Means algorithm and try improve it. Also, it would useful if you provide datasets for final testing otherwise I will use random generated data for the issue for investigation and testing.

I have create the issue related to your problem: #372 .

[annoviko]

@TemetNosce95, I have started multi-core implementation for X-Means. And fixed clustering result before optimization to be able to compare results. I have found that in case of millions points situation is not so dramatic, there are some results of performance test (before optimization):

Random sample: (size:600000) ', Execution time: ' 6.189236562524233
Random sample: (size:1200000) ', Execution time: ' 12.69288331531482
Random sample: (size:1800000) ', Execution time: ' 18.732027960623853

Are you sure that you have used 'ccore' flag to increase performance (pass clustering process to C/C++ implementation of the pyclustering library, so called CCORE)?

xmeans_instance = xmeans(data_points, start_centers, 20, ccore = True);

And may I ask to provide data (for example upload to somewhere) for testing?
Could you please provide answer in #372 ?

[annoviko]

[ RUN      ] utest_sync.perf_test_2500_sequential
[       OK ] utest_sync.perf_test_2500_sequential (83 ms)
[ RUN      ] utest_sync.perf_test_2500_parallel
[       OK ] utest_sync.perf_test_2500_parallel (81 ms)
[ RUN      ] utest_sync.perf_test_3600_sequential
[       OK ] utest_sync.perf_test_3600_sequential (81 ms)
[ RUN      ] utest_sync.perf_test_3600_parallel
[       OK ] utest_sync.perf_test_3600_parallel (77 ms)

[lemonysoft]

What should I do for using multi core of CURE? Thanks.