Learner2D + BalancingLearner too slow to use on many cores

[basnijholt]

(original issue on GitLab)

opened by Bas Nijholt (@basnijholt) at 2018-06-05T23:28:56.932Z

I am trying to run 50 Learner2Ds inside of a BalacingLearner on 1500 cores.

The code is currently too slow to handle so many cores at once because the BalancingLearner requests too many points for the local process to handle, basically the notebook becomes completely unusable.

Whenever https://gitlab.kwant-project.org/qt/adaptive/merge_requests/34 is implemented we could probably write a large part of the Learner2D in cython to get a massive speedup.

[basnijholt]

originally posted by Jorn Hoofwijk (@Jorn) at 2018-09-25T11:57:52.193Z on GitLab

Partially related to gitlab:!109 yet I am not sure that gitlab:!109 will solve this issue

[basnijholt]

originally posted by Bas Nijholt (@basnijholt) at 2018-12-07T19:18:43.218Z on GitLab

This indeed got better with gitlab:!109. I don't think there is a good reason to keep this issue open.