Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testsuite uses too many processes #755

Closed
kanigsson opened this issue Aug 26, 2021 · 2 comments · Fixed by #756
Closed

Testsuite uses too many processes #755

kanigsson opened this issue Aug 26, 2021 · 2 comments · Fixed by #756
Assignees
Labels

Comments

@kanigsson
Copy link
Collaborator

The RecordFlux testsuite runs too many processes at the same time currently, this is probably because of the new support for multiprocessing #625. My suspicion is that the test driver runs n processes in parallel, which then each attempt to use n cores at the same time, where n is the number of available cores.

A possible solution is to add a configuration option to RecordFlux to control the number of processes to be used in parallel, and set it to 1 when run in the testsuite. The testsuite driver would continue to run n processes in parallel.

One could also do it the other way around, but usually there are many small tests that don't benefit from parallelism, so it seems the better tradeoff.

@senier
Copy link
Member

senier commented Aug 26, 2021

A possible solution is to add a configuration option to RecordFlux to control the number of processes to be used in parallel, and set it to 1 when run in the testsuite. The testsuite driver would continue to run n processes in parallel.

That's how I've done it. Parallelism can now be configured in the constructor of the Message class, which as a default value of 1. In the CLI this value is configured to the number of available cores while the default is used in the tests (which themselves are parallelized). @kanigsson Please try whether the issue_755 branch fixes the issue on your system.

@senier senier moved this from In progress to Under review in RecordFlux 0.6 Aug 26, 2021
@kanigsson
Copy link
Collaborator Author

kanigsson commented Aug 27, 2021

Yes, CPU load is back to normal with this patch.

RecordFlux 0.6 automation moved this from Under review to Merged Aug 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

2 participants