Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Launch Synapse processes using fork() in Complement with workers CI #13051

Closed
reivilibre opened this issue Jun 14, 2022 · 0 comments · Fixed by #13127
Closed

Launch Synapse processes using fork() in Complement with workers CI #13051

reivilibre opened this issue Jun 14, 2022 · 0 comments · Fixed by #13127
Assignees
@reivilibre
Labels
T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks.
Milestone
Q2 2022 — Testing...

Comments

@reivilibre
Copy link
Contributor

reivilibre commented Jun 14, 2022 •
edited

The basic idea is that we load the Python bytecode once and then fork to spawn all the needed worker processes, thus reducing CPU contention.

Empirical attempts show that this reduces CI time of Complement-with-workers by a half.

It's an attempt at making #12848 better.
@reivilibre reivilibre added the T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks. label Jun 14, 2022
@reivilibre reivilibre self-assigned this Jun 14, 2022
This was referenced Jun 14, 2022
Merged
Merged
@reivilibre reivilibre mentioned this issue Jun 28, 2022
Merged
@reivilibre reivilibre mentioned this issue Jul 1, 2022
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@reivilibre reivilibre

Labels
T-Task Refactoring, removal, replacement, enabling or disabling functionality, other engineering tasks.
Projects
None yet
Milestone
Q2 2022 — Testing Workerised Synapse ...
1 participant
@reivilibre