Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Give each session its own inter-thread communication channels #2

Open
dktapps opened this issue Mar 9, 2020 · 0 comments
Open

Give each session its own inter-thread communication channels #2

dktapps opened this issue Mar 9, 2020 · 0 comments

Comments

@dktapps
Copy link
Member

dktapps commented Mar 9, 2020

Currently RakLib uses two global channels for inter-thread communication (one for each direction).

These global channels are a major bottleneck for RakLib scaling because every packet sent or received from any session passes through these channels. This has two effects:

  • Frequent lock contention: it's likely that contention for mutex locking will occur when pushing stuff into channels while the other thread is trying to pull it out.
  • Inability to use multiple CPU cores: these central channels are a big showstopper for parallel scaling, because inevitably worker threads would end up contending for locks on the channels with other worker threads.

This issue proposes that each RakLib session should have its own ITC channel pair. This may be tricky to implement due to pthreads stupidity, but in terms of code complexity it's simple and effective.

dktapps referenced this issue in pmmp/RakLib Apr 29, 2021
this is a working implementation of #52.

I have my doubts that this gigantic refactor was worth the effort, because there's still lock contention on Snooze to consider; however, this does substantially reduce the problem described in pmmp/ext-pmmpthread#42, as well as reducing the overhead of inter-thread communication by removing the need to transmit session IDs.

This currently relies on each end of the IPC channels to roundtrip session open/close notifications to setup/cleanup stuff; I'll try to improve this before landing this in main.
@dktapps dktapps transferred this issue from pmmp/RakLib May 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant