Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

main: fd_listen fhs alloc rewrite #805

Merged
merged 14 commits into from
Sep 18, 2023
Merged

main: fd_listen fhs alloc rewrite #805

merged 14 commits into from
Sep 18, 2023

Conversation

sreimers
Copy link
Member

@sreimers sreimers commented May 15, 2023

If an application uses multiple (maybe thousands) of libre event loop threads, the unused pre-allocated memory for fhs can be quite high. This PR uses a application allocated fhs with fd_listen. This way EPOLL and KQUEUE have no hard fd limit anymore (re->maxfds limits only returned active event handlers per fd_poll). The fd limit for SELECT stays the same.

Alternative approach with less overhead compared to #428

src/udp/udp.c Outdated Show resolved Hide resolved
@sreimers sreimers added this to the v3.3.0 milestone May 19, 2023
@alfredh alfredh removed this from the v3.3.0 milestone Jun 6, 2023
src/mbuf/mbuf.c Outdated Show resolved Hide resolved
@sreimers sreimers force-pushed the fhs_alloc branch 3 times, most recently from dfe99a9 to df5aeeb Compare August 16, 2023 14:19
@sreimers sreimers added this to the v3.5.0 milestone Aug 18, 2023
@sreimers sreimers changed the title main: fhs alloc main: fd_listen fhs alloc rewrite Aug 18, 2023
@sreimers
Copy link
Member Author

@Lastique Can you review if this PR matches your use case?

@sreimers sreimers marked this pull request as ready for review August 21, 2023 09:24
@sreimers sreimers marked this pull request as draft August 29, 2023 08:18
@sreimers sreimers marked this pull request as ready for review August 29, 2023 16:26
@sreimers sreimers modified the milestones: v3.5.0, v3.6.0 Aug 30, 2023
Copy link
Contributor

@Lastique Lastique left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the first glance, it looks like the general idea should work in our application. Though, I didn't test it, so I could be missing something.

src/main/main.c Outdated Show resolved Hide resolved
include/re_main.h Outdated Show resolved Hide resolved
src/main/main.c Outdated Show resolved Hide resolved
src/main/main.c Outdated Show resolved Hide resolved
src/main/main.c Show resolved Hide resolved
src/main/main.c Outdated Show resolved Hide resolved
include/re_main.h Outdated Show resolved Hide resolved
@sreimers sreimers merged commit 0a6fe33 into main Sep 18, 2023
44 checks passed
@sreimers sreimers deleted the fhs_alloc branch September 18, 2023 08:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants