Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes #16629: Use only one ZIO threadpool to enhance performances #2743

Conversation

fanf
Copy link
Member

@fanf fanf commented Jan 27, 2020

https://issues.rudder.io/issues/16629

This change leads to a 20% improvement in Rudder policy generation time, from around 25 minutes to below 20 minutes.

This change allowed to workaround three things described here: zio/zio#1275 (comment)

  • 1/ switching from a thread to an other cost a bit,
  • 2/ switching on the same threadpool is idempotent but all the fixe costs, like fiber pause etc, are paid (while it should be identity)
  • 3/ effectBlocking manage the discarding of the underlying thread, which we generally don't want.

The last two are already corrected in ZIO-RC18! And the first one is not that bad anymore.
So, I'm not sure I we should commit that complicated PR or just wait for RC18 (which is stabilising and will likely be ZIO 1.0.0)

@fanf fanf added WIP Use that label for a Work In Progress PR that must not be merged yet qa: Can't merge labels Jan 27, 2020
@fanf
Copy link
Member Author

fanf commented Jan 27, 2020

We need to test ZIO-RC18 snapshot before merging that one.

@fanf
Copy link
Member Author

fanf commented Mar 18, 2020

Following perf feedback from RC18-2 (#2817) we don't need that one anymore.

@fanf fanf closed this Mar 18, 2020
@fanf fanf deleted the bug_16629/use_only_one_zio_threadpool_to_enhance_performances branch March 15, 2024 10:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
qa: Can't merge WIP Use that label for a Work In Progress PR that must not be merged yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant