Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify configuration of threads in relation to partition count #8741

Open
Zelldon opened this issue Feb 5, 2022 · 3 comments
Open

Simplify configuration of threads in relation to partition count #8741

Zelldon opened this issue Feb 5, 2022 · 3 comments
Labels
component/scheduler component/zeebe Related to the Zeebe component/team kind/feature Categorizes an issue or PR as a feature, i.e. new behavior scope/broker Marks an issue or PR to appear in the broker section of the changelog

Comments

@Zelldon
Copy link
Member

Zelldon commented Feb 5, 2022

Is your feature request related to a problem? Please describe.

Users have issues to understand or see the relation between CPU and IO threads and partition counts on a node. We should try to make the users life easier and simplify this configuration or made it possible to configure with one thing multiple configurations, similar to the maxMessageSize, which sets also several properties in Zeebe.

Example: https://camunda-cloud.slack.com/archives/C6WGNHV2A/p1644039751236089?thread_ts=1643807352.592009&cid=C6WGNHV2A

Describe the solution you'd like

I would like to we have a property like the partition count per node, which directly also sets the need threads. For example if I set two partitions that I get two threads for processing (cpu threads) and two for exporting (io threads). Ideally if needed I could still configure them separately.

Describe alternatives you've considered

  1. Rename the properties to make the purpose or usage more clearer.
  2. Do nothing and cry

Additional context

I think it would make sense to make the users life in certain places easier, Zeebe is already complicated enough.

@Zelldon Zelldon added kind/feature Categorizes an issue or PR as a feature, i.e. new behavior scope/broker Marks an issue or PR to appear in the broker section of the changelog labels Feb 5, 2022
@aivinog1
Copy link
Contributor

aivinog1 commented Feb 7, 2022

I'm voting for that thread count should be equal by default partitions count. But it should be documented somehow of course :)

@Zelldon
Copy link
Member Author

Zelldon commented Feb 7, 2022

I did a small test and saw that with our current benchmarks for example we also get better results if they are equal to the partition count.

Thread count == Partition Count (=3)

threadeq

Our normal medic benchmark, where thread count is 4 and partition count 3.

medic

@npepinpe
Copy link
Member

Seems reasonable, however ease of configuration isn't a main focus at the moment. I'm happy to consider a more concrete proposal and accept a PR for it, but at this time we probably won't work on this.

@romansmirnov romansmirnov added the component/zeebe Related to the Zeebe component/team label Mar 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/scheduler component/zeebe Related to the Zeebe component/team kind/feature Categorizes an issue or PR as a feature, i.e. new behavior scope/broker Marks an issue or PR to appear in the broker section of the changelog
Projects
None yet
Development

No branches or pull requests

5 participants