Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fetching constantly static information #12034

Closed
Tracked by #12033
Zelldon opened this issue Mar 15, 2023 · 0 comments · Fixed by #12475
Closed
Tracked by #12033

Fetching constantly static information #12034

Zelldon opened this issue Mar 15, 2023 · 0 comments · Fixed by #12475
Assignees
@Zelldon
Labels
area/performance Marks an issue as performance related backport stable/8.2 Backport a pull request to 8.2.x component/engine kind/toil Categorizes an issue or PR as general maintenance, i.e. cleanup, refactoring, etc. version:8.1.11 Marks an issue as being completely or in parts released in 8.1.11 version:8.2.3 Marks an issue as being completely or in parts released in 8.2.3

Comments

@Zelldon
Copy link
Member

Zelldon commented Mar 15, 2023

Description

Related to #12033
Came up in #11813

Whenever a new process instance is started by process id, then Zeebe gets the latest version number of the process definition by fetching it from RocksDB

https://github.com/camunda/zeebe/blob/eb90558e367a107ace06240adbf8092d0b304b7a/engine/src/main/java/io/camunda/zeebe/engine/state/deployment/DbProcessState.java#L213

Since the version number doesn't change that frequently over time, the key-value pair moves to the highest level in RocksDB. Consequently, getting the version number results in a steadily growing number of hits an L3.

To avoid the lookup with every process instance to create, the version number could be cached and only be retrieved from RocksDB when not available, see 211a2ca That way, the hits in L3 (i.e., L2 and up) stops growing over time and remain constant.
@Zelldon Zelldon added kind/toil Categorizes an issue or PR as general maintenance, i.e. cleanup, refactoring, etc. area/performance Marks an issue as performance related component/engine labels Mar 15, 2023
@Zelldon Zelldon mentioned this issue Mar 15, 2023
Closed
@Zelldon Zelldon self-assigned this Apr 4, 2023
@Zelldon Zelldon mentioned this issue Apr 18, 2023
Merged
14 tasks
@megglos megglos added the backport stable/8.2 Backport a pull request to 8.2.x label Apr 20, 2023
@backport-action backport-action mentioned this issue Apr 20, 2023
Merged
zeebe-bors-camunda bot added a commit that referenced this issue Apr 20, 2023
@zeebe-bors-camunda @Zelldon
merge: #12495
012de45 
12495: [Backport stable/8.2] Introduce cache for process versions r=megglos a=backport-action

# Description
Backport of #12475 to `stable/8.2`.

relates to #12034

Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
@megglos megglos mentioned this issue Apr 20, 2023
Merged
zeebe-bors-camunda bot added a commit that referenced this issue Apr 21, 2023
@zeebe-bors-camunda @Zelldon
merge: #12496
7931aa0 
12496: [Backport stable/8.1] Introduce cache for process versions r=oleschoenburg a=megglos

# Description
Backport of #12475 to `stable/8.1`.

Note: This doesn't contain the rename commit ec2f789 of the NextValueManager to ProcessVersionManager, as `NextValueManager` is used in other occasions on the 8.1 branch.

relates to #12034

Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
@lenaschoenburg lenaschoenburg added version:8.2.3 Marks an issue as being completely or in parts released in 8.2.3 version:8.1.11 Marks an issue as being completely or in parts released in 8.1.11 labels Apr 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Zelldon Zelldon

Labels
area/performance Marks an issue as performance related backport stable/8.2 Backport a pull request to 8.2.x component/engine kind/toil Categorizes an issue or PR as general maintenance, i.e. cleanup, refactoring, etc. version:8.1.11 Marks an issue as being completely or in parts released in 8.1.11 version:8.2.3 Marks an issue as being completely or in parts released in 8.2.3
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Introduce cache for process versions camunda/camunda
3 participants
@megglos @lenaschoenburg @Zelldon