Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Avoid parsing DMN decisions for every evaluation #8571

Closed
saig0 opened this issue Jan 11, 2022 · 3 comments · Fixed by #13831
Closed

Avoid parsing DMN decisions for every evaluation #8571

saig0 opened this issue Jan 11, 2022 · 3 comments · Fixed by #13831
Assignees
@megglos @remcowesterhoud
Labels
area/performance Marks an issue as performance related backport stable/8.2 Backport a pull request to 8.2.x component/engine impact/high Marks issues relieving a recurring pain or fulfilling an immediate need kind/toil Categorizes an issue or PR as general maintenance, i.e. cleanup, refactoring, etc. scope/broker Marks an issue or PR to appear in the broker section of the changelog support Marks an issue as related to a customer support request version:8.1.16 Marks an issue as being completely or in parts released in 8.1.16 version:8.2.12 Marks an issue as being completely or in parts released in 8.2.12 version:8.3.0-alpha5 Marks an issue as being completely or in parts released in 8.3.0-alpha5 version:8.3.0-alpha6 Marks an issue as being completely or in parts released in 8.3.0-alpha6 version:8.3.0 Marks an issue as being completely or in parts released in 8.3.0
Milestone
Evaluate DMN deci...

Comments

@saig0
Copy link
Member

saig0 commented Jan 11, 2022 •
edited by megglos
Loading

Description

The DMN engine parses a DMN resource into an in-memory data object that is used to evaluate the decision. The parsing of DMN decisions is (relatively) costly. Instead of parsing it every time, we can cache the parsed DMN (e.g. using an LRU cache).

  • store the parsed DMN decisions in a cache
  • the cache should have a fixed capacity (i.e. using an LRU cache - similar to the cache from agrona)
  • the cache may be used transparently in a behavior class and is filled by the DMN state

Relates to https://jira.camunda.com/browse/SUPPORT-17656
@saig0 saig0 added kind/toil Categorizes an issue or PR as general maintenance, i.e. cleanup, refactoring, etc. scope/broker Marks an issue or PR to appear in the broker section of the changelog labels Jan 11, 2022
@saig0 saig0 added this to the Evaluate DMN decisions milestone Jan 11, 2022
@saig0 saig0 mentioned this issue Jan 11, 2022
Closed
43 tasks
@saig0 saig0 added this to Planned in Zeebe Jan 11, 2022
@KerstinHebel KerstinHebel removed this from Planned in Zeebe Mar 23, 2022
@saig0 saig0 added the area/performance Marks an issue as performance related label Jul 19, 2023
@megglos megglos added the support Marks an issue as related to a customer support request label Jul 28, 2023
@megglos
Copy link
Contributor

megglos commented Jul 28, 2023

A PoC to assess the impact of this improvement would be a great first scope.

@megglos megglos added the planning/discuss To be discussed at the next planning. label Jul 28, 2023
@megglos
Copy link
Contributor

megglos commented Jul 28, 2023 •
edited
Loading

ZDP-Planning:

  • it surfaced for 3-4+ customers so far
  • PoC focus would be: build a prototype to be evaluated in a performance test

@megglos megglos removed the planning/discuss To be discussed at the next planning. label Jul 28, 2023
@megglos megglos mentioned this issue Jul 31, 2023
Closed
@megglos
Copy link
Contributor

megglos commented Aug 3, 2023 •
edited
Loading

As synced with @remcowesterhoud here #13713 (comment)
we need to include this into the next iteration aiming for the September patch or earlier for two customers

@remcowesterhoud remcowesterhoud added the impact/high Marks issues relieving a recurring pain or fulfilling an immediate need label Aug 3, 2023
@megglos megglos assigned remcowesterhoud and unassigned npepinpe Aug 3, 2023
@megglos megglos added backport stable/8.1 backport stable/8.2 Backport a pull request to 8.2.x and removed backport stable/8.1 labels Aug 3, 2023
@remcowesterhoud remcowesterhoud mentioned this issue Aug 8, 2023
Merged
14 tasks
@remcowesterhoud remcowesterhoud mentioned this issue Aug 14, 2023
Merged
zeebe-bors-camunda bot added a commit that referenced this issue Aug 14, 2023
@zeebe-bors-camunda @remcowesterhoud
merge: #13874
aa66642 
13874: [Backport stable/8.2] Cache parsed DRGs r=koevskinikola a=remcowesterhoud

# Description
Backport of #13831 to `stable/8.2`.

relates to #8571 

Co-authored-by: Remco Westerhoud <remco@westerhoud.nl>
@remcowesterhoud remcowesterhoud mentioned this issue Aug 14, 2023
Merged
zeebe-bors-camunda bot added a commit that referenced this issue Aug 14, 2023
@zeebe-bors-camunda @remcowesterhoud
merge: #13882
3e4bfa8 
13882: [Backport stable/8.1] Cache parsed DRGs r=koevskinikola a=remcowesterhoud

# Description
Backport of #13831 to `stable/8.1`.

relates to #8571 

---------

This one was some effort to backport as at some point we extracted some functionality of the `BpmnDecisionBehavior` to a more generic `DecisionBehavior` which is used for standalone decision evaluation. This doesn't exist on 8.1 so I had to recreate some of the changes in the `BpmnDecisionBehavior` instead.

Co-authored-by: Remco Westerhoud <remco@westerhoud.nl>
@megglos megglos added the version:8.2.12 Marks an issue as being completely or in parts released in 8.2.12 label Aug 16, 2023
@abbasadel abbasadel added version:8.3.0-alpha5 Marks an issue as being completely or in parts released in 8.3.0-alpha5 version:8.1.16 Marks an issue as being completely or in parts released in 8.1.16 version:8.3.0-alpha6 Marks an issue as being completely or in parts released in 8.3.0-alpha6 labels Sep 7, 2023
@megglos megglos added the version:8.3.0 Marks an issue as being completely or in parts released in 8.3.0 label Oct 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@megglos megglos

@remcowesterhoud remcowesterhoud

Labels
area/performance Marks an issue as performance related backport stable/8.2 Backport a pull request to 8.2.x component/engine impact/high Marks issues relieving a recurring pain or fulfilling an immediate need kind/toil Categorizes an issue or PR as general maintenance, i.e. cleanup, refactoring, etc. scope/broker Marks an issue or PR to appear in the broker section of the changelog support Marks an issue as related to a customer support request version:8.1.16 Marks an issue as being completely or in parts released in 8.1.16 version:8.2.12 Marks an issue as being completely or in parts released in 8.2.12 version:8.3.0-alpha5 Marks an issue as being completely or in parts released in 8.3.0-alpha5 version:8.3.0-alpha6 Marks an issue as being completely or in parts released in 8.3.0-alpha6 version:8.3.0 Marks an issue as being completely or in parts released in 8.3.0
Projects
None yet
Milestone
Evaluate DMN decisions
Development

Successfully merging a pull request may close this issue.

Cache parsed DRGs camunda/camunda
7 participants
@npepinpe @megglos @menski @abbasadel @Zelldon @saig0 @remcowesterhoud