Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multi-Instance with messageevent-based subprocess that uses inputElement as correlationKey fails #11578

Closed
j-lindner opened this issue Feb 8, 2023 · 3 comments · Fixed by #12877
Closed

Multi-Instance with messageevent-based subprocess that uses inputElement as correlationKey fails #11578

j-lindner opened this issue Feb 8, 2023 · 3 comments · Fixed by #12877
Assignees
@korthout
Labels
component/engine kind/bug Categorizes an issue or PR as a bug severity/mid Marks a bug as having a noticeable impact but with a known workaround support Marks an issue as related to a customer support request version:8.1.13 Marks an issue as being completely or in parts released in 8.1.13 version:8.2.6 Marks an issue as being completely or in parts released in 8.2.6 version:8.3.0-alpha2 Marks an issue as being completely or in parts released in 8.3.0-alpha2 version:8.3.0 Marks an issue as being completely or in parts released in 8.3.0

Comments

@j-lindner
Copy link
Contributor

j-lindner commented Feb 8, 2023 •
edited by abbasadel

Describe the bug
Multi-Instance doesn't get started if it contains a Message Event-based Subprocess that uses the inputElement as correlationKey. Instead an Incident gets created.

To Reproduce
Update: For better understanding I created a GitHub Repo with a minimalistic example: https://github.com/j-lindner/multi-instance-with-message-subprocess-bug

Expected behavior
Multi-Instance should get started, get processed and should listen to possible Messages on the event-based subprocess.

Log/Stacktrace

Incident Info in Operate:

failed to evaluate expression 'myObject.myId': no variable found for name 'myObject'

Environment:

  • OS: WebModeler or alternatively zeebe-process-test-extension-testcontainer
  • Zeebe Version: 8.1.7
  • Configuration: -

Support:
https://jira.camunda.com/browse/SUPPORT-17059
@j-lindner j-lindner added kind/bug Categorizes an issue or PR as a bug component/engine labels Feb 8, 2023
@remcowesterhoud remcowesterhoud self-assigned this Feb 15, 2023
@remcowesterhoud
Copy link
Contributor

Thanks for reporting and reproducing @j-lindner! This is very helpful 🚀

I will have chat within the team to see if this is expected behaviour or not. From there we will prioritise it accordingly.

@remcowesterhoud remcowesterhoud added the severity/mid Marks a bug as having a noticeable impact but with a known workaround label Feb 15, 2023
@remcowesterhoud remcowesterhoud removed their assignment Feb 15, 2023
@abbasadel abbasadel added the support Marks an issue as related to a customer support request label May 24, 2023
@korthout korthout self-assigned this May 26, 2023
korthout added a commit that referenced this issue May 26, 2023
@korthout
test: trigger event subprocess in multi-instance
b4d5ec8 
An event sub process existing inside a multi-instance sub process should
exist for each inner instance of the multi-instance. Each event sub
process should be able to be triggered individually.

There was a bug #11578, where the
event sub process's message start event could not create a message
subscription because the correlation key expression referred to the
input element. The input element exists on each inner instance, but the
engine attempted to subscribe to the event sub process before activating
any of the inner instances. In turn, this also means that the event sub
processes couldn't be triggered.

Instead of testing that we can subscribe to each event sub process, it
makes more sense to also test that we can trigger each individually.
This tests the reason why we want to be able to subscribe in the first
place.
korthout added a commit that referenced this issue May 26, 2023
@korthout
fix: don't attach event subprocess to multi-instance body
abb5bb1 
The event sub process should only be attached to the inner instance of
the multi-instance marked element. Not to the body.

The event sub process should be triggered for a specific instance of a
multi-instance marked element only. It only exists in the flow scope of
the inner instance, not in the flow scope of the multi-instance body.

There was a bug #11578 with this
as the root problem. It surfaced because the event sub process's start
event had a correlation key expression that referred to the input
element as variable. That variable had not yet been created when we
subscribed the multi-instance body to the event sub process. We should
not have attempted to subscribe the multi-instance body to the event sub
process. Instead, each inner instance of the multi-instance should be
subscribed to its own instance of the event sub process.

See the previous commit for a test case with that scenario.
@korthout korthout mentioned this issue May 26, 2023
Merged
14 tasks
@korthout
Copy link
Member

Thanks for reporting this @j-lindner 👍 It was easy to reproduce in one of Zeebe's engine test cases as well. I think I have a fix, but let's await the review.

zeebe-bors-camunda bot added a commit that referenced this issue May 30, 2023
@zeebe-bors-camunda @korthout
merge: #12877
486f928 
12877: Event subprocess attaches to inner instance of multi-instance only r=saig0 a=korthout

## Description

<!-- Please explain the changes you made here. -->

Event sub processes inside a multi-instance sub process would be attached to the multi-instance body in addition to the inner instance. However, there is no reason why an event sub process should be attached to the multi-instance body. Instead, event sub processes should only be attached to the inner instances.

**Example scenario**

The parallel multi-instance sub process creates an `item` variable as input element in each inner instance.
<img width="865" alt="Screenshot 2023-05-26 at 18 25 08" src="https://github.com/camunda/zeebe/assets/3511026/cf37717b-50cf-4478-84d3-d32009f54972">

Each message event sub process subscribes to the same message with a correlation key expression that uses the input element variable `item`.
<img width="922" alt="Screenshot 2023-05-26 at 18 25 39" src="https://github.com/camunda/zeebe/assets/3511026/9fc40f17-a2e7-4628-a9fd-a70cb6bffe90">

The bug will surface as an incident on the multi-instance body stating:

>failed to evaluate expression 'item.id': no variable found for name 'item'

This was because we attempted to subscribe to the event sub process already when activating the multi-instance body, and **before** activating any of the inner instances.

The problem is resolved by no longer attaching the event sub process to the multi-instance body.

## Related issues

<!-- Which issues are closed by this PR or are related -->

closes #11578



Co-authored-by: Nico Korthout <nico.korthout@camunda.com>
zeebe-bors-camunda bot added a commit that referenced this issue May 30, 2023
@zeebe-bors-camunda @npepinpe @korthout
merge: #12698 #12868 #12877
f30b8ed 
12698: Replace go-bindata with go's embed FS r=npepinpe a=npepinpe

## Description

Took some slack time to complete this quick win: replaces usage of go-bindata, an external tool dependency, with the built-in capabilities of the go compiler using `go:embed` and the
embedFS.

The version file is embedded at build time into the binary, much like when we used go-bindata, and read back as a package field.

## Related issues

closes #6034 



12868: fix(journal): avoid gaps in logs due to reset r=npepinpe a=npepinpe

## Description

Avoid gaps in the log when resetting the log by deleting the segments in reverse order. This may cause data loss, but that's acceptable in the context of a reset operation. Deleting segments in reverse order will ensure we have no gaps between the remaining snapshot and segments should the operation fail midway.

Note that this does not make persisting a new snapshot/reseting the log atomic unfortunately.

Unfortunately tests were also not ideal, and require heavy use of mocks. It's also not possible to write integration test without major refactoring. As this is a quick fix, the hope is that the proper solution to make the Raft install operation atomic will redeem this.

Not marked as closing the original issue because this is, after all, a quick fix. It only fixes a specific case we're aware of.

## Related issues

<!-- Which issues are closed by this PR or are related -->

related to #12754 



12877: Event subprocess attaches to inner instance of multi-instance only r=saig0 a=korthout

## Description

<!-- Please explain the changes you made here. -->

Event sub processes inside a multi-instance sub process would be attached to the multi-instance body in addition to the inner instance. However, there is no reason why an event sub process should be attached to the multi-instance body. Instead, event sub processes should only be attached to the inner instances.

**Example scenario**

The parallel multi-instance sub process creates an `item` variable as input element in each inner instance.
<img width="865" alt="Screenshot 2023-05-26 at 18 25 08" src="https://github.com/camunda/zeebe/assets/3511026/cf37717b-50cf-4478-84d3-d32009f54972">

Each message event sub process subscribes to the same message with a correlation key expression that uses the input element variable `item`.
<img width="922" alt="Screenshot 2023-05-26 at 18 25 39" src="https://github.com/camunda/zeebe/assets/3511026/9fc40f17-a2e7-4628-a9fd-a70cb6bffe90">

The bug will surface as an incident on the multi-instance body stating:

>failed to evaluate expression 'item.id': no variable found for name 'item'

This was because we attempted to subscribe to the event sub process already when activating the multi-instance body, and **before** activating any of the inner instances.

The problem is resolved by no longer attaching the event sub process to the multi-instance body.

## Related issues

<!-- Which issues are closed by this PR or are related -->

closes #11578



Co-authored-by: Nicolas Pepin-Perreault <nicolas.pepin-perreault@camunda.com>
Co-authored-by: Nico Korthout <nico.korthout@camunda.com>
@backport-action backport-action mentioned this issue May 30, 2023
Merged
backport-action pushed a commit that referenced this issue May 30, 2023
@korthout @github-actions
test: trigger event subprocess in multi-instance
f2da27e 
An event sub process existing inside a multi-instance sub process should
exist for each inner instance of the multi-instance. Each event sub
process should be able to be triggered individually.

There was a bug #11578, where the
event sub process's message start event could not create a message
subscription because the correlation key expression referred to the
input element. The input element exists on each inner instance, but the
engine attempted to subscribe to the event sub process before activating
any of the inner instances. In turn, this also means that the event sub
processes couldn't be triggered.

Instead of testing that we can subscribe to each event sub process, it
makes more sense to also test that we can trigger each individually.
This tests the reason why we want to be able to subscribe in the first
place.

(cherry picked from commit b4d5ec8)
backport-action pushed a commit that referenced this issue May 30, 2023
@korthout @github-actions
fix: don't attach event subprocess to multi-instance body
2f50670 
The event sub process should only be attached to the inner instance of
the multi-instance marked element. Not to the body.

The event sub process should be triggered for a specific instance of a
multi-instance marked element only. It only exists in the flow scope of
the inner instance, not in the flow scope of the multi-instance body.

There was a bug #11578 with this
as the root problem. It surfaced because the event sub process's start
event had a correlation key expression that referred to the input
element as variable. That variable had not yet been created when we
subscribed the multi-instance body to the event sub process. We should
not have attempted to subscribe the multi-instance body to the event sub
process. Instead, each inner instance of the multi-instance should be
subscribed to its own instance of the event sub process.

See the previous commit for a test case with that scenario.

(cherry picked from commit abb5bb1)
@backport-action backport-action mentioned this issue May 30, 2023
Merged
backport-action pushed a commit that referenced this issue May 30, 2023
@korthout @github-actions
test: trigger event subprocess in multi-instance
e7711aa 
An event sub process existing inside a multi-instance sub process should
exist for each inner instance of the multi-instance. Each event sub
process should be able to be triggered individually.

There was a bug #11578, where the
event sub process's message start event could not create a message
subscription because the correlation key expression referred to the
input element. The input element exists on each inner instance, but the
engine attempted to subscribe to the event sub process before activating
any of the inner instances. In turn, this also means that the event sub
processes couldn't be triggered.

Instead of testing that we can subscribe to each event sub process, it
makes more sense to also test that we can trigger each individually.
This tests the reason why we want to be able to subscribe in the first
place.

(cherry picked from commit b4d5ec8)
backport-action pushed a commit that referenced this issue May 30, 2023
@korthout @github-actions
fix: don't attach event subprocess to multi-instance body
60d93da 
The event sub process should only be attached to the inner instance of
the multi-instance marked element. Not to the body.

The event sub process should be triggered for a specific instance of a
multi-instance marked element only. It only exists in the flow scope of
the inner instance, not in the flow scope of the multi-instance body.

There was a bug #11578 with this
as the root problem. It surfaced because the event sub process's start
event had a correlation key expression that referred to the input
element as variable. That variable had not yet been created when we
subscribed the multi-instance body to the event sub process. We should
not have attempted to subscribe the multi-instance body to the event sub
process. Instead, each inner instance of the multi-instance should be
subscribed to its own instance of the event sub process.

See the previous commit for a test case with that scenario.

(cherry picked from commit abb5bb1)
@backport-action backport-action mentioned this issue May 30, 2023
Merged
backport-action pushed a commit that referenced this issue May 30, 2023
@korthout @github-actions
test: trigger event subprocess in multi-instance
c9c6875 
An event sub process existing inside a multi-instance sub process should
exist for each inner instance of the multi-instance. Each event sub
process should be able to be triggered individually.

There was a bug #11578, where the
event sub process's message start event could not create a message
subscription because the correlation key expression referred to the
input element. The input element exists on each inner instance, but the
engine attempted to subscribe to the event sub process before activating
any of the inner instances. In turn, this also means that the event sub
processes couldn't be triggered.

Instead of testing that we can subscribe to each event sub process, it
makes more sense to also test that we can trigger each individually.
This tests the reason why we want to be able to subscribe in the first
place.

(cherry picked from commit b4d5ec8)
backport-action pushed a commit that referenced this issue May 30, 2023
@korthout @github-actions
fix: don't attach event subprocess to multi-instance body
132acff 
The event sub process should only be attached to the inner instance of
the multi-instance marked element. Not to the body.

The event sub process should be triggered for a specific instance of a
multi-instance marked element only. It only exists in the flow scope of
the inner instance, not in the flow scope of the multi-instance body.

There was a bug #11578 with this
as the root problem. It surfaced because the event sub process's start
event had a correlation key expression that referred to the input
element as variable. That variable had not yet been created when we
subscribed the multi-instance body to the event sub process. We should
not have attempted to subscribe the multi-instance body to the event sub
process. Instead, each inner instance of the multi-instance should be
subscribed to its own instance of the event sub process.

See the previous commit for a test case with that scenario.

(cherry picked from commit abb5bb1)
zeebe-bors-camunda bot added a commit that referenced this issue May 30, 2023
@zeebe-bors-camunda @korthout
merge: #12893
960535e 
12893: [Backport stable/8.2] Event subprocess attaches to inner instance of multi-instance only r=korthout a=backport-action

# Description
Backport of #12877 to `stable/8.2`.

relates to #11578

Co-authored-by: Nico Korthout <nico.korthout@camunda.com>
zeebe-bors-camunda bot added a commit that referenced this issue May 30, 2023
@zeebe-bors-camunda @korthout
merge: #12892
104cb4c 
12892: [Backport stable/8.1] Event subprocess attaches to inner instance of multi-instance only r=korthout a=backport-action

# Description
Backport of #12877 to `stable/8.1`.

relates to #11578

Co-authored-by: Nico Korthout <nico.korthout@camunda.com>
zeebe-bors-camunda bot added a commit that referenced this issue May 30, 2023
@zeebe-bors-camunda @korthout
merge: #12890
5ff3043 
12890: [Backport stable/8.0] Event subprocess attaches to inner instance of multi-instance only r=korthout a=backport-action

# Description
Backport of #12877 to `stable/8.0`.

relates to #11578

Co-authored-by: Nico Korthout <nico.korthout@camunda.com>
zeebe-bors-camunda bot added a commit that referenced this issue May 30, 2023
@zeebe-bors-camunda @korthout
merge: #12892
3f711a0 
12892: [Backport stable/8.1] Event subprocess attaches to inner instance of multi-instance only r=korthout a=backport-action

# Description
Backport of #12877 to `stable/8.1`.

relates to #11578

Co-authored-by: Nico Korthout <nico.korthout@camunda.com>
zeebe-bors-camunda bot added a commit that referenced this issue May 31, 2023
@zeebe-bors-camunda @korthout
@dependabot @lenaschoenburg
merge: #12890 #12908
2884a0e 
12890: [Backport stable/8.0] Event subprocess attaches to inner instance of multi-instance only r=korthout a=backport-action

# Description
Backport of #12877 to `stable/8.0`.

relates to #11578

12908: [Backport stable/8.0] Remove JDK 8 specific build r=oleschoenburg a=npepinpe

## Description

This PR backports removing the JDK 8 build, and upgrading Mockito to 5.

## Related issues

backport #12533



Co-authored-by: Nico Korthout <nico.korthout@camunda.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ole Schönburg <ole.schoenburg@gmail.com>
@korthout
Copy link
Member

@j-lindner Bug is fixed and will be patched in upcoming releases of 8.0.15, 8.1.13, 8.2.6, and 8.3.0. You should notice that this issue will be labeled accordingly during those releases.

@lenaschoenburg lenaschoenburg added version:8.3.0-alpha2 Marks an issue as being completely or in parts released in 8.3.0-alpha2 version:8.1.13 Marks an issue as being completely or in parts released in 8.1.13 version:8.2.6 Marks an issue as being completely or in parts released in 8.2.6 release/8.0.16 labels Jun 7, 2023
@megglos megglos added the version:8.3.0 Marks an issue as being completely or in parts released in 8.3.0 label Oct 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@korthout korthout

Labels
component/engine kind/bug Categorizes an issue or PR as a bug severity/mid Marks a bug as having a noticeable impact but with a known workaround support Marks an issue as related to a customer support request version:8.1.13 Marks an issue as being completely or in parts released in 8.1.13 version:8.2.6 Marks an issue as being completely or in parts released in 8.2.6 version:8.3.0-alpha2 Marks an issue as being completely or in parts released in 8.3.0-alpha2 version:8.3.0 Marks an issue as being completely or in parts released in 8.3.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Event subprocess attaches to inner instance of multi-instance only camunda/camunda
6 participants
@megglos @abbasadel @lenaschoenburg @korthout @remcowesterhoud @j-lindner