Message correlation

BPMN message events connect a workflow to the outside world. VanillaBP's API for this is small on purpose. The message's content never travels to the BPMS (business process management system), because the workflow aggregate is the single source of truth. First put the data of an incoming message into the aggregate, then correlate. Correlating means handing the message to the workflow which waits for it. BPMN expressions read the aggregate, not message payloads.

1. Correlating with a running workflow
   The call, the transaction it belongs in, and what happens when the workflow is not there.
2. Correlation ids: several occurrences of the same message
   How to address one of several catch events which wait for the same message name.
3. Idempotency
   What is deduplicated with a correlation id and what is not without one, inside a handler and outside it.
4. Starting a workflow by message
   The one call for a process which begins with a message start event, and why it starts no other process.
5. BPMS specifics
   What your model has to carry per BPMS, and what fails at your call site and not later.
6. Neither is a changed aggregate
   When to push the aggregate instead of inventing a message for it.
7. Signals are not messages
   What a broadcast reaches, what it costs, and why it takes no aggregate.

Correlating with a running workflow

@Transactional
public void paymentReceived(final String rideId, final PaymentDetails payment) {
    final var ride = rides.findById(rideId).orElseThrow();
    ride.setPayment(payment.amount());               // aggregate = source of truth
    bpms.correlateMessage(ride, "PaymentReceived");
}

• Call it within a transaction. The correlation runs on the BPMS AFTER your commit (transaction-outbox pattern), so a rollback removes it too. An embedded engine could join your transaction, but on purpose it does not. A command which loses a concurrency conflict marks that transaction as rollback-only, so it cannot be repeated inside it. The outbox, in contrast, repeats it until it succeeds. The wiki of the Camunda 7 adapter says what such an engine still answers while your transaction is open. (why)
• VanillaBP finds the BPMS which runs the workflow by itself. It asks the prioritized adapters, and caches the answer after the first successful lookup. So the migration feature applies to correlation, too.
• For an unknown workflow, a WorkflowNotFoundException explains the causes and tells you what to do: never started, started elsewhere, or already ended. If you meant to start the workflow, it points to startWorkflowByMessage. If the workflow is reported as completed, correlating does nothing and writes a warning.
• Correlating right after the start is fine. This is also true on a BPMS which needs a moment before it can find a workflow it just created. VanillaBP knows where it started that workflow and keeps asking there, see the election cache.
• The technical correlation key is the workflow aggregate's ID, so you do not have to model anything for it (see the BPMS specifics below).

Correlation ids: several occurrences of the same message

correlateMessage(aggregate, messageName, correlationId) tells apart several waiting occurrences of the same message within one workflow, for example one catch event per ordered item. Whether the BPMN model has to be prepared for it, and how the id is matched against the waiting occurrence, depends on the BPMS (see below).

Idempotency

• With a correlation id, the operation deduplicates the correlations which still wait to reach the BPMS. If you schedule the same correlation twice inside one transaction, or again before the first one was sent, nothing happens. If you correlate the same message name and correlation id again after the first one reached the BPMS, this is a NEW operation and it is carried out. This is what makes a second round of a loop work. Say an offer round asks partner 42 again. It sends the same message name with the same correlation id, and VanillaBP no longer swallows it. (why)
• Without a correlation id there is NO deduplication, on purpose. The same message may arrive several times over a workflow's lifetime, and rightly so. As a result, a crash between sending and acknowledgement may correlate twice (at-least-once). Model catch events so that they are idempotent, or use correlation ids where this matters.

Correlating from inside a task handler

A correlation planned while a @WorkflowTask method runs carries the element instance which the BPMS is running. This happens invisibly, and you pass nothing. It is what tells the elements of a multi-instance activity apart. Take a multi-instance call activity. It starts a called process, which is a secondary workflow of the SAME aggregate. So three elements share workflow module, BPMN process and aggregate ID down to the character. If their correlation id comes from business data and is the same for all three, only the activation differs. And that is enough. The same holds for a workflow which the BPMS started itself. (why)

Two correlations planned by ONE activation still collide, because both are waiting when the second one is scheduled. A loop which correlates twice in one pass of one handler needs a correlation id per round.

An adapter which cannot name its activations reports none. Then everything below applies unchanged.

Correlating from outside a task handler

A REST endpoint, a scheduler or a thread which your handler started itself has no activation to name. So two correlations of the same message name and correlation id for one aggregate look exactly like a repeat of one another. Here the correlation id is the only thing which can differ. So take it from whatever the business data offers: the item, the partner, the round number. One id for the whole loop is what breaks.

Nothing is dropped silently. VanillaBP logs a warning which names what was not planned. It names both possible causes, because it cannot tell them apart. If a correlation which already happened was sent again, nothing is lost. If it was a second correlation which was meant, the message is lost. The warning also says whether it saw an activation, because the fix differs. If you find this warning in a log, the workflow it names may be waiting for a message which will never arrive.

Your BPMS may have a net of its own

Everything above is VanillaBP's deduplication, and it ends when the entry is sent. A BPMS which deduplicates messages itself adds a second net with a window of its own. VanillaBP neither sets nor shortens that window. So the engine can still refuse a repetition which your application planned on purpose. Your adapter's wiki says whether that happens, how long it lasts and what the adapter logs when it does.

Multi-instance siblings depend on the adapter's message id. VanillaBP hands the activation to the adapter when the entry is sent, and an adapter which derives a message id of its own puts the activation into it. If it does not, the three elements of a multi-instance call activity are three operations for VanillaBP but one message for the cluster. Then each of them needs a correlation id of its own.

Starting a workflow by message

startWorkflowByMessage(aggregate, messageName) starts a workflow through its BPMN message start event. The usual start rules apply. The adapter with the highest priority starts it, and at most one workflow is started per aggregate (idempotent like startWorkflow).

The message has to start the process of this ProcessService. Each ProcessService belongs to one BPMN process, and startWorkflowByMessage starts that process and no other. Pass the message name as your model writes it, without any prefix which VanillaBP adds to avoid name clashes. VanillaBP compares it with the message start events of the model which the starting adapter deployed when your application started. That is the newest version, and a message always starts the newest version. If the name is not among them, your call is refused with an IllegalArgumentException, before anything is saved:

Message 'InvoicePaid' does not start BPMN process 'OrderProcess' of workflow module 'order-module'!
ProcessService#startWorkflowByMessage starts the process of its own ProcessService and no other.
The messages which start this process in the model adapter 'c7' deployed are: 'OrderPlaced'. Pass
one of them, or use the ProcessService of the process the message is meant to start. VanillaBP 1
started any process which knew the message.

To start another process, inject the ProcessService of that process' workflow aggregate and call it there. During a BPMS migration the adapter which starts new workflows decides. So if a message was renamed in the new model, it is checked against the new name.

For this check, the adapter has to read its model and say which messages start a process. If the adapter cannot do that, the call goes to the BPMS unchecked, as in version 1. Your application then says so once per process when it starts:

Adapter 'pea' does not say which messages start BPMN process 'OrderProcess' of workflow module 'order-module', so
ProcessService#startWorkflowByMessage is not checked for it. A message which starts another process starts that
one, as in VanillaBP 1.

Your adapter's wiki says whether it can be checked.

BPMS specifics

What a model has to carry so it correlates

The point of the migration feature is that the same model correlates on every BPMS. Correlation is the one place where this costs a modelling step. Without a correlation id you do not have to model anything anywhere. With one, Camunda 7 and Camunda 8 each want something, and they want different things:

BPMS	With a correlation id, the model has to	Without one
Camunda 7	carry a local variable at the scope which waits for the message, holding the expected id	nothing
Camunda 8	say which expression the subscription correlates by	nothing
Process-Engine-API	nothing	nothing

So a model which correlates by aggregate ID moves between the three without a change. A model which uses correlation ids is the one place where the migration promise costs a modelling step. Each adapter's Configuration page says what exactly to write. It also says what the adapter puts into the deployed model so that you do not have to:

1. Camunda 7: Learn more...
2. Camunda 8: Learn more...
3. bpm-crafters Process-Engine-API: Learn more...

What the call checks before your transaction commits

Correlating happens after your commit. So what fails at your call site is whatever the BPMS can be asked while the transaction is still open. This is not the same everywhere, and the reason matters more than the rule (why):

• Camunda 7 fails your transaction when no execution of that workflow waits for the message. With a correlation id, it also fails when no execution expects that id. The engine is in your JVM and answers from your own transaction, so the check costs nothing.
• Camunda 8 checks the model, not the cluster. A message name which no deployed model of the workflow module declares fails at your call. A missing subscription does not, because the cluster buffers a published message until its time-to-live passes. The adapter's wiki says what that check covers and why the cluster is not asked.
• The Process-Engine-API checks nothing. Its correlation command cannot carry the preflight mode (GAPS.md, entry 11). So a correlation which nothing waits for is found only when the entry is sent.

On Camunda 7 an application learns that correlating without a waiting subscription fails at once. Before moving to Camunda 8 it has to unlearn this, because there the message waits for its time-to-live instead.

Neither is a changed aggregate

Correlating a message moves on a workflow which waits for it. Sometimes all you want is that the BPMS SEES the current state of your aggregate. For example, a conditional event waits for it, or a gateway is evaluated before your next task runs. Then push the aggregate instead of inventing a message for it: Telling the BPMS that the aggregate changed.

Signals are not messages

A message is addressed to ONE workflow. VanillaBP finds the BPMS which holds it and correlates there. A signal is a broadcast. Every subscription in the scope of the workflow module reacts, and the signal starts processes which have a signal start event. So sendSignal takes no workflow aggregate (why):

bpms.sendSignal("DriversStrikeStarted");

• Its scope is the workflow module of the ProcessService you called. A signal is a broadcast across PROCESSES, not across modules. A workflow module is a scope of its own, kept apart by a tenant or by prefixed identifiers, and VanillaBP addresses exactly that scope. It uses the client and the tenant configured for the adapter. If the module prefixes its identifiers, it also puts the module's prefix in front of the signal name. If you want a signal in several modules, send it through the ProcessService of each of them. Which modules are meant is a business question, and VanillaBP cannot answer it for you.
• It reaches every BPMS the workflow module is deployed to, not only the one a particular workflow runs in. During a migration, the workflows which wait for the signal are spread across the BPMS. A broadcast which reaches only half of them would be worse than none.
• It needs a transaction, like every operation which reaches a BPMS. The broadcast is planned in your transaction as an outbox entry, and it happens once that transaction committed. This is so on an embedded BPMS as well as on a remote one. A rolled-back transaction broadcasts nothing.
• It is not buffered. A signal reaches whoever waits for it at that very moment. A workflow which arrives at its catch event a second later gets nothing. That is how BPMN defines a broadcast, not a limit of VanillaBP. If you need a delivery which waits for its recipient, correlate a message to that workflow instead.
• It cannot be deduplicated. A signal has no key by which a BPMS could recognize a repetition. So after a crash, the at-least-once sending of the outbox may broadcast a second time. Model your processes so that they tolerate this, or use a message when you need one delivery to one workflow.
• The name you pass is the name you modelled. If the workflow module prefixes its identifiers, VanillaBP adds the prefix to the signal name on the way out. Your code never sees a prefix.

On purpose, there is no way to send a signal to a single workflow. Camunda 7 could do it, Camunda 8 cannot. An API which works on one BPMS and silently does something else on another is worse than none. Use a message for that case. That is what messages are for.

BPMS	What a broadcast reaches
Camunda 7	the engine of that adapter id, after your transaction committed, in the module's scope: its tenant, or the prefixed signal name where identifiers are prefixed
Camunda 8	the cluster of that adapter id, after the commit, in the module's scope, the same way
Process-Engine-API	the engine behind the API, after the commit. An engine implementation which offers no signals says so and does not drop the broadcast silently

Each adapter's own wiki says which command it sends for this.

One consequence of the name-clash-avoidance mode none: it scopes nothing. So neither a tenant nor a prefix keeps the workflow modules apart in the BPMS. A signal then reaches every subscription of that BPMS which uses the same name, whatever module it belongs to. That is the price of that mode, and the reason why the default is not none.