Architecture

The cockpit is one service, the applications running the workflows are others, and the user sees one web application. This page says how that is put together, and it is the page every other one builds on.

1. The parts and how they fit
   The cockpit server, the workflow modules, and the browser which is served by both.
2. Module federation and the proxy
   How a form written in a workflow module ends up inside a cockpit screen.
3. Work which lives in another application
   What the cockpit can do for a system which has no form to hand over.
4. The path a report takes
   From an event in the engine to a row in a list, step by step, and what it costs a read when a details provider writes.
5. Why the store is a document database
   What the lists have to do that a table cannot.
6. What it is built from
   The technologies, and what each of them is there for.

The parts and how they fit

The Business Cockpit is a microservice. It serves the user interface, keeps what has been reported to it, and answers the user interface's questions. It runs nothing else. That also makes it the natural home for what a whole company shares. A holiday request is a workflow module like any other. What goes beyond workflows is built into a cockpit of your own, and that is where such functions have been put: one company added pages for arranging who stands in for somebody who is away, kept that in the user record and loaded the stand-in's roles when the person logs in, and another added a way of looking at the lists the way a colleague sees them. Neither comes with the delivered cockpit. They belong here rather than in every application, because here they are written once.

A workflow module is a bundle inside a business application: the BPMN files, the business code which serves them, and the user interface for the user tasks of those workflows. It has its own release cycle. An application may hold one workflow module or several, and several applications may report to the same cockpit.

Two libraries connect the two sides. The cockpit adapter goes into the workflow module and reports what happens there. The BPMS adapter runs the workflows and is VanillaBP's own; the cockpit adapter sits next to it and only listens.

flowchart TB
    B["Browser<br>one web application"]

    subgraph C["Business Cockpit server"]
        UI["Static files<br>of a user interface, if any"]
        GUI["GUI API<br>lists, search, one task"]
        PROXY["Proxy<br>to the workflow modules"]
        BPMS["BPMS API<br>takes in what is reported"]
        DB[("MongoDB<br>modules, workflows, user tasks")]
    end

    subgraph W["Workflow module, inside a business application"]
        BA["VanillaBP BPMS adapter"]
        FED["Federated user interface<br>forms and status sites"]
        CA["Cockpit adapter"]
        OB[("Outbox store<br>next to the workflow aggregate")]
        BIZ["Business code<br>details providers"]
    end

    E["BPMS<br>Camunda 7, Camunda 8, ..."]

    B --> UI
    UI --> GUI
    GUI --> DB
    BPMS --> DB
    B -. "forms, loaded through" .-> PROXY
    PROXY --> FED
    BA --> E
    E --> CA
    CA -- "one entry per event" --> OB
    OB -- "when it is dispatched" --> BIZ
    BIZ -- "the report" --> BPMS
    %% holds the federated user interface above the outbox store, so that the proxy's
    %% arrow and the report's arrow do not cross
    FED ~~~ OB

Loading

The browser talks to the cockpit and to nothing else. Everything a workflow module serves reaches the browser through the cockpit's proxy, so there is one origin, one session and one place where access is decided. Security is about that.

Module federation and the proxy

The cockpit cannot know the forms of a workflow module. They are written by the team which wrote the workflow, they change with it, and they are deployed with it. So they are not built into the cockpit at all. They are loaded by the browser at the moment they are needed, from the workflow module which serves them, using module federation.

A workflow module registers itself with the cockpit while it starts. The registration says the address the module itself answers on, its workflow-module-uri, and where its user interface is served from. From then on the cockpit offers a path of its own for that module and forwards everything below it, so the browser loads the form from the cockpit's address even though the workflow module produced it. That is what keeps the browser on one origin.

What a workflow module publishes this way are the entry points the delivered cockpit knows: the form of a user task, the status site of a business case, and the column definitions and cell renderers of the two lists. User task forms and status sites says what each of them is handed. A module may publish anything else as well, a button in the header or an entry in the menu for instance. Which regions exist for such a part is decided by the user interface, not by the cockpit, and since nearly every company writes its own, that set is theirs to choose. The delivered application offers none of them today.

Work which lives in another application

Not every system which offers work has a form the cockpit could load. A ticket system or a document archive has its own pages, and nobody is going to rebuild them. What the cockpit can still do is put their work into the same list as everything else.

A workflow module for such a system reports the whole address of a page as its ui-uri-path, starting with http:// or https://. That is not a file the browser loads, and clicking the task opens that page in a window of its own. Clicking the same task again brings that window to the front.

Where the address comes from is up to the module. The configured path is the same for every task, which is enough when the other application has one page to start from. A details provider may set an address per task instead, and the user then lands on the ticket the task is about. A case and its status page work the same way.

The cockpit hands the address out the way it was reported. It does not proxy the other application, so the browser goes there directly and the address has to be one the browser can reach. Nothing travels along with it, neither the session nor the name of the user. Who is looking is the other application's own question, and the user may have to log in there a second time.

What the cockpit records about the task stays the cockpit's. Read, claimed and assigned work as they do for any other task, and the other application hears nothing about them. The lists show the columns the cockpit knows for every task, because a column filled with business data comes from a federated module and there is none here.

The path a report takes

Nothing appears in a list unless a workflow module reported it. A report is not sent while the engine is working. It is written down first and sent afterwards, so that a workflow which is rolled back reports nothing and a cockpit which is down loses nothing.

sequenceDiagram
    participant E as BPMS
    participant A as Cockpit adapter
    participant P as Your details provider
    participant O as Outbox store
    participant D as Outbox dispatcher
    participant S as Cockpit server
    participant B as Browser

    E->>A: a user task was created
    A->>A: assignee, candidates, dates, out of the event
    A->>P: enrich it
    P-->>A: title, columns, candidates
    A->>O: one entry carrying the report, in the workflow's own transaction
    O-->>O: the transaction commits
    D->>S: the report, over REST or Kafka
    S->>S: stored, and the change stream fires
    S->>B: the list updates itself

Loading

The steps in words:

1. The engine does something the cockpit cares about: a user task is created, changed, completed or cancelled, or a workflow starts or ends. The cockpit adapter of that BPMS hears it. How it hears it differs per engine, and each adapter's own wiki describes that.
2. The adapter puts the report together right there. It reads what the event says about the task or the case, calls your @UserTaskDetailsProvider or @WorkflowDetailsProvider method to enrich it, and renders the titles from your templates.
3. The adapter writes one outbox entry, and the finished report travels with it. The entry goes into the same store and the same transaction as the workflow aggregate the event is about. Until that transaction commits, nothing has happened.
4. VanillaBP's outbox sends the report afterwards, over REST or over Kafka, and asks nobody anything on the way. Connecting a workflow module says how that is configured.
5. The cockpit stores it and tells every open browser about it, so a list which is on screen updates itself.

What the report says is what was true at step 2. The assignee, the candidates, the dates, the business key and the version of the deployed process come from the event itself, and your details provider reads the workflow aggregate as the transaction of that event sees it. If a workflow service changes its case and reports the change in the same method, the report holds what it wrote. So the cockpit shows the state at the event, not the state of some later moment. That stays true when the cockpit server is down for an hour and the entries pile up.

The entry itself stays a row of identifiers. VanillaBP keeps the report beside it and hands it back when the entry goes out. So a report does not have to fit into the 2048 characters an outbox entry holds for its arguments. A report of a user task written as JSON is about a kilobyte, and a megabyte is the limit.

An end is built the same way. A task which is over, whether somebody completed it or the process withdrew it, goes through the same steps as a change, and so does a case which reached its end. Your details provider runs, with COMPLETED or CANCELED in hand, and the report carries the same fields. So the list of finished work shows the business data a case had when it ended, not whatever the last change happened to say.

Sometimes the BPMS says nothing about the task or the case. What happens then depends on what was being reported. A report about work which is still running is dropped and the log says so. An end is sent anyway, with its identifiers and without business data, because a completion which never arrives leaves a task the cockpit shows as open forever. The cockpit keeps what it already stored about that task, so the list falls back to the data of the last change rather than losing it.

Two things follow from sending at step 4 rather than at step 2. A send which fails leaves the entry where it is, so it is repeated until it gets through. And a burst of changes to one task still arrives as one report, because a second report about one task takes the place of the one which is still waiting. The one which is left is the youngest of them.

Think twice before a details provider writes the case. What happens depends on where the event came from. Your provider reads the aggregate in the transaction of the event, and that transaction writes back what the provider changed. With an embedded engine, and with every report you make through BusinessCockpitService, the event reaches you inside your own transaction. Then that write is your own, and there is no second writer at all.

Where the event arrives on a worker thread of a remote engine, the adapter opens a transaction for the report, because there is none to join. A provider which writes the case then writes it beside your own code. A change made in between is silently written over, unless the aggregate has a version attribute. With one, the later of the two writers gets a version conflict instead. If the report is the later one, it is repeated. If your own transaction is the later one, it gets the conflict and repeats.

Nothing tells you at startup that the attribute is missing. The cockpit tells VanillaBP that its details providers only read, so VanillaBP saves nothing after them and warns about nothing. A provider which writes the aggregate anyway is written by your own persistence, and the version attribute is then yours to add. Reporting workflows and user tasks says why a provider is not saved for and why a change you make there is written all the same, and the platform's page Workflow aggregates shows the ways out next to the other pairs of writers which meet this way.

The same provider runs when your own code reads a task through BusinessCockpitService.getUserTask, and there the bill goes to the reader. That read works in the transaction the caller is in, so a change the provider makes is left for the caller to commit. So a read of your own can end in a version conflict when a report of a remote engine wrote the same case. When a read fails on a version conflict is about that.

The order is the third thing, and it comes from the outbox rather than from step 3. The outbox promises no order. Entries are dispatched in parallel, and an entry whose send failed comes back after the entries planned later have already gone through. A change can therefore reach the cockpit after the change which followed it, and the end of a task can reach it before the report that the task was created at all.

So the cockpit weighs every report against what it holds, and it weighs by the timestamp of the event and not by the moment the report arrived. Over Kafka as over REST. A report which is older than what is stored changes nothing, and nothing arriving after an end undoes that end. An end of a task or a case the cockpit does not know yet creates it, finished; the creation arriving later then fills in what the end could not say, instead of opening a task which never closes.

When a read fails on a version conflict

You meet this as a failure at the end of a method which only read something. On Spring and JPA the message looks like this:

ObjectOptimisticLockingFailureException: Unexpected row count (expected row count 1 but was 0)
  [update ride set customer=?,note=?,version=? where id=? and version=?]

The update in the brackets is what gives it away. Nothing in your method asked for it. It writes the workflow aggregate your details provider changed while getUserTask was putting its answer together, because JPA writes a managed object which changed when the transaction commits. A report of an event of a remote engine wrote the same row in the meantime and counted its version up, so your update finds no row left to write and your read is the one which pays.

The first way out is the one to take: let the provider read. It is asked what the cockpit should show, and a provider which fills that in and changes nothing never brings a reader into this. Reporting workflows and user tasks says what a provider may ask for and what it should leave alone. A provider cannot keep the writing for the reporting path either, because it is not told which of the two it is in: a read reaches it as an UPDATED call, the same way a change does.

The other way out is to treat the read like a write. A method which calls getUserTask while its provider writes has to expect the conflict, so it belongs somewhere it can be repeated, and the repeat reads the case again rather than reusing what it read before. Every caller of getUserTask carries that then, which is why the first way is the better one.

None of this is behaviour of VanillaBP. VanillaBP saves nothing after a details provider. The write comes from your persistence, which treats this aggregate like every other managed object it holds, and the version attribute which turns a silent loss into a conflict is the one your application put on the class. It is ordinary database work between two writers of one row. It is written down here because this is where it tends to catch people.

Why the store is a document database

The cockpit does not know what a list column holds. A workflow module reports business data of its own, an amount here and a customer there, and the user wants to search, sort and filter by exactly that. A schema would have to be known before the first workflow module ever registers, and it never is.

That is what a document database is for. The cockpit keeps what was reported as it was reported, and indexes it as it arrives.

It uses MongoDB from version 4.4 on. MongoDB has to run as a replica set, because the cockpit reads its change stream and MongoDB offers that on a replica set only. The change stream is where the live updates of the lists come from. Compatible databases work as well, and Azure Cosmos DB for MongoDB is in use. If you need another document database, please open an issue.

What it is built from

The server is a Spring Boot application on Java 21, written in blocking style on Spring MVC with virtual threads. It used to be reactive, and virtual threads made that trade pointless: a thread waiting for MongoDB or for a workflow module behind the proxy costs almost nothing now, so the code can be plain Java without giving up concurrency. The two parts which looked like arguments for staying reactive are not: the live updates are SseEmitter instances a scheduled collector writes to, and the proxy copies each exchange chunk by chunk rather than buffering it. Both cost one virtual thread while they are open. A second scheduled task decides which of those streams a change concerns. It reads the changed documents once and decides for every stream in memory, so the load on MongoDB does not grow with the number of open browser tabs.

The cockpit delivers no user interface since 0.9.0. The one it delivered before is a React application in the Git history of the cockpit repository. It uses the components of the NPM package @vanillabp/bc-ui, and a workflow module's own parts use @vanillabp/bc-shared, which both sides share. It is also the part which fits an organisation least well. Every company brings its own framework, its own version of it, its own component library and its own design, so the delivered application gets copied and rebuilt, and the copy drifts away from what the cockpit expects of it. Today the answer is to treat the React application as an example and to write your own against the same APIs; Customizing the user interface says how far that gets you and where it hurts.

The reports arrive over REST or over Kafka. The REST side is described by an OpenAPI specification, which the clients and the servers are generated from; the Kafka side carries the same content as protobuf messages. Either way the interface is written down, which is why reporting to the cockpit does not depend on the library the cockpit happens to ship: a system which is not a workflow module at all can report over the same API.

Inside a workflow module the choice is those two ways or one of your own. A bean of type BusinessCockpitTransport takes the place of both, which is what version 1 allowed through its publishing beans. The call sits at the end of the outbox dispatch, so a transport written by an application is handed a finished report and rides the same transaction and the same repetition as a shipped one. Connecting a workflow module shows the bean.

What listens on that API does not have to be the VanillaBP Business Cockpit either. A team which wants a task list of its own can take the reports and build one, and the gain is the one the cockpit adapter brings everywhere else: the task list is written against this API instead of against Camunda or whichever engine runs underneath, and a change of engine leaves it alone.