Domain-typed REST API for Camunda POC development workflow

This repository contains the source, tools, and development workflow for developing the proof-of-concept for a domain typed REST API for Camunda 8.

This workflow is designed to ensure that the domain typed REST API is kept up-to-date with the latest version of the REST API, while also ensuring that the generated files are valid and do not break the build or change the behaviour of the API.

It uses camunda/camunda, camunda/camunda-8-js-sdk, and camunda/camunda-docs.

Structure

• rest-api.yaml - This file contains the latest version of the REST API specification. It is used as the source for generating the domain file and the generated file.
• rest-api.domain.yaml - This file contains the domain typed REST API specification. It is hand made from the rest-api.yaml file and contains additional domain information.
• rest-api.generated.yaml - This file is generated from the rest-api.domain.yaml file. It contains a generated REST API specification compatible with the Camunda codebase.
• java/openapi-camunda-key-flattener/src/ - This directory contains the source code for the transformation from the domain file to the generated file. This tool rewrites all schemas that extend CamundaKey as the primitive string. It is a pre-processing step that produces a version of the API spec that is compatible with the existing build chain in camunda/camunda.
• java/rest-api-consistency-analyzer - This tool analyses the controllers in the Zeebe gateway and reasons about their eventual consistency. It then checks the spec to assert that all eventually consistent endpoints have the x-eventually-consistent vendor extension on them. It also picks up other issues as a side-effect (like method mismatch between the spec and a controller).
• vacuum - Vacuum linting for the generated spec
• functions - Spectral functions for linting the domain spec
• docs-patch - a patch for camunda-docs/api/camunda/generate-strategy.js to support the docs pre-processing of the new features in the spec.
• camunda - checkout camunda/camunda here for scripts to fetch the upstream rest-api.yaml
• camunda-docs - checkout camunda/camunda-docs here for script to interact with documentation generation

Setup

To set up the development environment for the domain typed REST API for Camunda, follow these steps:

1. Clone the camunda/camunda, camunda/camunda-docs, and camunda/camunda-8-js-sdk repositories.
2. Clone this repository.
3. Ensure you have Node.js and npm installed.
4. Install the necessary Node.js dependencies by running npm install in the root directory of this repository.
5. Install the Spectral CLI globally by running npm install -g @stoplight/spectral-cli.
6. Install the Vacuum CLI globally by running npm install -g @stoplight/vacuum-cli.
7. Install the Spectral VSCode extension for live linting of the domain file.
8. Install the Vacuum VSCode extension for live linting of the generated file.
9. Ensure you have Maven installed and configured on your system.
10. Ensure you have Docker installed and running, as it will be used to run Elasticsearch for testing purposes.
11. Ensure you have a Java Development Kit (JDK) installed, as it is required to build and run Camunda.

Workflow

These are the steps to follow when working on the domain typed REST API for Camunda. The process involves generating a domain file from the latest version of the REST API, transforming it, and ensuring that the generated files are valid and do not break the build or change the behaviour of the API.

Establish a green baseline

• Pull the latest changes from main and build Camunda:

npm run spec:update
npm run spec:build
npm run spec:verify

• Start Camunda following the instructions in the Running Camunda section.
• Run the Node.js SDK tests against the started Camunda instance to ensure everything works as expected.

This gives us a green baseline to work from. You can stop Camunda now.

Update the domain typed REST API

When updating the domain typed REST API, follow these steps to ensure that the changes are valid and do not break the build or change the behaviour of the API.

Read the scripts in package.json if you want to know what they do.

1. You got the latest version of zeebe/gateway-protocol/src/main/proto/rest-api.yaml by running npm run spec:update.
2. Use the git status diff to see what has changed.
3. Copy the changes to rest-api.domain.yaml, adding domain information.
4. Run npm run lint:domain to ensure the domain file is valid. (You can use the Spectral plugin in VSCode for live linting.)
5. Run npm run spec:generate - this will generate the rest-api.generated.yaml file.
6. Run npm run lint:generated to lint the generated files= with Vacuum.

This ensures that the generated files are valid and follow the expected structure.

Verify the generated file

1. Run npm run spec:install to copy the generated file to the zeebe/gateway-protocol/src/main/proto/rest-api.yaml directory of camunda/camunda.
2. Run npm run spec:build to build with the generated file.
3. Run npm run spec:verify to verify the build.

This ensures that the generated file does not contain any changes that would break the build of Camunda 8.

Verify the API behaviour

1. Run Camunda, following the instructions in the Running Camunda section.
2. Run the Node.js SDK tests against the started Camunda instance to ensure the API behaves as expected.

Inspect the impact of the changes on documentation

1. Run npm run docs:generate to regenerate API docs in camunda/camunda-docs.
2. Run npm run docs:start to start the live docs server.

You will now be able to inspect the impact of the changes on the documentation by navigating to http://localhost:3000/.

Update: the docs are now hosted live here.

Everything looks good?

Congratulations! You have successfully updated the domain typed REST API for Camunda.

• Commit the changes to the rest-api.domain.yaml and rest-api.yaml files.

Running Camunda

• Run npm run camunda:es:stop to stop any running Elasticsearch Docker container.

• Run npm run camunda:es:start to start an ES container. (Alternative: open the file operate/docker-compose.yml in IntelliJ and click the green arrow next to the elasticsearch service to start it).

• Run npm run camunda:es:shhh to suppress deprecation warnings in the logs.

• Run npm run camunda:clean to delete the data directory from any previous run of Camunda.

• Open the file src/main/java/io/camunda/application/StandaloneCamunda.java from camunda/camunda in IntelliJ.

• Ensure that the class run configuration has the following environment variables set: CAMUNDA_SECURITY_AUTHENTICATION_UNPROTECTEDAPI=true;CAMUNDA_SECURITY_AUTHORIZATIONS_ENABLED=false;ZEEBE_BROKER_EXPORTERS_CAMUNDAEXPORTER_CLASSNAME=io.camunda.exporter.CamundaExporter

• Click the green arrow next to the class declaration to run Camunda.

Running the Node.js SDK tests

These test function as behavioural tests for the REST API. They ensure that the API behaves as expected and that the generated files are correct.

In the SDK, source the local configuration:

source env/c8run

Run the 8.8 integration tests:

npm run test:local-integration:8.8

SDK Generation

This project contains proof-of-concept work for SDK generation from the OpenAPI spec, in the sdks directory.

SDK generation-related tasks:

npm run sdks:install # Install SDK generation tooling
npm run sdks:generate # Generate all SDKs
npm run sdks:clean # Clean SDK generated code output
npm run sdks:list" # List all generated SDKs
npm run build:sdks # Build all SDks
npm run watch:spec # Watch domain spec and rebuild all SDKs on change
npm run test:sdks # Build and test all SDKs
npm run test:typescript # Run the acceptance tests for the TypeScript SDK

See the README.md file in sdks for information about generating SDKs.

Eventual Consistency

This project adds metadata to the OpenAPI spec about the eventually-consistent nature of endpoints, based on static analysis of the controller code.

See the EVENTUAL.md file for more information on eventual consistency.

Test Suite Deployment Artifacts

The Playwright-based API tests (under api-test/path-analyser) deploy sample artifacts (BPMN, DMN, Forms):

• Editable registry: api-test/path-analyser/fixtures/deployment-artifacts.json
  • Describes artifact kind, path (relative to fixtures/), and a human-readable description.
• Generated manifest: api-test/path-analyser/dist/output/deployment-artifacts.manifest.json
  • Machine-readable list of artifacts actually referenced by generated scenarios/tests, suitable for programmatic build/packaging.

See api-test/path-analyser/README.md for details.

Response Shape Recorder

The generated Playwright tests can record real response shapes and aggregate them into a compact summary to guide schema defaults and error handling.

Purpose

• Capture sanitized runtime responses from test runs (status, top-level keys, example bodies).
• Help spot optional vs required fields and document error behavior.

How it works

• The path-analyser emitter calls a recorder on every request.
• Observations are appended to a JSONL file: api-test/path-analyser/dist/runtime-observations/responses.jsonl.
• An aggregator script produces: api-test/path-analyser/dist/runtime-observations/summary.json.

Usage

• From the path-analyser package:

  • Generate tests and run them, then aggregate:
    • cd api-test/path-analyser && npm run observe:run
  • Or aggregate existing logs only:
    • cd api-test/path-analyser && npm run observe:aggregate

• From the repo root (shortcuts):

  • npm run testsuite:observe:run
  • npm run testsuite:observe:aggregate

Details and troubleshooting in api-test/path-analyser/README.md.