Camunda Repository Connector

Camunda Repository Connector

Module: oc-camunda-repository-connector
Connector Class: org.opencrawling.camunda.CamundaRepositoryConnector
Security Mapper: org.opencrawling.camunda.CamundaSecurityMapper
Spring Bean: @Component (activated when spring.opencrawling.connector.type=camunda or CONNECTOR_TYPE=camunda)

---

📌 Overview

The Camunda Repository Connector enables OpenCrawling to crawl, extract, and index workflow instances, historic process executions, BPMN variables, and runtime/historic identity link authorizations from Camunda Platform 7 engines via the Engine REST API (/engine-rest).

By uniting Camunda 7 with the Open Ingestion Standard (OIS), Java 25 Virtual Threads (StructuredTaskScope), and zero-trust security mapping, this connector allows AI agents, LLM assistants, and enterprise RAG systems to query business workflows, identify operational bottlenecks, and perform semantic similarity searches that strictly adhere to Camunda's native authorization model.

flowchart TD
    subgraph Camunda["Camunda Platform 7 Engine"]
        REST["Camunda REST API (/engine-rest)"]
        HPI["Historic Process Instances (/history/process-instance)"]
        HVI["Historic Variables (/history/variable-instance)"]
        HIL["Historic Identity Links (/history/identity-link-log)"]
        
        REST --> HPI
        REST --> HVI
        REST --> HIL
    end

    subgraph OpenCrawling["OpenCrawling Ingestion Runtime (Java 25)"]
        Connector["CamundaRepositoryConnector (oc-camunda-repository-connector)"]
        SecMapper["CamundaSecurityMapper (Identity Link Resolution)"]
        VThreads["Virtual Threads (StructuredTaskScope)"]
        
        REST <-->|HTTP/REST Basic Auth| Connector
        Connector --> VThreads
        VThreads --> SecMapper
        
        OIS["OIS Document Builder (RepositoryDocument with SecurityConfig & Entity Links)"]
        SecMapper --> OIS
        VThreads --> OIS
    end

    subgraph Destinations["Dual-Mode Zero-Trust Storage & Retrieval"]
        direction TB
        subgraph Collocated["1. Collocated DB Pattern (pgvector on Camunda PostgreSQL)"]
            PGVector[("PostgreSQL / pgvector (vector_store table)")]
            CamundaDB[("Camunda Tables: ACT_RU_IDENTITYLINK / ACT_HI_IDENTITYLINK")]
            SQLPushdown["Query-Time SQL EXISTS Pushdown (Zero Sync Lag, No Chunk Bloat)"]
            PGVector <-->|Direct SQL JOIN / Subquery| CamundaDB
            SQLPushdown --> PGVector
        end
        
        subgraph Distributed["2. Distributed DB Pattern (External Vector DBs)"]
            ExtVector[("Qdrant / Milvus / OpenSearch / Vespa")]
            SecFilter["Metadata Pre-Filtering (security_allowed_read / security_denied_read)"]
            SecFilter --> ExtVector
        end
    end

    OIS -->|Collocated Architecture| PGVector
    OIS -->|Decoupled Architecture| ExtVector

    subgraph AI["Enterprise AI & RAG Orchestrators"]
        Agents["AI Agents / Spring AI RAG / MCP Assistants"]
        Agents --> SQLPushdown
        Agents --> SecFilter
    end

Loading

---

🚀 Key Features

• Historic & Active Process Instance Crawling:
  • Paginates through historical and running workflow instances (GET /history/process-instance).
  • Ingests core execution metadata: processDefinitionId, processDefinitionKey, businessKey, startUserId, startTime, endTime, durationInMillis, and tenantId.
  • Configurable scan scope: all, completed, or active.
• Dynamic BPMN Variables Extraction:
  • Fetches and parses process variables (/history/variable-instance), indexing them as metadata attributes prefixed with camunda_var_<varName> and within the document JSON content body.
• Identity Links & Zero-Trust Security Mapping:
  • Queries /history/identity-link-log?processInstanceId={id} for assignees, candidate users, candidate groups, and owners.
  • CamundaSecurityMapper translates candidate groups and user links into OIS SecurityConfig and PermissionRule records, deduplicating identities and distinguishing user vs. group principals.
• Collocated Database Pattern (Entity-Level Linking):
  • Eliminates distributed transaction synchronization by stamping entity reference metadata (camunda_proc_inst_id, camunda_scope_id, camunda_scope_type = "processInstance").
  • Enables query-time authorization via SQL EXISTS subqueries against native ACT_RU_IDENTITYLINK and ACT_HI_IDENTITYLINK tables without duplicating identity links per vector chunk.
• Distributed Database Pattern (OIS Metadata Pre-Filtering):
  • Emits security_allowed_read, security_denied_read, and security_inheritance for external vector databases (Qdrant, Milvus, OpenSearch, Vespa).
• Java 25 Structured Concurrency:
  • Uses Java 25 Virtual Threads (Thread.ofVirtual()) and StructuredTaskScope.open() to process instance batches, variable queries, and identity links concurrently with high throughput.

---

⚙️ Configuration Properties

Property Key	Environment Variable	Default	Description
spring.opencrawling.connector.camunda.url	CAMUNDA_URL	http://localhost:8080/engine-rest	Base URL of the Camunda Engine REST API
spring.opencrawling.connector.camunda.username	CAMUNDA_USERNAME	demo	Camunda Basic Auth username
spring.opencrawling.connector.camunda.password	CAMUNDA_PASSWORD	demo	Camunda Basic Auth password
spring.opencrawling.connector.camunda.batch-size	CAMUNDA_BATCH_SIZE	100	Number of process instances per page
spring.opencrawling.connector.camunda.process-definition-key	CAMUNDA_PROCESS_DEFINITION_KEY	""	Optional process definition key filter (e.g. order-process)
spring.opencrawling.connector.camunda.include-variables	CAMUNDA_INCLUDE_VARIABLES	true	Extract historic BPMN process variables
spring.opencrawling.connector.camunda.scope	CAMUNDA_SCOPE	all	Filter by state: all, completed, or active
spring.opencrawling.connector.camunda.include-acls	CAMUNDA_INCLUDE_ACLS	true	Extract and map identity links and start user into OIS SecurityConfig and ACLs

---

📋 Document Metadata Schema

Every process instance emitted by the connector produces an OIS RepositoryDocument populated with both standard BPMN attributes and collocated entity references:

Metadata Attribute	Description	Example
processDefinitionId	Camunda Process Definition ID	order-process:1:1001
processDefinitionKey	Process Definition Key	order-process
businessKey	Domain Business Key	ORD-2026-9812
startUserId	User ID who initiated the process	john.doe
startTime / endTime	ISO-8601 execution timestamps	2026-07-24T10:00:00.000+0000
durationInMillis	Total execution duration in milliseconds	300000
tenantId	Multi-tenant identifier (if applicable)	tenant-alpha
camunda_proc_inst_id	Native Process Instance ID for collocated DB joins	proc-10042
camunda_scope_id	Engine scope identifier for entity-level authorization	proc-10042
camunda_scope_type	Scope type identifier	processInstance
camunda_identity_users	List of allowed user identities	["john.doe", "approver_1"]
camunda_identity_groups	List of allowed group identities	["finance-managers"]
camunda_var_<varName>	Dynamically extracted BPMN variable	camunda_var_amount: ["1500"]

---

🔒 Collocated Database Pattern & Query Pushdown

See Zero-Trust Security & ACL Management for complete architecture details on collocated query pushdowns and distributed vector pre-filtering across Camunda, Flowable, Alfresco, and CMIS.

When OpenCrawling stores vector embeddings in a PostgreSQL database with pgvector alongside Camunda, you can execute zero-trust semantic queries directly joining with native Camunda identity link tables:

SELECT v.id, v.content, (1 - (v.embedding <=> :query_vector)) AS similarity
FROM vector_store v
WHERE (
    -- Public document access
    v.metadata->>'security_inheritance' = 'public'
    OR
    -- Direct starter or identity user match
    v.metadata->'camunda_identity_users' ? :current_user
    OR
    -- Group membership match
    v.metadata->'camunda_identity_groups' ?| :current_user_groups
    OR
    -- Collocated DB query-time subquery against native Camunda runtime identity links
    EXISTS (
        SELECT 1
        FROM act_ru_identitylink il
        WHERE il.proc_inst_id_ = (v.metadata->>'camunda_proc_inst_id')
          AND (il.user_id_ = :current_user OR il.group_id_ = ANY(:current_user_groups))
    )
    OR
    -- Collocated DB query-time subquery against native Camunda historic identity links
    EXISTS (
        SELECT 1
        FROM act_hi_identitylink hil
        WHERE hil.proc_inst_id_ = (v.metadata->>'camunda_proc_inst_id')
          AND (hil.user_id_ = :current_user OR hil.group_id_ = ANY(:current_user_groups))
    )
)
ORDER BY v.embedding <=> :query_vector
LIMIT 10;