Repository navigation
Confluence Repository Connector
Module: oc-confluence-repository-connector
Connector Class: org.opencrawling.confluence.ConfluenceRepositoryConnector
Client Class: org.opencrawling.confluence.ConfluenceClient
Storage Converter: org.opencrawling.confluence.ConfluenceStorageConverter
Document Builder: org.opencrawling.confluence.ConfluenceDocumentBuilder
Security Mapper: org.opencrawling.confluence.ConfluenceSecurityMapper
Properties Class: org.opencrawling.confluence.ConfluenceProperties
Crawl State: org.opencrawling.confluence.ConfluenceCrawlState & org.opencrawling.runtime.connector.ControlPlaneConfluenceStateStore
Discovery & Single-Doc SPI: Implements DocumentFetchingRepositoryConnector (DiscoverableRepositoryConnector: explore, dryRun, crawlStart, fetchDocument)
Spring Bean: @Component (activated when spring.opencrawling.connector.type=confluence or SPRING_OPENCRAWLING_REPOSITORY_CONNECTOR_TYPE=confluence)
Introduced in: Issue #341, PR #440
The Atlassian Confluence Repository Connector (oc-confluence-repository-connector) crawls Confluence Cloud (REST API v2 + v1 restrictions/CQL) and Confluence Data Center / Server (REST API v1) using native Java 25 java.net.http.HttpClient, Jackson 2, and jsoup—with zero external Atlassian SDK dependencies.
It transforms Confluence Storage Format XHTML (body.storage) into clean, semantic HTML5 preserving headings (h1–h6), lists, tables, blockquotes, code blocks, and content-bearing macros (expand, panel, info, note, warning, tip, status, excerpt, code, noformat) while dropping boilerplate navigation macros (toc, children, pagetree, jira, recently-updated, contributors, contentbylabel, livesearch). Pages, blog posts, and coupled binary attachments are emitted as Open Ingestion Standard (OIS) documents with clickable url citations, ancestor breadcrumbs, filterable tags (Confluence labels), multi-replica ControlPlaneStore CQL watermarks, periodic full reconciliation with an empty-listing guard, explicit TOO_LARGE ledger reporting, pre-crawl discovery, and fail-closed Zero-Trust Space + inherited page read restriction ACLs.
Important
Cross-Origin Redirect Security & Fail-Closed ACLs: When downloading binary attachments, ConfluenceClient follows redirects manually and automatically strips the Authorization header whenever a redirect targets a different origin (such as Confluence Cloud's signed CDN download URLs). Anonymous Space permissions are ignored by default (allow-anonymous-public=false), and unreadable Space permissions or page restrictions always fail closed (SecurityConfig.none()).
graph TD
subgraph Atlassian["Atlassian Confluence (Cloud & Data Center)"]
Auth["Auth (Cloud API Token / OAuth 2.0 3LO / DC Bearer PAT / Basic)"]
CloudApi["Confluence Cloud REST API v2 + v1 (/api/v2/spaces, /pages, /labels, /attachments + /rest/api/content/search & restrictions)"]
DcApi["Confluence Data Center REST API v1 (/rest/api/space, /content, /restriction, /content/search)"]
CDN["Signed CDN Attachment Download (Auth Stripped Cross-Origin)"]
Auth --> CloudApi
Auth --> DcApi
CloudApi -. 302 Redirect .-> CDN
end
subgraph OpenCrawling["OpenCrawling Ingestion Runtime (Java 25)"]
Client["ConfluenceClient (Java 25 HttpClient, Retry-After, Group UUID Cache, Server Date)"]
Connector["ConfluenceRepositoryConnector (CQL Delta, Tree, Discovery SPI, Cascade Tombstones)"]
StateStore["ControlPlaneConfluenceStateStore (Shared ControlPlaneStore / oc_setting)"]
VThreads["Virtual Threads (Bounded Parallelism)"]
Converter["ConfluenceStorageConverter (jsoup XML -> Semantic HTML5 + Macro Policy)"]
SecMap["ConfluenceSecurityMapper (Space View ∩ Inherited Read Restrictions, Fail-Closed)"]
DocBld["ConfluenceDocumentBuilder (Pages, Blog Posts, Coupled Attachments, TOO_LARGE)"]
TempStream["TempFileInputStream (Constant-Heap Spool & Auto-Delete)"]
Orchestrator["JobOrchestrator (TOO_LARGE -> SKIPPED Ledger + Trace WARN)"]
ClaimCheck["Claim-Check Store (Local / Apache Ozone / S3)"]
Tika["Apache Tika 4 Text Extraction"]
CloudApi <-->|REST JSON| Client
DcApi <-->|REST JSON| Client
CDN -->|Binary Stream| Client
Client --> Connector
Connector <--> StateStore
Connector --> VThreads
VThreads --> Converter
VThreads --> SecMap
VThreads --> TempStream
Converter --> DocBld
SecMap --> DocBld
DocBld --> Orchestrator
TempStream --> ClaimCheck
ClaimCheck --> Tika
OIS["OIS Document (UPSERT / Cascading DELETE Tombstone)"]
Orchestrator --> OIS
Tika --> OIS
OIS --> Kafka["Kafka Pipeline (opencrawling-ingestion -> chunks -> embedded)"]
Kafka --> Sinks[("Vector & Lakehouse Stores (PGVector, Vespa, Qdrant, Milvus, Solr, OpenSearch, Elasticsearch, Iceberg, ...)")]
end
subgraph AI["AI Applications & Zero-Trust MCP"]
Sinks --> SecureMcp["OpenCrawling Secure MCP Server (McpAclEvaluator & DiscoveryMcpTools)"]
SecureMcp -->|Zero-Trust Space & Page Read Restriction Enforcement| Agent["Enterprise RAG Assistant / Agent"]
end
-
Confluence Cloud:
-
API Token (recommended): Create an Atlassian API token for a dedicated read-only service account (
username= Atlassian email,api-token= API token,auth-type: api_token,deployment-type: cloud). Grant the service account View permissions to the target Space(s). -
OAuth 2.0 3LO (
auth-type: oauth2): Register an Atlassian OAuth 2.0 app with read scopes (read:confluence-content.all,read:confluence-space.summary,read:confluence-props,read:confluence-groups,read:confluence-user,offline_access), configurecloud-id,client-id,client-secret,refresh-token, and setweb-base-urlto your tenant site (e.g.https://acme.atlassian.net/wiki) so citations stay clickable while API calls route throughhttps://api.atlassian.com/ex/confluence/{cloudId}/wiki.
-
API Token (recommended): Create an Atlassian API token for a dedicated read-only service account (
-
Confluence Data Center / Server:
-
Personal Access Token (recommended): Issue a PAT for a read-only crawl user (
auth-type: pat,api-token= PAT,deployment-type: data_center,base-url= full base URL including context path such ashttps://confluence.internal.corp/confluence). -
Basic Auth: Set
auth-type: basicwithusernameandapi-token(orpassword).
-
Personal Access Token (recommended): Issue a PAT for a read-only crawl user (
-
Connection Verification: Click Test Connection in the Admin UI or run
oc connector check --name <name> --type repository. The checker probes/api/v2/spaces?limit=1(Cloud) or/rest/api/space?limit=1(Data Center), verifies every configuredspace-keysentry, and checks reachability ofroot-page-idwhen set.
-
Unified Cloud & Data Center Support:
- Auto-detects
cloudvsdata_centerwhendeployment-type: auto(based oncloud-id,.atlassian.net, orapi.atlassian.com), or binds explicitly viacloud/data_center.
- Auto-detects
-
Space, Subtree & Label Scoping:
- Filter spaces by
space-keys(personal spaces~userexcluded by default unlessinclude-personal-spaces: true). - Restrict crawling to a page subtree via
root-page-id, toggle Space blog posts viainclude-blog-posts, and filter byincluded-labels/excluded-labels(excludes always take precedence, e.g.excluded-labels: draft,archived). - Emits Confluence labels as multi-value
tagsandconfluence.labelsmetadata on both pages and their coupled attachments.
- Filter spaces by
-
Structure-Aware Storage Format Conversion (
#385):- Parses
body.storageXHTML viajsoup's XML parser (Parser.xmlParser()) into clean semantic HTML5 preservingh1–h6, lists, tables (colspan/rowspan), blockquotes, code blocks,ac:task-list([x]/[ ]), andac:linktitles. - Preserves content-bearing macros (
expand,panel,info,note,warning,tip,status,excerpt,section,column,layout,code,noformat) and strips navigation boilerplate (toc,toc-zone,children,pagetree,pagetreesearch,jira,jiraissues,recently-updated,contributors,contentbylabel,livesearch,spaces,attachments,anchor,index), both customizable viakeep-macrosanddrop-macros.
- Parses
-
Coupled Binary Attachments, Cascading Tombstones &
TOO_LARGEReporting (#397):- Indexes MIME-filtered binary attachments (
included-attachment-mime-types,excluded-attachment-mime-typesexcludingimage/*,video/*,audio/*, and archives by default) as separate OIS documents inheriting the owning page's labels, breadcrumbs, and effective ACLs. - Streams downloads via auto-deleting
TempFileInputStreamwhile strippingAuthorizationacross cross-origin CDN redirects. - Oversized attachments exceeding
max-attachment-size-bytes(default52428800= 50 MB) emitopencrawling.document.status=TOO_LARGE, soJobOrchestratorrecordsSKIPPED(errorCode="TOO_LARGE") in the execution ledger and aWARNevent in traces. - Trashing, deleting, moving out of scope, or label-excluding a page automatically cascades
DELETEtombstones to all its tracked attachments.
- Indexes MIME-filtered binary attachments (
-
Incremental CQL Delta, Multi-Replica State (
#395) & Periodic Full Reconciliation (#396):-
cql_deltamode (default): Persists server-Date-anchoredlastModifiedUtc,lastFullScanAt,renderSignature, and per-item(version, labelFingerprint, aclFingerprint, parentId, attachmentIds)in OpenCrawling's sharedControlPlaneStore(ControlPlaneConfluenceStateStore, keyconnector-state/confluence-<sha256>.jsoninoc_setting), allowing scheduled jobs to run on any runtime replica. - Queries modified pages/blogposts, modified attachments (reconciling owning pages even when only an attachment changed), and
status = trashedcontent using a configurablecql-overlap-minuteslookback window. - Because Confluence does not bump
version.numberon label-only or restriction-only edits,full-scan-interval(defaultP7Din Admin UI, optionally constrained byfull-scan-windowandfull-scan-timezone) runs periodic full reconciliation guarded by an empty-listing guard that prevents mass tombstoning on transient empty upstream listings.
-
-
Pre-Crawl Discovery & On-Demand Document Fetching:
- Implements
DiscoverableRepositoryConnector(explore,dryRun,crawlStart) andDocumentFetchingRepositoryConnector(fetchDocument) for pages, blog posts, and attachments with live page-by-pageVisibilityFilterenforcement.
- Implements
Configure via Spring Boot properties (spring.opencrawling.connector.type=confluence and spring.opencrawling.connector.confluence.*) or via the Admin UI / Platform API / connectors.json (org.opencrawling.confluence.ConfluenceRepositoryConnector, camelCase keys).
Spring Property (spring.opencrawling.connector.confluence.*) |
connectors.json Key |
Default | Description |
|---|---|---|---|
base-url |
baseUrl |
https://example.atlassian.net/wiki |
Confluence base URL (Cloud /wiki or Data Center URL including context path) |
web-base-url |
webBaseUrl |
(same as base-url) |
User-facing browser URL for clickable citations (recommended with OAuth 2.0 cloud-id) |
deployment-type |
deploymentType |
cloud |
cloud, data_center, or auto
|
auth-type |
authType |
api_token |
api_token (Cloud email + token), pat (DC Bearer PAT), oauth2 (Cloud 3LO), or basic
|
username |
username |
— | Atlassian email (api_token) or username (basic) |
api-token |
apiToken |
— | Cloud API token, Data Center PAT, or Basic password (masked in API/UI) |
oauth2-token-url |
oauth2TokenUrl |
https://auth.atlassian.com/oauth/token |
OAuth 2.0 token endpoint URL |
cloud-id, client-id, client-secret, refresh-token
|
cloudId, clientId, clientSecret, refreshToken
|
— | Cloud OAuth 2.0 3LO credentials (auth-type: oauth2) |
space-keys |
spaceKeys |
(all global spaces) | Comma-separated Space keys (e.g., ENG,KB,SEC) |
include-personal-spaces |
includePersonalSpaces |
false |
Include personal spaces (~user) when space-keys is empty |
root-page-id |
rootPageId |
— | Optional page ID restricting crawl to that page and its descendants |
include-blog-posts |
includeBlogPosts |
true |
Index Space blog posts |
include-attachments |
includeAttachments |
true |
Index coupled binary attachments |
include-labels |
includeLabels |
true |
Emit Confluence page labels as tags and confluence.labels metadata |
include-custom-properties |
includeCustomProperties |
true |
Emit custom content properties as confluence_prop_<key>
|
included-labels |
includedLabels |
— | Optional label whitelist (at least one must match) |
excluded-labels |
excludedLabels |
(empty in core; draft,archived in UI default) |
Label blacklist (always takes precedence over included-labels) |
body-format |
bodyFormat |
storage |
storage (recommended XHTML→HTML5), export_view, or view
|
keep-macros |
keepMacros |
expand,panel,info,note,warning,tip,status,excerpt,section,column,layout,code,noformat |
Body-bearing macros converted to clean semantic HTML |
drop-macros |
dropMacros |
toc,toc-zone,children,pagetree,pagetreesearch,jira,jiraissues,recently-updated,contributors,contentbylabel,livesearch,spaces,attachments,anchor,index |
Boilerplate/navigation macros stripped completely |
included-attachment-mime-types |
includedAttachmentMimeTypes |
— | Optional MIME allow-list (supports wildcards like application/pdf) |
excluded-attachment-mime-types |
excludedAttachmentMimeTypes |
image/*,video/*,audio/*,application/zip,application/x-zip-compressed,application/gzip,application/octet-stream |
MIME types to skip ("" or "none" clears defaults) |
max-attachment-size-bytes |
maxAttachmentSizeBytes |
52428800 |
Maximum attachment size in bytes (50 MB); larger files report TOO_LARGE
|
crawl-mode |
crawlMode |
cql_delta |
cql_delta (incremental CQL) or tree (full space enumeration) |
cql-overlap-minutes |
cqlOverlapMinutes |
5 |
CQL watermark overlap lookback window in minutes |
state-file |
stateFile |
(blank — shared ControlPlaneStore) |
Optional local JSON state file override |
full-scan-interval |
fullScanInterval |
(empty; P7D in UI default) |
ISO-8601 duration for periodic full reconciliation (catches label/ACL drift) |
full-scan-window / full-scan-timezone
|
fullScanWindow / fullScanTimezone
|
(anytime) / UTC
|
Optional 24h maintenance window (e.g. 02:00-05:00) & IANA timezone |
include-acls |
includeAcls |
true |
Enforce Space View + inherited ancestor/page read restrictions |
user-identity |
userIdentity |
email_and_id |
email_and_id, email, or id (GDPR-safe Cloud accountId) |
expand-group-members |
expandGroupMembers |
false |
Expand Confluence group members via /rest/api/group/...
|
allow-anonymous-public |
allowAnonymousPublic |
false |
Opt-in mapping of anonymous Space view to OIS public
|
page-size / parallelism
|
pageSize / parallelism
|
50 / 4
|
API page size (1–250) and Virtual Thread concurrency |
max-retries / initial-backoff-ms / timeout-seconds
|
maxRetries / initialBackoffMs / timeoutSeconds
|
5 / 1000 / 30
|
Retry/backoff on HTTP 429/5xx and HTTP timeout controls |
spring:
opencrawling:
repository-connector-type: confluence
connector:
type: confluence
confluence:
base-url: "https://acme.atlassian.net/wiki"
deployment-type: "cloud"
auth-type: "api_token"
username: "crawler-bot@acme.com"
api-token: "${CONFLUENCE_API_TOKEN}"
space-keys: "ENG,KB,SEC"
included-labels: "kb"
excluded-labels: "draft,archived"
include-blog-posts: true
include-attachments: true
body-format: "storage"
max-attachment-size-bytes: 52428800
crawl-mode: "cql_delta"
cql-overlap-minutes: 5
full-scan-interval: "P7D"
full-scan-window: "02:00-05:00"
full-scan-timezone: "UTC"
include-acls: true
user-identity: "email_and_id"
allow-anonymous-public: false
page-size: 50
parallelism: 4Canonical URI format: confluence://<host>/spaces/<spaceKey>/pages/<contentId> (or /blogposts/<contentId>):
{
"id": "confluence://acme.atlassian.net/spaces/ENG/pages/100",
"action": "UPSERT",
"acl": "alice@example.com,engineering,security",
"metadata": {
"source_type": ["confluence"],
"title": ["Architecture Root"],
"name": ["Architecture Root.html"],
"filename": ["Architecture Root.html"],
"mimeType": ["text/html"],
"url": ["https://acme.atlassian.net/wiki/spaces/ENG/pages/100"],
"confluence.webUrl": ["https://acme.atlassian.net/wiki/spaces/ENG/pages/100"],
"path": ["/ENG/Engineering/Architecture Root.html"],
"breadcrumbs": ["Engineering"],
"confluence.breadcrumbs": ["Engineering"],
"tags": ["kb", "architecture"],
"confluence.labels": ["kb", "architecture"],
"confluence.contentType": ["page"],
"confluence.contentId": ["100"],
"confluence.spaceKey": ["ENG"],
"confluence.spaceId": ["10"],
"confluence.version": ["1"],
"confluence.versionKey": ["v:1"],
"confluence_prop_owner": ["platform-team"]
},
"security": {
"inheritanceEnabled": false,
"permissions": [
{ "identity": "alice@example.com", "identityType": "user", "displayName": "Alice Engineer", "access": "read" },
{ "identity": "engineering", "identityType": "group", "displayName": "engineering", "access": "read" },
{ "identity": "security", "identityType": "group", "displayName": "security", "access": "read" }
]
}
}Canonical URI format: confluence://<host>/spaces/<spaceKey>/pages/<parentPageId>/attachments/<attachmentId>. Attachments automatically inherit the owning page's tags (confluence.labels), breadcrumbs, and effective security ACLs.
When a page is deleted, trashed, moved out of scope, or excluded by labels (e.g. labeled archived), the connector emits a DELETE tombstone for the page and cascades DELETE tombstones for every tracked attachment ID in its state record:
{
"id": "confluence://acme.atlassian.net/spaces/ENG/pages/100/attachments/att-pdf",
"action": "DELETE",
"metadata": {
"source_type": ["confluence"],
"confluence.contentId": ["att-pdf"],
"confluence.spaceKey": ["ENG"],
"confluence.contentType": ["attachment"],
"confluence.status": ["DELETED"]
}
}Attachments exceeding max-attachment-size-bytes emit opencrawling.document.status=TOO_LARGE and opencrawling.document.reason=<detail>, which JobOrchestrator records as DocumentLedgerEntry.skipped(..., "TOO_LARGE", reason) in the audit ledger rather than failing the job.
For each item (include-acls: true), ConfluenceSecurityMapper combines Space View Permissions with every restricted ancestor and page read restriction (ignoring update restrictions, which do not restrict viewing in Confluence):
| Confluence Permission Source | Principal Emitted (identity) |
identityType |
Notes |
|---|---|---|---|
| Space View + Unrestricted Page | Space read users & groups |
user / group
|
Cloud group UUIDs resolved & cached via /rest/api/group/by-id
|
Inherited Ancestor / Page read Restrictions |
Intersection across every restricted ancestor level, the page, and Space View |
user / group
|
Supports optional expand-group-members: true for Space-group ∩ page-user intersection |
GDPR User Identity (user-identity) |
email_and_id (default), email, or id
|
user |
Emits accountId/userKey cleanly when Atlassian Cloud GDPR settings hide emails |
| Anonymous Space Access |
public (only when allow-anonymous-public: true and page is unrestricted) |
public |
Ignored by default (allow-anonymous-public: false, fail-closed) |
| Empty / Unreadable Permissions | (None — SecurityConfig.none()) |
— |
Fail-closed (permissions: [], acl: ""); never defaults to public
|
{
"name": "secureVectorSearch",
"arguments": {
"query": "Confluence architecture ingestion pipeline security escalation",
"userPrincipal": "alice@example.com",
"userRoles": "engineering",
"maxResults": 10
}
}| Operation | Confluence Cloud (v2 + v1) | Confluence Data Center (v1) |
|---|---|---|
| Connectivity Probe / List Spaces | GET /api/v2/spaces |
GET /rest/api/space?expand=permissions |
| Space View Permissions |
GET /api/v2/spaces/{id}/permissions (+ /rest/api/group/by-id) |
Inline permissions on /rest/api/space/{key}?expand=permissions
|
| List Current Pages / Blog Posts |
GET /api/v2/spaces/{id}/pages?status=current&body-format=..., /blogposts
|
GET /rest/api/space/{key}/content/page?status=current&expand=..., /rest/api/content?type=blogpost
|
| Fetch Single Page / Ancestors / Children |
GET /api/v2/pages/{id}, /ancestors, /children
|
GET /rest/api/content/{id}?expand=..., /child/page
|
| Labels & Custom Properties |
GET /api/v2/pages/{id}/labels, /properties
|
Inline metadata.labels.results & GET /rest/api/content/{id}/property
|
Direct read Restrictions |
GET /rest/api/content/{id}/restriction/byOperation/read?expand=restrictions.user,restrictions.group |
Same v1 endpoint |
| Attachments & Binary Download |
GET /api/v2/pages/{id}/attachments + downloadLink (cross-origin auth stripped) |
GET /rest/api/content/{id}/child/attachment + _links.download
|
| CQL Incremental Search | GET /rest/api/content/search?cql=... |
Same v1 endpoint |
| OAuth 2.0 Token Refresh | POST https://auth.atlassian.com/oauth/token |
N/A (PAT Bearer or Basic auth) |
- Open Connectors → New Connector and select Atlassian Confluence (Cloud & Data Center) (
org.opencrawling.confluence.ConfluenceRepositoryConnector). - Select Deployment Type (
cloud,data_center, orauto) and Authentication Mode (api_token,pat,oauth2, orbasic). All secret fields (apiToken,clientSecret,refreshToken) are automatically masked as********. - Configure
spaceKeys, optionalrootPageId,includedLabels/excludedLabels, macro lists, attachment MIME filters & size cap, CQL periodic reconciliation schedule (fullScanInterval), and Zero-Trust ACL options. - Click Test Connection to validate credentials, target spaces, and optional
rootPageId.
# Validate Confluence credentials, target space keys, and rootPageId
oc connector check --name Confluence_Repository --type repositorymvn test -pl oc-confluence-repository-connector -amRuns 19 unit and embedded HTTP mock tests (ConfluencePropertiesTest, ConfluenceStorageConverterTest, ConfluenceSecurityMapperTest, ConfluenceClientTest, ConfluenceRepositoryConnectorTest) covering Cloud v2+v1 and Data Center v1 parsing, OAuth 2.0 401 refresh, 429 Retry-After backoff, cross-origin CDN redirect auth stripping, Storage XHTML macro keep/drop conversion, label filtering + cascading attachment tombstones, TOO_LARGE reporting, inherited read restriction intersection, periodic reconciliation for ACL-only drift, empty-listing guard, and Discovery/DocumentFetching SPIs.
./scripts/test-confluence-connector.shStarts the zero-dependency Python 3 Confluence Cloud & Data Center mock (oc-confluence-repository-connector/confluence-mock/confluence_mock.py), validates HTTP 401 rejection, Cloud API Token Basic auth (/wiki/api/v2/spaces, /pages, /attachments, PDF download), Data Center PAT Bearer auth (/confluence/rest/api/space, /content/page, /restriction/byOperation/read), and OAuth 2.0 /oauth/token, then executes all 19 connector tests and 158 oc-runtime wiring tests (ConnectorKindTest, ConnectorFactoryTest, ConnectorCheckerServiceTest, ControlPlaneConfluenceStateStoreTest verifying two independent JDBC replicas share CQL state and JobOrchestrator records TOO_LARGE attachments as SKIPPED, PlatformApiV1UnitTest secret masking for apiToken/personalAccessToken, and PlatformApiV1UiParityTest). Wired into .github/workflows/integration-tests.yml as "Confluence Connector Test Suite".
./scripts/test-confluence-decoupled.sh # Production Dockerfiles
./scripts/test-confluence-decoupled.sh --local # Fast local BuildKit ~/.m2 cache overlay
./scripts/test-confluence-decoupled.sh -s -k # Skip build and keep containers for inspectionBoots oc-confluence-repository-connector/docker/docker-compose-decoupled-with-confluence.yml and verifies 9 end-to-end steps:
- Initial
oc-crawler(cql_delta) crawl intopgvector(vector_store_1024): exact live count (5docs covering Architecture page with keptinfo/expandmacros and droppedtoc/jiramacros, Security Playbook page, Q4 Blog Post, fail-closedPUBanonymous page, and coupled Tika-extracted PDF attachmentatt-pdf),draftpage &image/pngattachment excluded (0rows), 100 MB oversized PDF attachment reported as1 too_large(0rows,0 failed). - Rich metadata (
title,tags,confluence_prop_owner), coupled attachment label & parent inheritance, fail-closedPUBanonymous space, and shared JDBC Control PlanelastModifiedUtcwatermark persistence in PostgreSQLoc_setting. - Secure MCP Server (
secureVectorSearch) enforcement across Alice (engineering), Bob (security), and Outsider. - Live Confluence mutations + incremental CQL delta sync (edit page 101, create new page 105, trash page 100 triggering cascading tombstone for coupled attachment
att-pdf). - Restriction-only update (without bumping
lastModified) reconciled cleanly by atreefull reconciliation crawl.
-
Label & Permission Drift Between Full Scans: Confluence does not increment
version.number(and in many deployments does not update CQLlastModified) when only labels or page restrictions are edited. Keep periodic full reconciliation enabled (full-scan-interval: P7Dor shorter) incql_deltajobs so label and ACL edits always converge. -
Dynamic / Server-Side Macros: Macros that pull live external data at render time (
jira,recently-updated,contentbylabel,livesearch,pagetree,children) are intentionally dropped instoragemode to avoid empty boilerplate chunks. Switchbody-formattoexport_viewif static server-rendered macro HTML is required. -
Comments & Historical Revisions: Only current published versions (
status=current) are indexed; inline/footer comments and old page revisions are not indexed as separate documents.
- Home
- User & Installation Guide
- Quick Start
- VM & Bare-Metal Distribution Installation
- Local Development & Fast Docker Builds
- User Guide
- Architecture
- Zero-Trust Security & ACL Management
- Security and Identity Management
- Document Authorization & OpenFGA
- Enterprise Multi-Agent Integration
- Zero-Trust Data Provider Agent & Active Ingestion
- Document Lifecycle Actions & Tombstones
- Apache Tika 4 Text Extraction
- Configuration Guide
- Native Job Scheduler
- Platform API (
/api/v1) - Platform API: Traceability
- Connectors Reference
- Alfresco Content Services (ACS) Connector
- Alfresco Process Services (APS) Connector
- Flowable Repository Connector
- Camunda Repository Connector
- OASIS CMIS Repository Connector
- Doxis Repository Connector
- M-Files Repository Connector
- SharePoint Online Repository Connector
- Google Drive Repository Connector
- Atlassian Confluence Repository Connector
- IBM FileNet P8 Repository Connector
- OpenText Documentum Repository Connector
- Zammad Repository Connector
- Hyland OpenArch Integration (Repository, Output & MCP)
- Hyland Content Innovation Cloud (CIC) Connector
- Relational Database (JDBC) Connector
- Web Repository Connector
- Apache StormCrawler Repository Connector
- gRPC Push Repository Connector
- Elasticsearch Output Connector
- Apache Solr Output Connector
- Luxir Output Connector
- Apache SeaTunnel Output Connector
- Apache Ozone Output Connector (Migration Mode)
- AWS S3 Output Connector (Migration Mode)
- Alfresco Content Services Output Connector (Migration Mode)
- Apache Iceberg Output Connector
- Doxis Output Connector
- Connector Execution Modes
- Model Context Protocol
- Apache Ozone Claim Check Store
- Lakehouse Audit Reporting & Ledger
- AI-Powered Observability (AIOps)
- Auto-Narrativization Copilot
- Maven Connector Archetypes
- Java Client SDK
- Python Client SDK
- OpenCrawling CLI
- OpenAPI Specification & Swagger UI
- gRPC Internal Transport
- Migration Guide (ManifoldCF)
- Integration Test — Decoupled Pipeline
- M-Files Integration Test
- SharePoint Online Integration Test
- Google Drive Integration Test
- Confluence Integration Test
- IBM FileNet Integration Test
- OpenText Documentum Integration Test
- Zammad Integration Test
- Hyland OpenArch Integration Test
- Hyland CIC Integration Test
- Web Crawler Test
- Elasticsearch Integration Test
- Local Docker Build Test
- Root CLI Runner Scripts Test
- Secure MCP Server Integration Test
- Narrativization Integration Test
- Reporting Integration Test
- AWS S3 Migration Integration Test
- Alfresco Migration Integration Test
- OpenFGA Authorization Integration Test
- CLI Decoupled Integration Test
- Java Client SDK Decoupled Test
- Apache Iceberg Decoupled Test
- Doxis Output Integration Test
- Platform API v1 Test Suite