Repository navigation
Doxis Output Connector
Module: oc-doxis-output-connector
Optional Content-Link Module: oc-doxis-blueline-content-link (Maven profile -Pdoxis-blueline)
Shared Client Module: oc-doxis-client (shared with the Doxis Repository Connector)
Connector Class: org.opencrawling.doxis.output.DoxisOutputConnector (OutputConnector SPI / Doxis_Output)
Decoupled Kafka Writer: org.opencrawling.doxis.output.messaging.DoxisStoreWriterConsumer (topic opencrawling-documents, group opencrawling-doxis-writer)
Configuration Properties: org.opencrawling.doxis.output.config.DoxisOutputProperties (spring.opencrawling.output.doxis.*)
UI Settings Adapter: org.opencrawling.doxis.output.config.DoxisConnectorSettings (doxis* keys)
Spring Activation: @ConditionalOnProperty(name = "spring.opencrawling.output.type", havingValue = "doxis")
The Doxis Output Connector archives crawled OpenCrawling documents into a Doxis 4 (SER Group) DMS repository through the Doxis Content Service Bus (CSB) REST API (/restws/publicws/rest/api/v1, verified against CSB 14.4.1).
It writes the target document type, typed metadata descriptors, Zero-Trust OIS Access Control Lists (ACLs), electronic file (Akte) filings, document versions, and deletion tombstones. By strictly separating the metadata plane from the content plane, DoxisOutputConnector plans how each binary reaches Doxis solely from document metadata — enabling streamed uploads for normal files, reference-only archiving, or zero-copy external content links (CONTENT_LINK) for multi-terabyte assets.
Important
Licence & Session Policy: Archiving into Doxis requires a valid, unexpired Doxis 4 licence (SER Group) for the target CSB whose licence permits API / technical-user sessions. OpenCrawling includes neither Doxis nor any Doxis licence, nor does it bundle SER proprietary client JARs. CSB counts technical-user sessions against your licence:
- A direct UI/CLI/API job creates a per-job connector (
ConnectorFactory) holding one CSB session for the run and automatically logs out (POST /logout) when the job finishes viaConnectorFactory.release(). - A Test Connection check (
ConnectorCheckerService) opens a short-lived session and closes it immediately. - Plan concurrent job session capacity with your Doxis administrator.
Verify all items below before configuring DoxisOutputConnector or starting an archiving job:
-
Valid Doxis 4 Licence & CSB REST API:
- Doxis 4 CSB 14.4 or later (verified on 14.4.1) with
restwsenabled and reachable at<scheme>://<csb-host>:<port>/restws/publicws/rest/api/v1. - Known customer (tenant) short name (listable via
GET /customers). - API / technical-user session capacity permitted by your SER licence.
- For zero-copy
CONTENT_LINK: separately licensed SER Doxis Java (Blueline) client JARs (compiled for Java 17 bytecode, e.g.14.4.1-1), loaded externally viacontent-link.client-lib-dir.
- Doxis 4 CSB 14.4 or later (verified on 14.4.1) with
-
Technical User & Role Permissions (e.g.
role: adminswhen the user's default role lacks write privileges):Operation Required Doxis Right API Sign-in ( POST /login)Login allowed; "change password at next login" must be disabled (otherwise API login is blocked) Document creation & versioning createDocumenton target document classesPer-document ACLs ( security.mode: DOCUMENT)Instance rights enabled on target document classes E-file filing ( PRIMARY_PARENT)Document - Change primary parent object ( SET_PRIMARY_PARENT) on target document classes (otherwiseSECU0015 … setPrimaryParent permission)E-file removal / deletion All instances - Read and All instances - Write (high-volume write right) on the e-file class (otherwise SECU0015 … high volume write permission)Creating folder nodes ( RELATIONSHIPfiling)CREATE_FOLDERon the e-file (granted automatically to the connector user whensecurity.grant-connector-user=true)Resolving OIS identities to principals Read access to /usersand/groups -
Repository Configuration (
cubeDesigner) (cannot be modified via CSB REST API):- Target DMS repository (
repository) exists and allows the configureddocument-type(otherwise creation fails withINSTANCE0014). - Target document type's
allowedMimeTypesincludes the mime types being archived (orapplication/octet-streamas fallback, otherwiseSEDNA0204). - E-file filing requires a content repository for primary parent objects configured under DMS -> Databases (otherwise
INSTANCE0207). - For
CONTENT_LINKon large/external assets, configure a dedicated document class in Doxis Designer with Allow linking contents = True, content indexing off, Creation of representations disabled (no Rendition Server conversion rules), thumbnails/ML off, and File from file system -> File deletion -> Delete files = False.
- Target DMS repository (
-
OpenCrawling Runtime:
- Java 25 runtime (
oc-runtime, or decoupledoc-writer-consumer). - Supply
passwordvia Admin UI secrets, environment variables (SPRING_OPENCRAWLING_OUTPUT_DOXIS_PASSWORD), or an external secret store — never commit secrets to source control.
- Java 25 runtime (
Unlike vector databases that consume embedded text chunks from opencrawling-embedded, DoxisOutputConnector archives source binaries and document metadata directly:
flowchart TD
subgraph Sources["OpenCrawling Sources & Pipeline"]
Repo["Source Repository Connector"]
Orch["JobOrchestrator (Direct / UI Job Path)"]
KafkaDocs["Kafka Topic: opencrawling-documents"]
ClaimCheck["ClaimCheckStore (Lazy Read Only on UPLOAD)"]
Ingest["IngestionConsumer (Skips Tika Above max-extract-bytes)"]
end
subgraph OutputCore["oc-doxis-output-connector"]
Factory["ConnectorFactory / Release on Job End"]
Conn["DoxisOutputConnector"]
Writer["DoxisStoreWriterConsumer (Group: opencrawling-doxis-writer)"]
Planner["ContentPlanner + PrefixLocatorResolver + ContentLinkResolver"]
Mapper["DoxisDocumentMapper + DoxisAclMapper"]
Filer["DoxisRecordFiler (E-Files & Folder Nodes)"]
Client["DoxisClient (oc-doxis-client REST + CQL)"]
end
subgraph Blueline["Optional: -Pdoxis-blueline"]
Loader["ContentLinkWriterLoader (Isolated Child-First ClassLoader)"]
SOAP["BluelineContentLinkWriter (SER Java API via SOAP)"]
end
subgraph Doxis4["SER Doxis 4 CSB (14.4+)"]
REST["CSB REST API (/restws/publicws/rest/api/v1)"]
Transfer["CSB SOAP (/sedna-transfer-service-xf/services/)"]
Store["DMS Repository (Documents, Versions, E-Files, ACLs)"]
end
Repo -->|Direct Mode| Orch
Orch --> Conn
Factory -->|Builds Per-Job Instance & Logs Out| Conn
Repo -->|Decoupled Mode| KafkaDocs
KafkaDocs -->|Parallel Consumer Group| Writer
KafkaDocs -->|RAG Extraction When Enabled| Ingest
Writer -->|Lazy Stream Only If UPLOAD| ClaimCheck
Writer --> Conn
Conn --> Planner
Conn --> Mapper
Conn --> Filer
Conn -->|REST Metadata, Upload, ACLs, CQL, Verify, Delete| Client
Client --> REST
Planner -->|CONTENT_LINK| Loader
Loader --> SOAP
SOAP --> Transfer
REST --> Store
Transfer --> Store
-
Per-Job / Direct Path (
JobController->ConnectorFactory):- Selecting
Doxis_Output(org.opencrawling.doxis.output.DoxisOutputConnector) in the Admin UI, CLI, or REST API resolvesDoxisOutputPropertiesviaDoxisConnectorSettings.fromConfiguration(config)(OutputKind.DOXIS). -
JobControllerruns the job on a Java 25 Virtual Thread and callsconnectorFactory.release(finalOutputConnector)in afinallyblock so the per-job CSB session logs out cleanly when the job finishes (leaving any shared runtime bean untouched).
- Selecting
-
Decoupled Kafka Path (
DoxisStoreWriterConsumer):- Activated when
spring.opencrawling.output.type=doxisandopencrawling.consumer.writer.enabled=true. -
DoxisConnectionInitializervalidates login, repository, document type, and external-id attribute asynchronously at startup (failures warn and retry lazily on the first document). -
DoxisStoreWriterConsumerconsumesIngestionMessageevents directly fromopencrawling-documents(notopencrawling-embedded) in its own isolated consumer group (spring.opencrawling.output.doxis.consumer-group, defaultopencrawling-doxis-writer), running in parallel with text extraction. Any archiving failure is rethrown (IllegalStateException,#333/ PR#336) so the Kafka container'sDefaultErrorHandlerretries the record with exponential back-off and routes exhausted failures toopencrawling-documents.DLT(retaining its claim-check payload viaDeadLetteredClaims,#332). - Binaries are wrapped in
LazyInputStream: theClaimCheckStoreobject is never opened or downloaded unless the document'sContentPlanselectsUPLOAD.
- Activated when
To archive multi-gigabyte or multi-terabyte files without exhausting memory during Tika text extraction:
-
IngestionConsumerenforcesopencrawling.ingestion.max-extract-bytes(default536870912bytes / 512 MiB). - Files exceeding this threshold skip Apache Tika extraction and embedding completely, while
IngestionMessagemetadata still flows toDoxisStoreWriterConsumeronopencrawling-documentsso Doxis archives the metadata, descriptors, ACLs, and content links safely.
ContentPlanner chooses how each binary reaches Doxis by inspecting document metadata only (sizeInBytes/contentLength, sha256/hashValue, file: stat, URI prefixes) without reading content streams upfront:
| Strategy | Behavior | Use Case & Notes |
|---|---|---|
AUTO (Default)
|
Chooses per document: (1) PREDEFINED_LOCATOR if locator resolves (experimental); (2) UPLOAD if size is known and <= content.upload-max-bytes (or unknown size when fallback is UPLOAD); (3) CONTENT_LINK if link resolves and a ContentLinkWriter is loaded; (4) content.fallback (REFERENCE_ONLY by default). |
Best general-purpose default combining direct uploads for normal files with zero-copy links or references for oversized files. |
UPLOAD |
Streams binary as multipart inputStream alongside documentParams (POST …/documents) or documentVersionParams (POST …/versions). Fails if known length exceeds content.upload-max-bytes (default 2147483648 / 2 GiB). |
Standard small-to-medium DMS archiving when binaries should be copied into Doxis storage. |
CONTENT_LINK |
Zero-copy external UNC path or URL registered via oc-doxis-blueline-content-link (IDocument.addContentLinkPartDocument). Supported zero-copy mechanism (verified with a 5 TB UNC test on CSB 14.4.1). |
Multi-terabyte media/archives staying in place on NAS/file shares without copying bytes into Doxis. Requires content-link.client-lib-dir. |
REFERENCE_ONLY |
Creates the document metadata record without binary content; optionally records source URI in reference-attribute. |
Lightweight metadata cataloging pointing back to source systems. |
PREDEFINED_LOCATOR |
Experimental. Sends predefinedLocator, contentLength, and optional SHA-256 without inputStream. Off by default in Doxis storage adapters and unsupported by SER as a zero-copy path in CSB 14.4.1 (see Known Limitations). |
Experimental staging only where custom Doxis storage adapters explicitly enable predefined locators. |
When content.verify=true (default):
- After creating a new document or adding a version (
UPLOAD,PREDEFINED_LOCATOR, orCONTENT_LINK), the connector callsGET …/dmsRepositories/{repo}/documents/{uuid}/versions?initializeRepresentations=trueto inspect the default representation's content object. - Checks that a content object exists (catching silent
NO_CONTENTwrites), matches the expected bytelength(for uploads/locators), comparesfullFilenamefor UNCCONTENT_LINKs, and validatesSHA-256whenever CSB reports a non-blankhashValue. - If filing, ACL assignment, or verification fails on a newly created document,
DoxisOutputConnectorautomatically rolls back the write by physically deleting the partial document (DELETE …/documents/{uuid}) so orphaned, empty, or unprotected records never remain in Doxis.
-
External ID:
document.id()is stored inexternal-id-attribute(defaultObjectNumber) and searched before every write via CQL:SELECT * FROM <repoShortName> WHERE <field> = '<value>'. -
Hash Truncation Protection: IDs exceeding the Doxis descriptor's
lengthare hashed deterministically toh:<sha256Prefix>so lookups stay reliable. -
conflict-resolutionon existing documents:-
NEW_VERSION(Default): Appends a new document version viaPOST …/documents/{uuid}/versions(note: forCONTENT_LINK, re-crawls refresh descriptors and keep the existing link instead of adding versions). -
UPDATE_METADATA: Patches descriptors on the current version (updateAttributes) and syncs permissions without creating a new version or transferring content. -
SKIP: Leaves existing documents untouched.
-
-
Unchanged Detection (
content.change-marker-attribute): When configured (e.g.SystemId1), stores<lastModified>|<length>per version. OnNEW_VERSIONre-crawls, matching markers skip version creation cleanly.
-
title-attribute(defaultObjectName) receives OIStitle,name, file name, or document ID. - Optional
reference-attributestoresdocument.uri(). -
attribute-mapping(<OIS key>: <Doxis attribute>, orkey=Attributein UI/JSON) converts values automatically using CSB schema metadata (DoxisSchema):-
STRING: strips NUL bytes and truncates to descriptorlength. -
INTEGER,LONGINTEGER,FLOATINGPOINT,BOOL(true/1/y). -
DATE/DATETIME: accepts epoch milliseconds, ISO-8601 instants/offsets, orYYYY-MM-DDdates. - Multi-value descriptors preserve all list values; single-value descriptors take the first element.
- Pseudo-keys
id,uri, andlastModifiedmap top-levelRepositoryDocumentfields directly.
-
filing.mode controls where newly created documents are filed:
filing.mode |
Behavior |
|---|---|
METADATA (Default)
|
Files into record UUID from metadata key filing.record-id-metadata-key (default doxisRecordId), falling back to filing.record-id. |
NONE |
Do not file documents into any e-file. |
FIXED |
Always file into filing.record-id. |
SOURCE_FOLDER |
One e-file per parent folder extracted from document.uri(); looked up in filing.record-class by filing.record-key-attribute and auto-created if filing.auto-create=true. |
KEY_METADATA |
One e-file per value of metadata key filing.record-key-metadata-key (e.g. customer ID), looked up/auto-created like SOURCE_FOLDER. |
filing.method controls how the document attaches to the e-file:
-
PRIMARY_PARENT(Default): CallsPUT …/documents/{uuid}/primaryParentafter creation (works for uploads andCONTENT_LINK). -
RELATIONSHIP: SendsrelationshipParamsduring REST creation intofiling.folder-node-id(doxisFolderNodeIdmetadata overrides) or the document-capableSTATICfolder node namedfiling.folder-node-name(defaultDocuments), auto-created under the e-file'sNODES_ONLYroot when missing. Applies to uploads only;CONTENT_LINKfalls back toPRIMARY_PARENT. - Stale cached e-file/node IDs are evicted automatically on HTTP
404or DoxisINSTANCE*errors.
Identities in SecurityConfig are resolved via DoxisSchema to Doxis users (by name or login name) or groups; OIS public maps to Doxis group everybody.
OIS access
|
Document Permission (DocumentAceParams) |
E-File (Akte) Permission |
|---|---|---|
read |
GRANT VIEW_DOCUMENT_CONTENTS |
GRANT VIEW_FOLDER_CONTENTS |
write |
GRANT VIEW_DOCUMENT_CONTENTS, UPDATE_DOCUMENT, VERSION_DOCUMENT
|
GRANT VIEW_FOLDER_CONTENTS, UPDATE_FOLDER, EDIT_FOLDER_DESCRIPTORS
|
deny |
DENY VIEW_DOCUMENT_CONTENTS |
DENY VIEW_FOLDER_CONTENTS |
Security Controls (spring.opencrawling.output.doxis.security.*):
-
mode:DOCUMENT(default, per-document ACEs),RECORD(e-file ACEs inherited by documents when class enables "Primary parent objects -> Pass down permissions"),DOCUMENT_AND_RECORD, orNONE(class ACL only; also forced ifapply-security-acls=false). -
grant-connector-user(defaulttrue): Grants the connector's own technical userVIEW_FOLDER_CONTENTS,UPDATE_FOLDER,EDIT_FOLDER_DESCRIPTORS,DELETE_FOLDER,CREATE_FOLDER,SET_PRIMARY_PARENT, andVIEW_REMOVED_FOLDERon e-files it creates so instance ACLs do not lock the connector out of managing its own e-files. -
record-acl-sync:CREATE_ONLY(default) orADDITIVE(adds missing ACEs when later or re-crawled documents enter an e-file; never removes existing ACEs). -
remove-stale(defaultfalse): On document re-crawl, removes connector-managed document ACEs (VIEW_DOCUMENT_CONTENTS,UPDATE_DOCUMENT,VERSION_DOCUMENT) no longer present at source while leaving Doxis administrator-managed permissions untouched. -
strict(defaultfalse): Fails and rolls back a newly created document if its document class rejects per-document permissions (SECU0050) or if any OIS identity cannot be resolved in Doxis; whenfalse, logs warnings instead.
Lookups match archived documents by external-id-attribute and apply delete-mode:
-
LOGICAL(Default): ExecutesPOST …/remove— reversible removal that preserves binaries on disk (recommended when using in-place / content-link storage or shared content repositories). -
PHYSICAL: ExecutesDELETE …/documents/{uuid}— irrevocable deletion. (Verified on CSB 14.4.1: physical delete of aCONTENT_LINKdocument removes only the Doxis metadata record and leaves the external target file untouched.)
Because Doxis CSB REST API 14.4.x has no endpoint to attach external content links, OpenCrawling provides an optional companion module oc-doxis-blueline-content-link implementing ContentLinkWriter via SER's Doxis Java (Blueline) API (IDocument.addContentLinkPartDocument()).
- Excluded from default build: SER client JARs are proprietary and not redistributed with OpenCrawling.
-
Isolated ClassLoading:
ContentLinkWriterLoaderloadsoc-doxis-blueline-content-linkand the SER client JARs fromcontent-link.client-lib-dirin a child-first isolated class loader sharing only the writer interface, SLF4J, and JDK classes, preventing Spring/CXF/Jackson version conflicts with OpenCrawling. -
SOAP Endpoint: Speaks to
http://<csb-host>:<port>/sedna-transfer-service-xf/services/(defaults to host and port frombase-url; override viacontent-link.csb-host/content-link.csb-port).
# 1. Install SER client jar from your Doxis installation (e.g. webCube WEB-INF/lib)
mvn install:install-file \
-Dfile=/path/to/WEB-INF/lib/blueline-14.4.1-1.jar \
-DgroupId=de.ser.doxis4.java-api \
-DartifactId=blueline \
-Dversion=14.4.1-1 \
-Dpackaging=jar \
-DgeneratePom=true
# 2. Build the optional module (Java 25; SER jars are Java 17 bytecode 14.4.1-1)
mvn -Pdoxis-blueline install -pl oc-doxis-blueline-content-link
# 3. Stage all SER client jars + OpenCrawling Blueline adapter into client-lib-dir
mkdir -p /opt/doxis-client-lib
cp /path/to/WEB-INF/lib/*.jar /opt/doxis-client-lib/
cp oc-doxis-blueline-content-link/target/oc-doxis-blueline-content-link-*.jar /opt/doxis-client-lib/Required SER Maven coordinates referenced by the build/runtime: de.ser.doxis4.java-api:blueline, de.ser.doxis4.java-api:sednaclient-blueline, and de.ser.doxis4.csb.client:sednaclient-api-nodep.
Note
CSB Serving & Re-crawl Semantics: CSB does not stream CONTENT_LINK binaries over REST GET …/contentObjects/{id} (SEDNA0104); opening clients resolve UNC/URL paths directly. Re-crawling a CONTENT_LINK document updates descriptors and syncs ACLs while keeping the original link (no extra versions are created).
All Spring Boot properties bind under spring.opencrawling.output.doxis.*, and per-connector Admin UI / connectors.json entries map via DoxisConnectorSettings:
Spring Property (spring.opencrawling.output.doxis.*) |
UI / JSON Key (doxis*) |
Default | Description |
|---|---|---|---|
base-url |
doxisBaseUrl |
http://localhost:8080/restws/publicws/rest/api/v1 |
CSB REST API base URL |
customer-name |
doxisCustomerName |
— | Required. Doxis customer (tenant) short name |
username |
doxisUsername |
— | Required. Technical archiving user |
password |
doxisPassword |
— | Required. Technical user password |
role |
doxisRole |
— | Optional Doxis role for login (e.g. admins) |
repository |
doxisRepository |
— | Required. Target DMS repository name or UUID |
document-type |
doxisDocumentType |
BaseDocument |
Document class name or UUID allowed in repository |
external-id-attribute |
doxisExternalIdAttribute |
ObjectNumber |
Descriptor holding OIS ID for CQL upsert/delete lookup |
title-attribute |
doxisTitleAttribute |
ObjectName |
Descriptor receiving document title |
reference-attribute |
doxisReferenceAttribute |
— | Optional descriptor storing source URI |
attribute-mapping.<key> |
doxisAttributeMapping |
{} |
Map <OIS key> (id, uri, lastModified, or metadata) to Doxis descriptor |
conflict-resolution |
doxisConflictResolution |
NEW_VERSION |
Duplicate handling: NEW_VERSION, UPDATE_METADATA, SKIP
|
delete-mode |
doxisDeleteMode |
LOGICAL |
Tombstone deletion: LOGICAL (POST …/remove) or PHYSICAL (DELETE …) |
apply-security-acls |
doxisApplySecurityAcls |
true |
Enables OIS ACL mapping (setting false forces security.mode=NONE) |
content.strategy |
doxisContentStrategy |
AUTO |
AUTO, UPLOAD, CONTENT_LINK, REFERENCE_ONLY, PREDEFINED_LOCATOR
|
content.upload-max-bytes |
doxisUploadMaxBytes |
2147483648 |
Max binary size streamed for UPLOAD / AUTO (2 GiB) |
content.fallback |
doxisContentFallback |
REFERENCE_ONLY |
AUTO fallback (REFERENCE_ONLY or UPLOAD) |
content.verify |
doxisVerifyContent |
true |
Read-back verification of content object, length, and SHA-256 |
content.change-marker-attribute |
doxisChangeMarkerAttribute |
— | Descriptor storing <lastModified>|<length> to skip unchanged versions |
locator.metadata-key |
doxisLocatorMetadataKey |
doxisLocator |
Metadata key supplying explicit predefinedLocator
|
locator.uri-prefix |
doxisLocatorUriPrefix |
— | Source URI prefix stripped to form locator |
locator.prefix |
doxisLocatorPrefix |
"" |
Prefix prepended to derived locators |
filing.mode |
doxisFilingMode |
METADATA (or FIXED if record-id set)
|
NONE, FIXED, METADATA, SOURCE_FOLDER, KEY_METADATA
|
filing.method |
doxisFilingMethod |
PRIMARY_PARENT |
PRIMARY_PARENT or RELATIONSHIP
|
filing.record-id |
doxisFilingRecordId |
— | Target e-file UUID for FIXED / fallback |
filing.record-repository |
doxisFilingRecordRepository |
Target repository
|
Repository hosting target e-files |
filing.folder-node-id |
doxisFilingFolderNodeId |
— | Explicit folder node UUID for RELATIONSHIP filing |
filing.folder-node-name |
doxisFilingFolderNodeName |
Documents |
Auto-located/created STATIC folder node name for RELATIONSHIP
|
filing.record-id-metadata-key |
doxisFilingRecordIdMetadataKey |
doxisRecordId |
Metadata key specifying e-file UUID in METADATA mode |
filing.folder-node-metadata-key |
doxisFilingFolderNodeMetadataKey |
doxisFolderNodeId |
Metadata key overriding folder-node-id per document |
filing.record-key-metadata-key |
doxisFilingRecordKeyMetadataKey |
— | Metadata key identifying e-file in KEY_METADATA mode |
filing.record-class |
doxisFilingRecordClass |
— | E-file class used when auto-creating records |
filing.record-key-attribute |
doxisFilingRecordKeyAttribute |
ObjectNumberExternal |
Descriptor storing keyed e-file lookup key |
filing.record-title-attribute |
doxisFilingRecordTitleAttribute |
ObjectName |
Descriptor receiving e-file display title |
filing.auto-create |
doxisFilingAutoCreate |
true |
Auto-create missing e-files and STATIC folder nodes |
security.mode |
doxisSecurityMode |
DOCUMENT |
DOCUMENT, RECORD, DOCUMENT_AND_RECORD, NONE
|
security.strict |
doxisSecurityStrict |
false |
Roll back new docs on SECU0050 or unresolvable identities |
security.remove-stale |
doxisSecurityRemoveStale |
false |
Prune stale connector-managed document ACEs on re-crawl |
security.record-acl-sync |
doxisSecurityRecordAclSync |
CREATE_ONLY |
E-file ACL maintenance: CREATE_ONLY or ADDITIVE
|
security.grant-connector-user |
doxisSecurityGrantConnectorUser |
true |
Grant connector user rights on e-files it creates |
content-link.client-lib-dir |
doxisContentLinkClientLibDir |
— | Directory with Blueline + SER JARs; enables CONTENT_LINK
|
content-link.csb-host |
doxisContentLinkCsbHost |
Host of base-url
|
CSB SOAP host override for Blueline |
content-link.csb-port |
doxisContentLinkCsbPort |
Port of base-url (0 -> 8080) |
CSB SOAP port override for Blueline |
content-link.uri-prefix |
doxisContentLinkUriPrefix |
— | URI prefix rewritten for CONTENT_LINK
|
content-link.link-prefix |
doxisContentLinkPrefix |
"" |
Replacement UNC or URL prefix (/ -> \ for UNC) |
content-link.link-type |
doxisContentLinkType |
UNC |
External link format: UNC or URL
|
content-link.metadata-key |
doxisContentLinkMetadataKey |
doxisContentLink |
Metadata key supplying an explicit external link |
content-link.document-type |
doxisContentLinkDocumentType |
Main document-type
|
Dedicated Doxis class allowing content links |
max-retries |
doxisMaxRetries |
3 |
HTTP retry count on transport failures / 429 / 502–504
|
timeout-seconds |
doxisTimeoutSeconds |
120 |
CSB HTTP client timeout in seconds |
consumer-group |
— | opencrawling-doxis-writer |
Kafka consumer group of DoxisStoreWriterConsumer
|
Tip
Metadata Extraction Bound: Configure opencrawling.ingestion.max-extract-bytes (default 536870912 / 512 MiB) when running oc-ingestion-consumer alongside DoxisStoreWriterConsumer so multi-gigabyte binaries bypass Tika parsing while still being archived in Doxis.
spring:
opencrawling:
output:
type: doxis
doxis:
base-url: "http://csb.example.com:8080/restws/publicws/rest/api/v1"
customer-name: faststarter
username: crawler
password: "${DOXIS_PASSWORD}"
role: admins
repository: D_TEXTER
document-type: TX_MigratedDocument
external-id-attribute: ObjectNumberExternal
title-attribute: ObjectName
reference-attribute: URL
attribute-mapping:
author: ObjectAuthors
lastModified: ObjectDate
content:
strategy: AUTO
upload-max-bytes: 2147483648
verify: true
change-marker-attribute: SystemId1
content-link:
client-lib-dir: /opt/doxis-client-lib
uri-prefix: "file:///mnt/archive/"
link-prefix: "\\\\fileserver\\archive\\"
link-type: UNC
document-type: TX_LargeExternalDocument
filing:
mode: SOURCE_FOLDER
method: PRIMARY_PARENT
record-class: TX_SourceFolder
auto-create: true
security:
mode: RECORD
record-acl-sync: ADDITIVE
grant-connector-user: true
opencrawling:
consumer:
writer:
enabled: true
ingestion:
max-extract-bytes: 536870912-
Admin UI (
oc-admin-ui): Registered as default output connectorDoxis_Output— Doxis 4 ECM Archive (CSB REST API) (org.opencrawling.doxis.output.DoxisOutputConnector), exposing sections for connection credentials, descriptors, binary strategies, optional Blueline content links, e-file filing, and lifecycle/security controls. -
Connection Check (
oc-cli& UI Test Connection):Logs in to CSB (oc connector check --name Doxis_Output --type output
/login), fetches/session/userand/dmsRepositories/{repository}, and immediately logs out (/logout).
# Core output connector (77 tests against recorded CSB 14.4.1 fixtures & OkHttp MockWebServer)
mvn test -pl oc-doxis-client,oc-doxis-output-connector -am
# Optional Blueline content-link module (requires SER blueline jar installed locally)
mvn -Pdoxis-blueline test -pl oc-doxis-blueline-content-linkThere is no open-source Doxis Docker container; use scripts/test-doxis-connector.sh against a reachable CSB instance. When CSB variables are unset, the script skips cleanly (exit 0):
DOXIS_BASE_URL=http://csb-host:8080/restws/publicws/rest/api/v1 \
DOXIS_CUSTOMER=faststarter DOXIS_USER=crawler DOXIS_PASSWORD=... \
DOXIS_ROLE=admins DOXIS_REPOSITORY=D_TEXTER \
DOXIS_EXTERNAL_ID_FIELD=OBJECTNUMBER2 \
DOXIS_WRITE_TEST=true \
DOXIS_DOCUMENT_TYPE_UUID=<type-uuid> \
DOXIS_EXTERNAL_ID_ATTRIBUTE_UUID=<attr-uuid> \
DOXIS_MIME_TYPE=text/plain \
./scripts/test-doxis-connector.shVerifies:
- CSB liveness (
/sedna-transfer-service-xf/isAlive) and tenant presence (GET /customers). - JWT
/login,/session/user, repository inspection (shortName,informationObjectTypeIds), and CQL/documents/search. -
(Opt-in
DOXIS_WRITE_TEST=true) Multipart document creation, read-back verification of stored byte length via/versions?initializeRepresentations=true, physical delete cleanup, and/logout.
Observations recorded against Doxis CSB 14.4.1 (2026-10):
-
PREDEFINED_LOCATORis experimental and off by default: SER confirmed (2026-10-02) thatpredefinedLocatoris not a supported zero-copy mechanism in 14.4.1; CSB either rejectspredefinedLocatorwith content or silently storesNO_CONTENT(caught and rolled back byverify). Useoc-doxis-blueline-content-link(CONTENT_LINK) for supported zero-copy external binaries. -
No REST content-link endpoint in CSB 14.4.x:
CONTENT_LINKrequires-Pdoxis-bluelineplus licensed SER Java client JARs. Calls inBluelineContentLinkWriterare serialized per writer session. -
SHA-256 read-back: CSB 14.4.1 does not return client-supplied
hashValueon read-back;verify()validates exact byte length and validates SHA-256 only when CSB returns one. -
RELATIONSHIPfiling vs.PRIMARY_PARENT: Documents filed viaRELATIONSHIPinto aSTATICfolder node do not set aprimaryParent; content links always fall back toPRIMARY_PARENT. -
Single-use HTTP upload retries: HTTP
429/502–504exponential retries apply only to repeatable requests; non-reopenable binary upload streams are sent once. -
Decoupled upload vs. claim-check cleanup: In decoupled Kafka mode,
IngestionConsumerwithclaimcheck.cleanup-on-consume=truecan delete claim-check payloads beforeDoxisStoreWriterConsumeruploads them. Use direct job execution, content links, or disable premature cleanup when archiving binaries from Kafka. -
Effective non-admin inheritance check: Removing an already logically deleted e-file returns HTTP
500in CSB 14.4.1; non-admin effective inheritance on e-file ACLs should be validated directly in your target Doxis security model.
- Home
- User & Installation Guide
- Quick Start
- VM & Bare-Metal Distribution Installation
- Local Development & Fast Docker Builds
- User Guide
- Architecture
- Zero-Trust Security & ACL Management
- Security and Identity Management
- Document Authorization & OpenFGA
- Enterprise Multi-Agent Integration
- Zero-Trust Data Provider Agent & Active Ingestion
- Document Lifecycle Actions & Tombstones
- Apache Tika 4 Text Extraction
- Configuration Guide
- Native Job Scheduler
- Platform API (
/api/v1) - Platform API: Traceability
- Connectors Reference
- Alfresco Content Services (ACS) Connector
- Alfresco Process Services (APS) Connector
- Flowable Repository Connector
- Camunda Repository Connector
- OASIS CMIS Repository Connector
- Doxis Repository Connector
- M-Files Repository Connector
- SharePoint Online Repository Connector
- Google Drive Repository Connector
- Atlassian Confluence Repository Connector
- IBM FileNet P8 Repository Connector
- OpenText Documentum Repository Connector
- Zammad Repository Connector
- Hyland OpenArch Integration (Repository, Output & MCP)
- Hyland Content Innovation Cloud (CIC) Connector
- Relational Database (JDBC) Connector
- Web Repository Connector
- Apache StormCrawler Repository Connector
- gRPC Push Repository Connector
- Elasticsearch Output Connector
- Apache Solr Output Connector
- Luxir Output Connector
- Apache SeaTunnel Output Connector
- Apache Ozone Output Connector (Migration Mode)
- AWS S3 Output Connector (Migration Mode)
- Alfresco Content Services Output Connector (Migration Mode)
- Apache Iceberg Output Connector
- Doxis Output Connector
- Connector Execution Modes
- Model Context Protocol
- Apache Ozone Claim Check Store
- Lakehouse Audit Reporting & Ledger
- AI-Powered Observability (AIOps)
- Auto-Narrativization Copilot
- Maven Connector Archetypes
- Java Client SDK
- Python Client SDK
- OpenCrawling CLI
- OpenAPI Specification & Swagger UI
- gRPC Internal Transport
- Migration Guide (ManifoldCF)
- Integration Test — Decoupled Pipeline
- M-Files Integration Test
- SharePoint Online Integration Test
- Google Drive Integration Test
- Confluence Integration Test
- IBM FileNet Integration Test
- OpenText Documentum Integration Test
- Zammad Integration Test
- Hyland OpenArch Integration Test
- Hyland CIC Integration Test
- Web Crawler Test
- Elasticsearch Integration Test
- Local Docker Build Test
- Root CLI Runner Scripts Test
- Secure MCP Server Integration Test
- Narrativization Integration Test
- Reporting Integration Test
- AWS S3 Migration Integration Test
- Alfresco Migration Integration Test
- OpenFGA Authorization Integration Test
- CLI Decoupled Integration Test
- Java Client SDK Decoupled Test
- Apache Iceberg Decoupled Test
- Doxis Output Integration Test
- Platform API v1 Test Suite