Repository navigation
Apache Iceberg Output Connector
Module: oc-iceberg-output-connector
Direct Connector Class: org.opencrawling.iceberg.output.IcebergOutputConnector
Decoupled Kafka Writer: org.opencrawling.iceberg.output.messaging.IcebergStoreWriterConsumer
Catalog Factory: org.opencrawling.iceberg.output.catalog.IcebergCatalogFactory
Table & Schema Manager: org.opencrawling.iceberg.output.schema.IcebergTableManager
Record & Parquet Writer: org.opencrawling.iceberg.output.writer.IcebergRecordWriter
Spring Bean: @Component (activated when spring.opencrawling.output.type=iceberg)
The Apache Iceberg Output Connector integrates Apache Iceberg (v1.11.0) into OpenCrawling as an enterprise-grade lakehouse table destination. It allows structured, semi-structured, and unstructured documents crawled from enterprise repositories (SharePoint, Alfresco, Documentum, FileNet, M-Files, Doxis, JDBC, Web, gRPC Push) to be streamed into high-performance columnar Apache Parquet tables managed under ACID transactions with snapshot isolation, time travel, and partition pruning.
The connector supports both direct synchronous batching (IcebergOutputConnector.commitBatch(...)) and asynchronous decoupled streaming via Kafka (IcebergStoreWriterConsumer). Every ingested document is written with the canonical OpenCrawling lakehouse schema, complete with Zero-Trust Open Ingestion Standard (OIS) Access Control Lists (ACLs), cryptographic checksums, extracted text, and dynamic entity evolution.
flowchart TD
subgraph Ingestion["OpenCrawling Ingestion Pipeline"]
Doc["Crawled RepositoryDocument"]
Direct["IcebergOutputConnector<br/>(Direct / Standalone Batching)"]
Crawler["oc-crawler / Ingestion"]
Kafka[("Kafka Topic<br/>(opencrawling-documents)")]
Writer["IcebergStoreWriterConsumer<br/>(Decoupled Kafka Stage)"]
ClaimCheck[("Claim Check Store<br/>MinIO / S3 / Ozone")]
end
subgraph CoreEngine["Iceberg Lakehouse Core Engine"]
Factory["IcebergCatalogFactory<br/>(REST, Hive, Hadoop, Glue, JDBC, In-Memory)"]
TableMgr["IcebergTableManager<br/>(Namespace Provisioning & PartitionSpec)"]
SchemaEvol["Dynamic Schema Evolution<br/>(Custom Entity Columns)"]
RecWriter["IcebergRecordWriter<br/>(GenericRecord + Parquet FileAppender)"]
end
subgraph Storage["Lakehouse Storage Tier"]
Catalog["Iceberg REST Catalog / Metastore"]
S3IO["org.apache.iceberg.aws.s3.S3FileIO<br/>(AWS S3 / MinIO)"]
ParquetData[("Columnar Parquet Data Files<br/>(s3://warehouse/...)")]
Snapshots[("Iceberg Metadata & Snapshots<br/>(ACID Appends & Time Travel)")]
end
subgraph Consumption["Downstream Analytical Consumption"]
DuckDB["DuckDB OLAP Engine"]
Spark["Apache Spark"]
Trino["Trino / Starburst"]
Snowflake["Snowflake / BigQuery"]
end
Doc -->|Direct Mode| Direct
Doc -->|Decoupled Mode| Crawler
Crawler -->|Upload Binary| ClaimCheck
Crawler -->|Publish IngestionMessage| Kafka
Kafka --> Writer
Writer -->|Stream Binary| ClaimCheck
Direct & Writer --> TableMgr
Factory --> Catalog
TableMgr --> Factory
TableMgr --> SchemaEvol
SchemaEvol --> RecWriter
RecWriter --> S3IO
S3IO --> ParquetData
RecWriter --> Snapshots
Catalog --> Snapshots
ParquetData & Snapshots <--> DuckDB & Spark & Trino & Snowflake
-
Multi-Catalog Architecture: Pluggable support for enterprise catalog technologies through
IcebergCatalogFactory:-
REST Catalog (
catalog-type: rest): Standard REST catalog protocol compatible with Apache Iceberg REST fixture, Tabular, Apache Polaris, and Lakekeeper. -
Hive Metastore (
catalog-type: hive): Enterprise metadata integration viaiceberg-hive-metastore. -
Hadoop Catalog (
catalog-type: hadoop): Direct filesystem or HDFS/S3 directory-backed catalog. -
In-Memory Catalog (
catalog-type: in-memory): Ephemeral catalog for testing and isolated validation. -
AWS Glue Catalog (
catalog-type: glue): Native AWS lakehouse integration via AWS SDK Glue clients. -
JDBC Catalog (
catalog-type: jdbc): Relational catalog backend via JDBC.
-
REST Catalog (
- Canonical Enterprise Lakehouse Schema: Every document is mapped to a strongly typed schema including document identification, content typing, cryptographic SHA-256 validation, extracted text, and claim check URIs.
-
Zero-Trust OIS ACL Security Enforcement: Preserves document access control lists directly within Iceberg table schemas:
-
security_owner(string): Primary document owner / author. -
security_inheritance(boolean): Whether permission inheritance is active. -
allowed_principals(list<string>): Authorized users, groups, and roles mapped viaEffectiveAcl. -
denied_principals(list<string>): Explicitly denied identities.
-
-
Dynamic Runtime Schema Evolution: Custom extracted metadata and entities (e.g.
entity.vendor,entity.amount,entity.contract_id,invoice_date,iban) are detected at runtime. Whenauto-schema-evolution=true,IcebergTableManagerautomatically callstable.updateSchema().addColumn(...)to add new strongly-typed columns without table locking, downtime, or rewriting existing Parquet data files. -
Flexible Table Routing Strategies:
-
UNIFIED(default): Routes all documents into a centralized enterprise lakehouse table (enterprise_documents). -
PER_DOCUMENT_TYPE: Dynamically creates and routes documents into dedicated tables partitioned by document category (e.g.,invoices,contracts,claims,cmis_document).
-
-
Optimized Partitioning Transforms: Configurable partition strategies through
partition-by:-
YEAR_MONTH(default): Automatic month transform oncrawled_at(builder.month("crawled_at")), preventing small-file proliferation while maintaining fast range queries. -
SOURCE_AND_DATE: Partitioned by source repository (source_type) and month of ingestion (crawled_at). -
NONE: Unpartitioned tables for lightweight collections.
-
-
Columnar Parquet Storage & Compression: Writes native Parquet files using
GenericParquetWriterwith configurable compression codecs (zstdby default for superior compression and decompression throughput, orsnappy,gzip,uncompressed). -
S3FileIO Storage Layer: Bundled with
iceberg-awsand AWS SDK v2, enabling direct streaming to Amazon S3, MinIO, Ceph, and Apache Ozone S3 Gateway with path-style access, custom endpoints, and explicit region credentials. -
ACID Transactions & Snapshot Isolation: Writes are committed using Iceberg's atomic append API (
table.newAppend().appendFile(dataFile).commit()), ensuring writers never block readers and analytical engines always query clean snapshots. -
OIS Document Lifecycle & Tombstone Handling: Processes
DocumentAction.DELETEevents by recording explicit tombstone rows (action: DELETE) with snapshot audit trails, enabling downstream CDC and soft-deletion queries.
The default schema provisioned by IcebergTableManager (CANONICAL_SCHEMA) defines the following fields:
| Field ID | Column Name | Type | Required | Description |
|---|---|---|---|---|
1 |
id |
string |
Yes | Unique record UUID for this ingestion event |
2 |
document_id |
string |
Yes | Canonical repository document identifier |
3 |
source_type |
string |
No | Source repository connector type (e.g., sharepoint, alfresco, jdbc, web) |
4 |
document_uri |
string |
No | Original repository URI, file path, or locator |
5 |
title |
string |
No | Document title or filename |
6 |
mime_type |
string |
No | MIME content type (e.g., application/pdf, text/plain) |
7 |
content_length |
long |
No | Binary length in bytes |
8 |
sha256_checksum |
string |
No | SHA-256 cryptographic checksum of raw binary |
9 |
claim_check_uri |
string |
No | Object storage URI in Claim-Check store (s3://..., o3://...) |
10 |
extracted_text |
string |
No | Full-text extracted content (parsed via Apache Tika) |
11 |
extracted_entities |
map<string, string> |
No | Key-value dictionary of all extracted entities |
12 |
custom_metadata |
map<string, string> |
No | Key-value dictionary of repository-specific metadata |
13 |
security_owner |
string |
No | Primary document author or security owner |
14 |
security_inheritance |
boolean |
No | Whether folder/space permission inheritance is enabled |
15 |
allowed_principals |
list<string> |
No | Normalized allowed identities (u:<user>, g:<group>, role:<role>) |
16 |
denied_principals |
list<string> |
No | Normalized denied identities (explicit deny precedence) |
17 |
action |
string |
Yes | Open Ingestion Standard action (UPSERT or DELETE) |
18 |
crawled_at |
timestamp |
Yes | Crawl timestamp in UTC (used for partition transforms) |
| dyn |
* (evolved) |
string |
No | Dynamically evolved entity columns (e.g., vendor, revenue, contract_id) |
All properties are configured under spring.opencrawling.output.iceberg.* (IcebergOutputProperties):
| Property | Environment Variable | Default | Description |
|---|---|---|---|
spring.opencrawling.output.iceberg.catalog-type |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_CATALOG_TYPE |
in-memory |
Iceberg catalog type: rest, hive, hadoop, glue, jdbc, or in-memory
|
spring.opencrawling.output.iceberg.catalog-uri |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_CATALOG_URI |
"" |
URI for remote catalog service (e.g., http://localhost:8181 for REST, thrift://... for Hive) |
spring.opencrawling.output.iceberg.catalog-name |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_CATALOG_NAME |
enterprise_lakehouse |
Logical catalog name identifier |
spring.opencrawling.output.iceberg.warehouse |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_WAREHOUSE |
tmp/iceberg-warehouse |
Lakehouse root warehouse location URI (e.g., s3://warehouse, hdfs:///lakehouse) |
spring.opencrawling.output.iceberg.namespace |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_NAMESPACE |
opencrawling_ecm |
Target Iceberg namespace / database name |
spring.opencrawling.output.iceberg.table-name |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_TABLE_NAME |
enterprise_documents |
Target Iceberg table name (under UNIFIED table strategy) |
spring.opencrawling.output.iceberg.table-strategy |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_TABLE_STRATEGY |
UNIFIED |
Table routing strategy: UNIFIED or PER_DOCUMENT_TYPE
|
spring.opencrawling.output.iceberg.auto-create-table |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_AUTO_CREATE_TABLE |
true |
Automatically create namespace and table on startup if missing |
spring.opencrawling.output.iceberg.auto-schema-evolution |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_AUTO_SCHEMA_EVOLUTION |
true |
Automatically evolve table schema when new metadata entity fields are discovered |
spring.opencrawling.output.iceberg.partition-by |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_PARTITION_BY |
YEAR_MONTH |
Partition strategy: YEAR_MONTH, SOURCE_AND_DATE, or NONE
|
spring.opencrawling.output.iceberg.write-format |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_WRITE_FORMAT |
parquet |
Data file format: parquet
|
spring.opencrawling.output.iceberg.compression-codec |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_COMPRESSION_CODEC |
zstd |
Parquet compression codec: zstd, snappy, gzip, or uncompressed
|
spring.opencrawling.output.iceberg.batch-size |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_BATCH_SIZE |
500 |
Document batch threshold before flushing a Parquet data file |
spring.opencrawling.output.iceberg.flush-interval-ms |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_FLUSH_INTERVAL_MS |
5000 |
Flush interval in milliseconds for buffered documents |
spring.opencrawling.output.iceberg.consumer-group |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_CONSUMER_GROUP |
opencrawling-iceberg-group |
Kafka consumer group name for IcebergStoreWriterConsumer
|
spring.opencrawling.output.iceberg.consumer-concurrency |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_CONSUMER_CONCURRENCY |
3 |
Number of concurrent Kafka listener threads |
spring.opencrawling.output.iceberg.io-impl |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_IO_IMPL |
"" |
Custom FileIO class (automatically defaults to org.apache.iceberg.aws.s3.S3FileIO when warehouse starts with s3://) |
spring.opencrawling.output.iceberg.s3.endpoint |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_S3_ENDPOINT |
"" |
S3-compatible service endpoint URL (e.g., http://minio:9000) |
spring.opencrawling.output.iceberg.s3.path-style-access |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_S3_PATH_STYLE_ACCESS |
true |
Enable path-style access for MinIO and custom S3 implementations |
spring.opencrawling.output.iceberg.s3.access-key-id |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_S3_ACCESS_KEY_ID |
"" |
AWS / S3 access key ID |
spring.opencrawling.output.iceberg.s3.secret-access-key |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_S3_SECRET_ACCESS_KEY |
"" |
AWS / S3 secret access key |
spring.opencrawling.output.iceberg.s3.region |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_S3_REGION |
us-east-1 |
AWS / S3 region identifier |
spring.opencrawling.output.iceberg.entity-extraction.enabled |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_ENTITY_EXTRACTION_ENABLED |
true |
Populate extracted_entities map and evaluate dynamic column additions |
spring.opencrawling.output.iceberg.entity-extraction.include-custom-metadata |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_ENTITY_EXTRACTION_INCLUDE_CUSTOM_METADATA |
true |
Populate custom_metadata map with non-system repository attributes |
spring.opencrawling.output.iceberg.entity-extraction.include-ois-security |
SPRING_OPENCRAWLING_OUTPUT_ICEBERG_ENTITY_EXTRACTION_INCLUDE_OIS_SECURITY |
true |
Populate Zero-Trust ACL columns (security_owner, allowed_principals, denied_principals) |
OpenCrawling provides a turnkey decoupled Docker Compose configuration specifically tailored for Iceberg lakehouse streaming under oc-iceberg-output-connector/docker/docker-compose-decoupled-with-iceberg.yml:
services:
# S3 Object Storage Backend
minio:
image: milvusdb/minio:RELEASE.2024-12-18T13-15-44Z
container_name: minio-decoupled-iceberg
environment:
MINIO_ROOT_USER: admin
MINIO_ROOT_PASSWORD: password
ports:
- "9000:9000"
- "9001:9001"
# Apache Iceberg REST Catalog Fixture
iceberg-rest:
image: apache/iceberg-rest-fixture:latest
container_name: iceberg-rest-decoupled
ports:
- "8181:8181"
environment:
- AWS_ACCESS_KEY_ID=admin
- AWS_SECRET_ACCESS_KEY=password
- AWS_REGION=us-east-1
- CATALOG_WAREHOUSE=s3://warehouse/
- CATALOG_IO__IMPL=org.apache.iceberg.aws.s3.S3FileIO
- CATALOG_S3_ENDPOINT=http://minio:9000
- CATALOG_S3_PATH__STYLE__ACCESS=true
depends_on:
minio:
condition: service_healthy
# Decoupled Writer Consumer writing to Apache Iceberg
oc-writer-consumer:
image: ghcr.io/opencrawling/oc-writer-consumer:latest
container_name: oc-writer-service-iceberg
environment:
- SPRING_KAFKA_BOOTSTRAP_SERVERS=kafka:9094
- OPENCRAWLING_CONSUMER_WRITER_ENABLED=true
- SPRING_OPENCRAWLING_OUTPUT_TYPE=iceberg
- SPRING_OPENCRAWLING_OUTPUT_ICEBERG_CATALOG_TYPE=rest
- SPRING_OPENCRAWLING_OUTPUT_ICEBERG_CATALOG_NAME=opencrawling_lakehouse
- SPRING_OPENCRAWLING_OUTPUT_ICEBERG_CATALOG_URI=http://iceberg-rest:8181
- SPRING_OPENCRAWLING_OUTPUT_ICEBERG_WAREHOUSE=s3://warehouse
- SPRING_OPENCRAWLING_OUTPUT_ICEBERG_IO_IMPL=org.apache.iceberg.aws.s3.S3FileIO
- SPRING_OPENCRAWLING_OUTPUT_ICEBERG_NAMESPACE=enterprise_data
- SPRING_OPENCRAWLING_OUTPUT_ICEBERG_TABLE_NAME=documents
- SPRING_OPENCRAWLING_OUTPUT_ICEBERG_S3_ENDPOINT=http://minio:9000
- SPRING_OPENCRAWLING_OUTPUT_ICEBERG_S3_ACCESS_KEY_ID=admin
- SPRING_OPENCRAWLING_OUTPUT_ICEBERG_S3_SECRET_ACCESS_KEY=password
- SPRING_OPENCRAWLING_OUTPUT_ICEBERG_AUTO_SCHEMA_EVOLUTION=trueThe Apache Iceberg Output Connector is fully integrated into the OpenCrawling Web Admin UI (http://localhost:3000):
- Navigate to Connectors → Create Connector.
- Select Output Connector and choose "Apache Iceberg Lakehouse" (
org.opencrawling.iceberg.output.IcebergOutputConnector) from the dropdown. - Configure the lakehouse parameters directly via dedicated form controls:
-
Catalog Type:
REST,HIVE,HADOOP,GLUE,JDBC, orIN_MEMORY. -
Catalog URI: Endpoint of the catalog (e.g.
http://iceberg-rest:8181). -
Catalog Name: Target catalog name (e.g.
opencrawling_lakehouse). -
Warehouse Location: Storage URI (e.g.
s3://warehouse). -
Target Namespace & Table: E.g.
enterprise_data.documents. -
Table Strategy:
CREATE_IF_NOT_EXISTS,FAIL_IF_NOT_EXISTS, orRECREATE. -
Partition Strategy:
DAY,MONTH,YEAR,MIME_TYPE, orNONE. - Write Format & Compression: Parquet with Snappy, Gzip, or ZSTD compression.
- Schema Evolution: Toggle automated column addition for dynamic document entities.
- S3 / Object Store Credentials: S3 endpoint, region, access key, secret key, and path-style access.
-
Catalog Type:
- Save and test connector connectivity.
OpenCrawling includes an automated integration test script (scripts/test-iceberg-output-connector.sh) that verifies catalog connectivity, table creation, Zero-Trust OIS ACL schema fields, and dynamic schema evolution:
# Make executable and run
chmod +x scripts/test-iceberg-output-connector.sh
./scripts/test-iceberg-output-connector.shWhat the script validates:
- Checks for
curl,jq, anddockerutilities. - Checks Iceberg REST Catalog (
http://localhost:8181/v1/config) and spins up the fixture if offline. - Automatically ensures the MinIO
warehousebucket exists. - Ensures the target namespace exists (
POST /v1/namespaces). - Provisions an Iceberg table with canonical fields and verifies Zero-Trust ACL columns (
security_allowed_read,security_denied_read). - Tests dynamic schema evolution by posting schema updates that add new structured entity fields (
entity_vendor,entity_amount) and confirms the table upgrades toschema-id: 1.
To test the entire multi-service ingestion pipeline (crawler → Kafka → ingestion consumer → writer consumer → Iceberg Parquet files on MinIO):
chmod +x scripts/test-iceberg-output-decoupled.sh
# Standard execution
./scripts/test-iceberg-output-decoupled.sh
# Local BuildKit acceleration with shared ~/.m2 cache mount
./scripts/test-iceberg-output-decoupled.sh --local
# Fast iteration skipping Docker builds and preserving containers on failure
./scripts/test-iceberg-output-decoupled.sh -s -kWhat the decoupled test validates:
- Builds and starts all microservices (
oc-crawler,oc-ingestion-consumer,oc-writer-consumer,iceberg-rest,minio,kafka,redis). - Creates a human-readable enterprise contract document containing metadata, entities, and Zero-Trust ACLs (
role:lakehouse-analytics,user:piergiorgio@apache.org,group:data-engineering, and explicit denyrole:external-guest). - Executes the crawl, streaming the document across Kafka to the Iceberg writer consumer.
- Polls the Iceberg REST Catalog until snapshot commits are confirmed.
- Verifies the generated Parquet files in MinIO storage (
mc ls local/warehouse/). - Confirms Zero-Trust OIS security columns are present in the table schema.
Verify Apache Iceberg snapshot lineage, point-in-time getIcebergDocumentAsOf time-travel (with Zero-Trust ACL revocation enforcement), queryIcebergEntities, getIcebergIngestionStats, IcebergSidecarStore (listSidecarStores, readOisSidecars), and IcebergMcpSearchBackend over MCP Streamable HTTP:
chmod +x scripts/test-mcp-iceberg.sh
./scripts/test-mcp-iceberg.sh --local-buildOnce persisted into Apache Iceberg, data can be queried across various modern analytical engines without data movement:
INSTALL iceberg;
LOAD iceberg;
SELECT document_id, title, mime_type, security_owner, allowed_principals, entity_vendor
FROM iceberg_scan('s3://warehouse/enterprise_data/documents',
endpoint='http://localhost:9000',
access_key_id='admin',
secret_access_key='password')
WHERE 'role:lakehouse-analytics' = ANY(allowed_principals);SELECT document_id, title, action, crawled_at
FROM iceberg.enterprise_data.documents
WHERE contains(allowed_principals, 'user:piergiorgio@apache.org')
AND NOT contains(denied_principals, 'role:external-guest');SELECT document_id, title, extracted_entities['vendor'] AS vendor, crawled_at
FROM enterprise_lakehouse.enterprise_data.documents
WHERE crawled_at >= current_date() - INTERVAL 30 DAYS;- Home
- User & Installation Guide
- Quick Start
- VM & Bare-Metal Distribution Installation
- Local Development & Fast Docker Builds
- User Guide
- Architecture
- Zero-Trust Security & ACL Management
- Security and Identity Management
- Document Authorization & OpenFGA
- Enterprise Multi-Agent Integration
- Zero-Trust Data Provider Agent & Active Ingestion
- Document Lifecycle Actions & Tombstones
- Apache Tika 4 Text Extraction
- Configuration Guide
- Native Job Scheduler
- Platform API (
/api/v1) - Platform API: Traceability
- Connectors Reference
- Alfresco Content Services (ACS) Connector
- Alfresco Process Services (APS) Connector
- Flowable Repository Connector
- Camunda Repository Connector
- OASIS CMIS Repository Connector
- Doxis Repository Connector
- M-Files Repository Connector
- SharePoint Online Repository Connector
- Google Drive Repository Connector
- Atlassian Confluence Repository Connector
- IBM FileNet P8 Repository Connector
- OpenText Documentum Repository Connector
- Zammad Repository Connector
- Hyland OpenArch Integration (Repository, Output & MCP)
- Hyland Content Innovation Cloud (CIC) Connector
- Relational Database (JDBC) Connector
- Web Repository Connector
- Apache StormCrawler Repository Connector
- gRPC Push Repository Connector
- Elasticsearch Output Connector
- Apache Solr Output Connector
- Luxir Output Connector
- Apache SeaTunnel Output Connector
- Apache Ozone Output Connector (Migration Mode)
- AWS S3 Output Connector (Migration Mode)
- Alfresco Content Services Output Connector (Migration Mode)
- Apache Iceberg Output Connector
- Doxis Output Connector
- Connector Execution Modes
- Model Context Protocol
- Apache Ozone Claim Check Store
- Lakehouse Audit Reporting & Ledger
- AI-Powered Observability (AIOps)
- Auto-Narrativization Copilot
- Maven Connector Archetypes
- Java Client SDK
- Python Client SDK
- OpenCrawling CLI
- OpenAPI Specification & Swagger UI
- gRPC Internal Transport
- Migration Guide (ManifoldCF)
- Integration Test — Decoupled Pipeline
- M-Files Integration Test
- SharePoint Online Integration Test
- Google Drive Integration Test
- Confluence Integration Test
- IBM FileNet Integration Test
- OpenText Documentum Integration Test
- Zammad Integration Test
- Hyland OpenArch Integration Test
- Hyland CIC Integration Test
- Web Crawler Test
- Elasticsearch Integration Test
- Local Docker Build Test
- Root CLI Runner Scripts Test
- Secure MCP Server Integration Test
- Narrativization Integration Test
- Reporting Integration Test
- AWS S3 Migration Integration Test
- Alfresco Migration Integration Test
- OpenFGA Authorization Integration Test
- CLI Decoupled Integration Test
- Java Client SDK Decoupled Test
- Apache Iceberg Decoupled Test
- Doxis Output Integration Test
- Platform API v1 Test Suite