Skip to content

Alfresco Content Services Repository Connector

Jump to bottom
Piergiorgio Lucidi edited this page Oct 2, 2026 · 3 revisions

Alfresco Content Services (ACS) Repository Connector

Alfresco Java Spring Boot OIS Compliance Claim Check

Module: oc-alfresco-repository-connector
Connector Class: org.opencrawling.alfresco.AlfrescoRepositoryConnector
Security Mapper: org.opencrawling.alfresco.AlfrescoSecurityMapper
Crawl Mode Enum: org.opencrawling.alfresco.AlfrescoCrawlMode
Spring Bean: @Component (activated when spring.opencrawling.connector.type=alfresco or CONNECTOR_TYPE=alfresco)

📌 Overview

The Alfresco Content Services (ACS) Repository Connector provides OpenCrawling with native, high-performance connectivity to Alfresco Community Edition and Alfresco Enterprise Edition (7.x, 23.x, and beyond) using the native Alfresco Core REST API v1 and Alfresco Search REST API v1.

The connector supports dual-mode enterprise ingestion:

  1. Hierarchical Folder Tree Traversal: Recursively scans repository folders with support for folder exclusions, flat vs. recursive subfolder crawling, and automatic site document library scoping (/Sites/{siteId}/documentLibrary).
  2. Targeted Search API Execution: Executes declarative queries via the Alfresco Search API using AFTS (Alfresco Full Text Search), CMIS SQL, or Lucene directly against Solr/Elasticsearch/OpenSearch backends without crawling unrelated folders.

Built on Java 25 Virtual Threads (StructuredTaskScope), the connector extracts fine-grained Access Control Lists (ACLs) into zero-trust Open Ingestion Standard (OIS) permission rules, enriches documents with Alfresco custom aspects and content model properties, and supports size caps and metadata-only indexing.

graph TD
    subgraph ACS["Alfresco Content Services (ACS 7.x / 23.x)"]
        CoreAPI["Core REST API v1 (:8080/alfresco/api/-default-/public/alfresco/versions/1)"]
        SearchAPI["Search REST API v1 (:8080/alfresco/api/-default-/public/search/versions/1/search)"]
        Nodes["Nodes, Folders & Aspects"]
        Perms["Locally Set & Inherited Permissions (ACEs)"]
        Content["Binary Document Content Streams"]
        
        CoreAPI --> Nodes & Perms & Content
        SearchAPI --> Nodes & Perms
    end

    subgraph OpenCrawling["OpenCrawling Ingestion Runtime (Java 25)"]
        Connector["AlfrescoRepositoryConnector (oc-alfresco-repository-connector)"]
        VThreads["Virtual Threads (StructuredTaskScope)"]
        SecMap["AlfrescoSecurityMapper (ACE -> OIS SecurityConfig)"]
        ClaimCheck["Binary Content Stream (Claim Check Store)"]
        Tika["Apache Tika Text Extraction"]
        
        CoreAPI <-->|Folder Navigation & Content| Connector
        SearchAPI <-->|AFTS / CMIS SQL / Lucene POST| Connector
        Connector --> VThreads
        VThreads --> SecMap
        VThreads --> ClaimCheck
        ClaimCheck --> Tika
        
        OIS["OIS RepositoryDocument (Zero-Trust ACLs & Metadata)"]
        VThreads --> OIS
        Tika --> OIS
        
        OIS --> Kafka["Kafka Topic (opencrawling-raw-documents)"]
        Kafka --> Ingestion["oc-ingestion-consumer (Chunking & Embeddings)"]
        Ingestion --> Sinks[("Vector Stores: PGVector / Vespa / Milvus / Qdrant")]
    end
Loading

🚀 Key Capabilities

1. Dual Crawl Modes

  • Folder Traversal Mode (crawlMode: folder):

    • Traverses the Alfresco node tree starting from the repository root (-root-), a specific folder path (e.g. /Company Home/Sites/finance/documentLibrary), a Node UUID, or a site identifier (siteId).
    • Subfolder Recursion Toggle (includeSubfolders): Enables or disables deep folder recursion. When disabled, only documents residing directly in the target root folder are crawled.
    • Folder Exclusions (excludedFolders): Skips unwanted folders (e.g. Data Dictionary, /Sites/trash, System) by exact name or relative path.

  • Search API Query Mode (crawlMode: query):

    • Leverages the native POST /public/search/versions/1/search endpoint to query Alfresco via AFTS, CMIS SQL, or Lucene.
    • Ideal for targeted ingestion (e.g. only indexing contracts modified within the last 30 days, or documents carrying specific aspects).
    • Automatically handles pagination (skipCount and maxItems).

  • Dynamic Query Switching: When creating a crawl Job in the Admin UI or via API, if the job's scanPath starts with TYPE:, PATH:, ASPECT:, TAG:, SELECT , afts:, cmis:, or lucene:, the connector automatically executes in query mode regardless of the default connector setting.

2. Zero-Trust Security & ACL Mapping

The connector requests include=properties,aspectNames,permissions from Alfresco and maps all Access Control Entries (ACEs) via AlfrescoSecurityMapper:

  • Inherited & Locally Set: Evaluates both locally assigned and inherited permissions according to isInheritanceEnabled.
  • Public Access Detection: Recognizes GROUP_EVERYONE, guest, and ROLE_ANONYMOUS as public readers.
  • Groups vs. Users: Authorities prefixed with GROUP_ or ROLE_ are mapped to group identity rules, while individual accounts are mapped to user rules.
  • Collocated Metadata: Stores alfresco_node_id, alfresco_node_type, alfresco_identity_users, alfresco_identity_groups, and alfresco_aspects directly in the document metadata for query-time join and filtering.

See the dedicated Zero-Trust Security & ACL Management guide for architecture details on collocated query pushdowns and distributed vector pre-filtering.

3. Payload and Content Controls

  • includeContentStream: When false, OpenCrawling performs a lightweight metadata-only ingestion, indexing Alfresco aspects, titles, and tags without downloading multi-gigabyte binary files.
  • maxContentSizeBytes: Prevents out-of-memory errors by skipping binary payload downloads for files exceeding the configured threshold (default: 50 MB / 52428800 bytes).
  • mimeTypeFilter: Whitelists specific content types (e.g. application/pdf, application/vnd.openxmlformats-officedocument.*, text/plain).

⚙️ Configuration Properties

Property key Spring Property Key Environment Variable Default Description
url spring.opencrawling.connector.alfresco.url ALFRESCO_URL http://localhost:8080/alfresco/api/-default-/public/alfresco/versions/1 Alfresco REST API v1 Base URL
username spring.opencrawling.connector.alfresco.username ALFRESCO_USERNAME admin Alfresco username
password spring.opencrawling.connector.alfresco.password ALFRESCO_PASSWORD admin Alfresco password (Basic Auth)
batchSize spring.opencrawling.connector.alfresco.batch-size ALFRESCO_BATCH_SIZE 100 Number of items per API request page
crawlMode spring.opencrawling.connector.alfresco.crawl-mode ALFRESCO_CRAWL_MODE folder Ingestion mode: folder or query
queryLanguage spring.opencrawling.connector.alfresco.query-language ALFRESCO_QUERY_LANGUAGE afts Query language: afts, cmis, or lucene
searchQuery spring.opencrawling.connector.alfresco.search-query ALFRESCO_SEARCH_QUERY TYPE:'cm:content' Default query for query crawl mode
rootFolderPath spring.opencrawling.connector.alfresco.root-folder-path ALFRESCO_ROOT_FOLDER_PATH / Default folder path to scan
rootFolderId spring.opencrawling.connector.alfresco.root-folder-id ALFRESCO_ROOT_FOLDER_ID "" Optional starting node UUID
siteId spring.opencrawling.connector.alfresco.site-id ALFRESCO_SITE_ID "" Optional Site ID (auto-targets documentLibrary)
includeSubfolders spring.opencrawling.connector.alfresco.include-subfolders ALFRESCO_INCLUDE_SUBFOLDERS true Recursively traverse child folders
excludedFolders spring.opencrawling.connector.alfresco.excluded-folders ALFRESCO_EXCLUDED_FOLDERS Data Dictionary Comma-separated folder names/paths to skip
includeAcls spring.opencrawling.connector.alfresco.include-acls ALFRESCO_INCLUDE_ACLS true Extract ACE permissions and zero-trust ACLs
includeContentStream spring.opencrawling.connector.alfresco.include-content-stream ALFRESCO_INCLUDE_CONTENT_STREAM true Download binary payload for text extraction
maxContentSizeBytes spring.opencrawling.connector.alfresco.max-content-size-bytes ALFRESCO_MAX_CONTENT_SIZE_BYTES 52428800 Max payload size in bytes (50 MB)
mimeTypeFilter spring.opencrawling.connector.alfresco.mimetype-filter ALFRESCO_MIMETYPE_FILTER "" Comma-separated MIME whitelist (e.g. application/pdf)
timeoutSeconds spring.opencrawling.connector.alfresco.timeout-seconds ALFRESCO_TIMEOUT_SECONDS 30 HTTP request timeout

📋 Scan Path Formats in Job Creation

When defining an OpenCrawling Job targeting an Alfresco connector, the Scan Path input accepts multiple formats:

Format Example Ingestion Behavior
Root -root- or / Crawls from repository root downwards
Path /Company Home/Sites/legal/documentLibrary Crawls from the specified folder path
Node UUID 4b3e69f8-d421-4f76-bc39-a72dfa5f4c10 Crawls the specific node or folder
AFTS Query TYPE:'cm:content' AND PATH:'/app:company_home/st:sites/cm:legal//*' Executes search query via Alfresco Search API
CMIS Query SELECT * FROM cmis:document WHERE cmis:name LIKE 'Invoice%' Executes CMIS SQL query via Alfresco Search API
Prefixed Query afts:TAG:'confidential' AND @cm:modified:[NOW/DAY-7DAYS TO NOW/DAY] Explicitly sets search query language

🔍 Metadata and Collocated Attributes Extracted

OIS Metadata Field Source / Description
name File or document name (entry.name)
nodeType Alfresco content type (e.g. cm:content)
mimeType MIME type (e.g. application/pdf)
sizeInBytes Binary content length
alfresco_node_id Alfresco node UUID
alfresco_parent_id Parent folder node UUID
alfresco_node_type Alfresco content model type
alfresco_aspects Attached Alfresco aspects (e.g. cm:titled, cm:versionable)
alfresco_identity_users Comma-separated authorized reader users
alfresco_identity_groups Comma-separated authorized reader groups
alfresco_is_inherited Boolean indicating whether permissions are inherited
<custom_property> Any property present in entry.properties (e.g. cm:title, cm:description)

🗺️ Documentation

🧪 Testing

🤝 Community

Clone this wiki locally