Java Client SDK

OpenCrawling Java Client SDK (oc-java-client-sdk) ☕

The OpenCrawling Java SDK (oc-java-client-sdk) is a strongly typed, fluent Java client library designed to programmatically manage document ingestion jobs, connectors, system configurations, AIOps diagnostics, and Auto-Narrativization templates against the OpenCrawling Runtime REST APIs and Open Ingestion Standard (OIS) models.

---

🚀 Key Features

• Fluent Builder Client API: Clean, intuitive builder pattern for initialization with support for API keys, custom endpoints, timeouts, and transport configuration.
• Job Lifecycle Control: Programmatically create, list, get, start, pause, stop, and delete document ingestion jobs.
• Connector Administration: Dynamic registration and management for repository scanning, vector store output, and AI transformation connectors.
• Auto-Narrativization Copilot: Programmatically trigger narrative generation from schema field definitions using Ollama or OpenAI models.
• AIOps & Observability: Fetch automated Root Cause Analysis (RCA) reports, correlated OpenTelemetry spans, execution logs, and live throughput metrics.
• Spring Boot Auto-Configuration: Built-in Spring Boot support (@EnableOpenCrawling or auto-configured OpenCrawlingClient bean via opencrawling.client.* properties).
• Java 25 & Zero-Dependency Transport: Built with native java.net.http.HttpClient and Jackson Databind, requiring zero heavy third-party HTTP client frameworks.

---

📦 Installation

Maven

Add the oc-java-client-sdk dependency to your project's pom.xml:

<dependency>
    <groupId>org.opencrawling</groupId>
    <artifactId>oc-java-client-sdk</artifactId>
    <version>1.0.1</version>
</dependency>

Gradle

implementation 'org.opencrawling:oc-java-client-sdk:1.0.1'

---

💡 Code Examples

1. Initializing the SDK Client

import org.opencrawling.sdk.OpenCrawlingClient;
import java.time.Duration;

OpenCrawlingClient client = OpenCrawlingClient.builder()
        .baseUrl("http://localhost:8080")
        .apiKey("your-api-key")
        .connectTimeout(Duration.ofSeconds(10))
        .readTimeout(Duration.ofSeconds(30))
        .build();

2. Creating and Starting an Ingestion Job

import org.opencrawling.sdk.models.JobRequest;
import org.opencrawling.sdk.models.JobResponse;
import org.opencrawling.sdk.models.NarrativizationConfig;

// Create an ingestion job with Auto-Narrativization enabled
JobResponse job = client.jobs().create(
    JobRequest.builder()
        .name("Enterprise Documentation Crawler")
        .targetUrl("https://docs.example.com")
        .repositoryConnector("FileSystem_Local")
        .outputConnector("PGVector_Output")
        .transformationConnector("Ollama_Embedding_Default")
        .narrativization(NarrativizationConfig.builder()
            .enabled(true)
            .template("Document titled {{title}} with content: {{content}}")
            .build())
        .build()
);

System.out.println("Created Job ID: " + job.id());

// Trigger immediate job execution
client.jobs().start(job.id());

3. Connector Registration & Management

import org.opencrawling.sdk.models.ConnectorRequest;
import org.opencrawling.sdk.models.ConnectorResponse;

ConnectorResponse connector = client.connectors().create(
    ConnectorRequest.builder()
        .name("Custom_PGVector_Output")
        .description("Custom PGVector Output Store")
        .type("output")
        .className("org.opencrawling.vector.VectorOutputConnector")
        .maxConnections(20)
        .addConfiguration("pgVectorUrl", "jdbc:postgresql://localhost:5432/opencrawling")
        .build()
);

4. Auto-Narrativization Copilot Integration

import org.opencrawling.sdk.models.CopilotRequest;
import org.opencrawling.sdk.models.CopilotResponse;

CopilotResponse copilotResponse = client.narrativization().generateTemplate(
    CopilotRequest.builder()
        .connectorType("repository")
        .addField("title", "string", "Document Title")
        .addField("content", "string", "Extracted text content")
        .build()
);

System.out.println("Generated Mustache Template: " + copilotResponse.template());

5. AIOps Root Cause Analysis & Traces

import org.opencrawling.sdk.models.DiagnosticReport;
import org.opencrawling.sdk.models.JobTraceResponse;

// Run AI-powered Root Cause Analysis on a failing job
DiagnosticReport report = client.observability().diagnose("job-123");
System.out.println("Pipeline Health Status: " + report.status());
System.out.println("RCA Summary: " + report.summary());

// Query correlated OpenTelemetry spans
JobTraceResponse traces = client.observability().getTraces("job-123");
System.out.println("Total Spans: " + traces.totalSpans() + ", Duration: " + traces.totalDurationMillis() + "ms");

---

🍃 Spring Boot Starter Integration

When using Spring Boot, OpenCrawlingAutoConfiguration automatically detects and injects an OpenCrawlingClient bean into your Spring application context.

1. Configure Properties (application.yml)

opencrawling:
  client:
    base-url: http://localhost:8080
    api-key: your-api-key
    connect-timeout: 10s
    read-timeout: 30s

2. Inject OpenCrawlingClient

import org.opencrawling.sdk.OpenCrawlingClient;
import org.springframework.stereotype.Service;

@Service
public class DocumentIngestionService {

    private final OpenCrawlingClient client;

    public DocumentIngestionService(OpenCrawlingClient client) {
        this.client = client;
    }

    public void triggerJob(String jobId) {
        client.jobs().start(jobId);
    }
}

---

🛠️ Build & Publishing Strategy

Deploying oc-java-client-sdk to Maven Central is fully automated using GitHub Actions (.github/workflows/client-sdk-publish.yml).

• GPG Artifact Signing: Skipped by default (<gpg.skip>true</gpg.skip>) for fast local developer builds (mvn clean install).
• Maven Central Deployment: Automatically signs artifacts (-Dgpg.skip=false), compiles Javadoc and source JARs, and publishes directly to Sonatype Central Portal.