OpenCrawling CLI

OpenCrawling CLI (oc-cli / oc) 💻

Caution

Work in progress, this feature is not yet available.

The OpenCrawling CLI (oc) is a terminal-native tool designed for DevSecOps automation, CI/CD pipeline integration, connector health inspection, and command-line data ingestion management.

Built with Java 25, Picocli, and GraalVM Native Image, the oc command line tool delivers instant sub-10ms execution times and consumes the official oc-java-client-sdk.

---

🎯 Primary Use Cases

1. DevSecOps & CI/CD Automation: Trigger ingestion jobs, wait for completion, and assert status inside GitHub Actions, GitLab CI, or Kubernetes cronjobs.
2. Terminal Ingestion Operations: Start, stop, list, and monitor crawling jobs directly from the terminal without opening the Admin UI.
3. Connector Scaffolding: Fast wizard for generating custom repository, output, and transformation connectors using Maven Archetypes.
4. Auto-Narrativization Copilot: Generate natural language Mustache templates from schema definitions via terminal execution.
5. AIOps & Diagnostics: Query correlated OpenTelemetry spans and AI-generated Root Cause Analysis (RCA) reports during system outages.

---

💻 Command Syntax & Reference

oc [options] [command] [command-options]

Commands

Command	Sub-commands	Description
oc job	list, start, status, stop, delete	Manage data crawling & ingestion jobs
oc connector	list, check, test	Inspect & test repository / output connectors
oc archetype	init, generate	Scaffold custom connectors via Maven archetypes
oc copilot	narrativize, mock	Execute Auto-Narrativization Copilot tasks
oc aiops	rca, spans, health	Query OpenTelemetry traces and AI Root Cause Analysis
oc config	set, get, context	Manage CLI contexts and active runtime endpoints

---

⚡ Examples

1. Start a New Ingestion Job

oc job start \
  --name "SharePoint Legal Docs" \
  --connector filesystem \
  --path /data/legal \
  --wait

2. Check Vector Connector Health

oc connector check --name "Qdrant_Enterprise_KB"

3. Generate Schema Narrative Template

oc copilot narrativize \
  --schema ./schemas/cmis-document.json \
  --engine ollama \
  --model llama3.2

4. Fetch AIOps Root Cause Analysis Report

oc aiops rca --job-id 550e8400-e29b-41d4-a716-446655440000

---

📦 Installation & Packaging Options

• Homebrew (macOS / Linux):

  brew install opencrawling/tap/oc

• JBang:

  jbang oc@opencrawling job list

• Docker:

  docker run --rm -it ghcr.io/opencrawling/oc-cli:latest job list