Skip to content

austinmao/clawpipe

Repository files navigation

ClawPipe

Config-driven pipeline orchestration engine for OpenClaw.

ClawPipe implements 20 of 21 agentic design patterns at production grade — the most comprehensive single-library implementation of the Gulli & Sauco pattern taxonomy. 844 tests, 84% coverage, 45 source modules.

21 Agentic Design Patterns — Scorecard

# Pattern Score Module
1 Prompt Chaining 3/3 runner.py — stage sequencing via depends_on DAG
2 Routing 2/3 routing.py — intent-based pipeline selection (CLI utility)
3 Parallelization 3/3 dag.py — wave dispatch via ThreadPoolExecutor
4 Reflection 3/3 reflection.py — produce-critique-revise with rubric evaluation
5 Tool Use 3/3 Gateway plugin + mcp_server.py (FastMCP, 5 tools)
6 Planning 2/3 factory.py — intent→template→config with path validation
7 Multi-Agent 3/3 stage.py — agent dispatch via gateway sessions_spawn
8 Memory 3/3 integrations/rag.py — retrieval with freshness validation
9 Learning 3/3 lessons.py — priming effectiveness + log mining
10 MCP 3/3 mcp_server.py — FastMCP server (STDIO + HTTP)
11 Goal Setting 3/3 goals.py — KPI tracking with webhook metrics + trends
12 Exception Handling 3/3 retry.py — 7 failure classes with classified retry
13 Human-in-the-Loop 3/3 approval.py — multi-option gates with timeout
14 RAG 3/3 integrations/rag.py — typed lessons + time-windowed retrieval
15 Inter-Agent (A2A) 3/3 protocol.py + agent_card.py — Agent Card + message validation
16 Resource-Aware 3/3 model_router.py — auto complexity→model tier
17 Reasoning 3/3 reasoning.py — ReAct loops (thought→action→observation)
18 Guardrails 3/3 guardrails.py — expression eval with block/warn/require_approval
19 Evaluation 3/3 quality.py + Opik spans — 5-dimension scoring + regression
20 Prioritization 3/3 priority.py — scored queue with starvation prevention
21 Exploration 3/3 exploration.py — discovery triggers + exploration templates

Total: 60/63 (95%) — See PATTERNS.md for detailed implementation guide per pattern.

Installation

pip install -e .

# With MCP server support
pip install -e ".[mcp]"

Quick Start

As an OpenClaw Gateway Tool

{"tool": "clawpipe", "action": "run", "config_path": "pipelines/website-build.yaml"}

CLI

clawpipe run --config pipeline.yaml --pipeline-id my-run-001 --json
clawpipe show --config pipeline.yaml --pipeline-id my-run-001 --json
clawpipe resume --config pipeline.yaml --pipeline-id my-run-001 --approved --json
clawpipe goals --config pipeline.yaml --json
clawpipe factory --intent "Build a 3-email welcome sequence" --json
clawpipe route --intent "Send campaign" --config-dir pipelines/ --json
clawpipe agent-card --json
clawpipe rubric-gen --config pipeline.yaml --stage generate-copy --json
clawpipe watch --json
clawpipe lessons --config pipeline.yaml --pipeline-id my-run-001 --json
clawpipe learning --effectiveness --pipeline-type website-build --json

MCP Server

# STDIO transport (for Claude Desktop, Cursor, etc.)
clawpipe-mcp

# HTTP transport
clawpipe-mcp --transport http --port 8741

Pipeline Config

pipeline:
  name: website-build
  version: "1.0"

defaults:
  timeout: 300
  max_retries: 2
  backoff: exponential
  reasoning_mode: standard

goals:
  - name: quality-target
    metric_ref: "quality_score.composite"
    target: ">= 0.8"

guardrails:
  - name: contact-limit
    check: "contact_count <= 10000"
    on_fail: block

stages:
  - name: generate-copy
    type: reflection
    command: scripts/generate-copy.sh
    reflection:
      rubric_path: rubrics/email-content.yaml
      max_iterations: 3
    reasoning_mode: react
    rag_sources: [mem0, lessons]

  - name: review
    type: approval
    depends_on: [generate-copy]
    approval_options:
      - value: approve
        label: "Approve"
      - value: revise
        label: "Send Back"

  - name: deploy
    type: agent_dispatch
    agent_id: deployment-agent
    command: scripts/deploy.sh  # fallback if gateway unavailable
    depends_on: [review]
    condition: "$review.approval.value == approve"

quality:
  weights: {confidence: 0.25, cost: 0.15, completeness: 0.25, feedback: 0.20, speed: 0.15}

paths:
  base: "memory/runs/${pipeline_id}"
  artifacts: "${base}/artifacts"

Envelope Protocol

Every pipeline run returns a structured envelope:

Status Meaning
ok Completed — includes quality_score, goal_results, audit_entry_path
needs_approval Paused — includes options, timeout, preview
failed Halted — resumable, includes failure_class
cancelled Operator rejected approval gate

Architecture

Pipeline YAML → load_config() → PipelineRunner.run()
                                    │
                                    ├── Pre-run: guardrails → Agent Card publish
                                    ├── Per-stage: RAG retrieval → model routing → execute
                                    │     ├── type: command → subprocess
                                    │     ├── type: reflection → rubric loop
                                    │     ├── type: react → thought→action→observe
                                    │     ├── type: approval → needs_approval envelope
                                    │     └── type: agent_dispatch → gateway sessions_spawn
                                    ├── Post-run: quality scoring → goals → learning → audit
                                    └── Envelope response (JSON)

OpenClaw Gateway Plugin

ClawPipe ships with a gateway plugin that registers the clawpipe tool directly in the OpenClaw gateway. The plugin delegates all actions to the Python CLI via child_process.

Installation

  1. Copy the extensions/clawpipe/ directory into your OpenClaw workspace:
cp -r extensions/clawpipe/ ~/.openclaw/extensions/clawpipe/
  1. Register the plugin in your ~/.openclaw/openclaw.json:
{
  "extensions": {
    "clawpipe": {
      "repoRoot": "/path/to/your/workspace",
      "timeoutMs": 60000
    }
  }
}
  1. Restart the gateway:
openclaw gateway restart

Plugin Configuration

Option Type Default Description
repoRoot string $OPENCLAW_WORKSPACE or cwd Workspace root containing the clawpipe package
pythonBin string <repoRoot>/clawpipe/.venv/bin/python Path to Python binary
timeoutMs integer 60000 CLI execution timeout in milliseconds

Supported Actions

run, resume, show, lessons, watch, goals, factory, trigger, rubric-gen, heartbeat

See the openclaw.plugin.json for the full config schema.

ClawSuite

This package is part of ClawSuite — the OpenClaw agent infrastructure toolkit.

Package Description Repo
ClawPipe Config-driven pipeline orchestration austinmao/clawpipe
ClawSpec Contract-first testing for skills & agents austinmao/clawspec
ClawWrap Outbound policy & conformance engine austinmao/clawwrap
ClawAgentSkill Skill discovery, scanning & adoption austinmao/clawagentskill
ClawScaffold Agent/skill scaffold interviews austinmao/clawscaffold
ClawInterview Pipeline interview compilation & execution (coming soon)

All packages include OpenClaw gateway plugins for autonomous agent access.

Development

pip install -e ".[dev]"
pytest tests/ -v --cov=clawpipe   # 844 tests, 84% coverage

See CONTRIBUTING.md for TDD workflow and extension points.

License

Apache 2.0 — see LICENSE.

About

Config-driven pipeline orchestration engine for OpenClaw — reflection loops, quality scoring, goal tracking, approval gates

Resources

License

Contributing

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors