English | 中文

DevAgent Studio is an open-source multi-agent workbench for software project understanding and engineering governance. It is built with FastAPI, LangGraph, and React to help teams inspect project structure, review code risks, preserve project knowledge, govern AI and tool calls, and run traceable workflows.

It is deliberately not a code-writing IDE. Its focus is making software delivery work easier to understand, audit, evaluate, and improve.

• Multi-agent workflows, not isolated prompts: Planner, Project Analyzer, Code Reviewer, RAG Processor, Supervisor, and Reporter are composed through LangGraph.
• Governed runtime: Harness Runtime provides task context, event timelines, artifacts, review state, persistence, deterministic policy checks, and resume support.
• Visual workflows that execute: a drag-and-drop canvas is compiled into executable LangGraph workflows instead of serving as a static diagram.
• Observable LLM operations: prompt versions, model configuration, traces, token and cost data, fallback records, and A/B comparison are available from the UI.
• Extensible Skills with guardrails: Skills are versioned, permission-scoped, testable, dependency-aware, and usable from both the console and a workflow.
• Plugin Marketplace: install resource packs from built-in catalogs, local paths, URLs, GitHub-style sources, or an external SKILL.md file.
• Safe third-party code execution: Code Skills can run in a constrained Docker sandbox with no network, read-only mounts, resource limits, and audit logs.
• Governed RAG that can be evaluated: project knowledge now supports incremental indexing, document versions, ACL filtering, hybrid retrieval, optional LLM rerank, and editable chunk-level Gold Sets.
• Switchable Colorful and Clear UI: the React workbench can switch between a colorful theme with yellow/green/blue/pink environmental light pillars and a clear SVG-refraction liquid-glass theme with pointer ripples, hover lift, and connected mode transitions. The selected theme is persisted locally.

The screenshots below show the Clear theme with per-surface SVG displacement maps, refracted grid detail, reflective rims, layered shadows, and liquid button interactions. Use the header switch to move between the Colorful and Clear themes at any time.

Run workbench / 运行	Visual workflow / 编排

Reports / 报告	Interactive chat / 追问

Task history / 历史	LLM governance / LLM

MCP console / MCP	Skills console / Skills

Plugin Marketplace / Market	Benchmark dashboard / Bench

User / React workbench
        |
        v
FastAPI APIs ---- Marketplace ---- Skills Console ---- MCP Console
        |
        v
Harness Runtime
  context | policy | events | artifacts | human review | persistence
        |
        +-----------------------+
        |                       |
        v                       v
LangGraph workflows          Skill Runtime
planner / reviewer /         prompt skills / code skills /
RAG / supervisor / reporter  dependency and permission checks
        |                       |
        +-----------+-----------+
                    v
      LLM / MCP / RAG / SQLite / pgvector / Docker sandbox

Task request
  -> FastAPI creates a task
  -> Harness Runtime creates context and emits events
  -> LangGraph invokes agents, Skills, LLMs, RAG, or MCP tools
  -> policy and human-review checks gate sensitive operations
  -> traces, artifacts, logs, and state are persisted
  -> Reporter produces a governance report

The RAG layer is designed as a governed project knowledge base rather than a simple chunk search demo.

This turns RAG from "can retrieve something" into "can be versioned, permissioned, measured, and improved."

Conversation history is not written directly into the knowledge base. Durable memory follows a governed pipeline:

User message
  -> LLM Memory Extractor (or deterministic rule fallback)
  -> candidate memory + sensitive-content filter
  -> quality score + conflict detection + retention policy
  -> explicit user approval
  -> scoped RAG ingestion
  -> retrieval, review, expiry, or deletion audit trail

• LLM-first extraction with fallback: memory_extractor emits structured candidates for durable preferences, project facts, and team policies. Without an LLM, explicit-preference rules keep the feature usable offline.
• Confirmation before ingestion: candidates never enter RAG until the user confirms them. Rejection and deletion prevent unwanted persistence.
• Conflict-aware updates: confirming a conflicting value marks the prior record as superseded, while preserving history.
• Quality and retention: each record has a quality score and reasons. Stable project/team rules and language/security preferences persist; general preferences use review_90d and become expired instead of being silently deleted.
• Memory decay, not blind accumulation: stale or low-durability memories leave the retrieval path after their review window. They remain auditable as expired until explicitly refreshed or deleted, preventing old preferences and obsolete decisions from continuously biasing agents.
• Scoped access boundaries: user, project, and team memories are separated. The local API accepts X-DevAgent-Actor and X-DevAgent-Role; project writes require editor/admin, while team confirmation and deletion require admin.

DEV_AGENT_MEMORY_EXTRACTOR=llm
DEV_AGENT_LLM_MODEL_MEMORY_EXTRACTOR=gpt-4o-mini

Set DEV_AGENT_MEMORY_EXTRACTOR=rule to disable LLM extraction. Its traces appear in the LLM Console under memory_extractor.

A Skill is a reusable capability such as code review, RAG processing, learning coaching, security scanning, or workflow execution. A Skill can be tested in the Skills console or added to a visual workflow.

An external SKILL.md is imported as a Prompt Skill when there is no plugin.json. The system reads its instruction text and uses it as an LLM prompt. It never executes third-party code.

A Code Skill contains an executable entry point, for example:

plugin/
  plugin.json
  skills/
    security_scan.py

{
  "code": "security.scan",
  "execution_type": "python",
  "entrypoint": "skills/security_scan.py:run",
  "permissions": ["project-read"]
}

Code Skills can use a Docker sandbox. The runtime starts a temporary container and removes it when execution ends. The sandbox applies:

• --network none: no outbound network access.
• --read-only: immutable container root filesystem.
• read-only mount for the Skill package.
• memory, CPU, PID, and execution-time limits.
• dropped Linux capabilities and no-new-privileges.
• invocation result and failure logging.

This makes third-party extensions practical without treating them as trusted local code. Docker isolation is a defense layer, not a substitute for reviewing plugin source and permissions.

The Marketplace installs and tracks resource packages. Supported package types include skill_pack, rag_pack, mcp_pack, prompt_pack, workflow_pack, and benchmark_pack.

Supported sources:

• Built-in catalog packages.
• Local folders or a local plugin.json.
• URL and GitHub-style package sources.
• External SKILL.md files, automatically converted to a safe Prompt Skill.

After installation, the UI shows installed resources, source and trust details, available Skills, approval actions, test calls, workflow insertion, and uninstall status.

approval key = skill_code + agent_code

The two common execution identities are:

Approving skill_console does not approve workflow_runner. A Skill must be explicitly approved for the context in which it will run.

• Python 3.11+ (Python 3.13 is supported by the current project setup)
• Node.js 18+
• Docker Desktop, optional for pgvector and Docker Code Skill sandboxing

git clone https://github.com/biheto/DevAgent-Studio.git
cd DevAgent-Studio

python -m venv .venv
.\.venv\Scripts\activate
pip install -e ".[llm,vector,dev]"

cd web
npm install
npm run build
cd ..

copy .env.example .env

.\.venv\Scripts\python.exe -m uvicorn app.main:app --host 127.0.0.1 --port 8100

Open http://127.0.0.1:8100/ for the workbench and http://127.0.0.1:8100/docs for the API documentation.

Run pgvector:

docker compose -f docker-compose.pgvector.yml up -d

Configure Docker Code Skill sandboxing in .env:

DEV_AGENT_SKILL_SANDBOX=docker
DEV_AGENT_SKILL_SANDBOX_IMAGE=python:3.13-slim
DEV_AGENT_SKILL_SANDBOX_MEMORY=256m
DEV_AGENT_SKILL_SANDBOX_CPUS=0.5
DEV_AGENT_SKILL_SANDBOX_PIDS_LIMIT=64
DEV_AGENT_SKILL_SANDBOX_FALLBACK=false

subprocess is the default sandbox mode for local development. docker requires Docker Desktop to be running. The sandbox status can be checked at GET /api/v1/skills/sandbox/status.

The application works with deterministic fallback responses when no key is configured. Set an API key for real LLM calls:

OPENAI_API_KEY=your_api_key
OPENAI_BASE_URL=
DEV_AGENT_LLM_MODEL=gpt-4o-mini

# Optional per-agent overrides
DEV_AGENT_LLM_MODEL_PLANNER=gpt-4o-mini
DEV_AGENT_LLM_MODEL_REPORTER=gpt-4o-mini
DEV_AGENT_LLM_MODEL_CODE_REVIEWER=gpt-4o-mini

# Backend compilation
.\.venv\Scripts\python.exe -m compileall -q app

# Frontend production build
cd web
npm run build
cd ..

# Governed long-term memory tests
.\.venv\Scripts\python.exe -m unittest tests.test_memory_store -v

# RAG governance tests
.\.venv\Scripts\python.exe -m unittest tests.test_rag_governance -v

# Verify the Skill sandbox configuration
.\.venv\Scripts\python.exe -c "from app.skills.sandbox import python_skill_sandbox_status; print(python_skill_sandbox_status())"

DevAgent Studio/
  app/
    agents/              # Project, review, RAG, learning, and report logic
    api/                 # FastAPI route modules
    graphs/              # LangGraph graphs and visual workflow compiler
    harness/             # Context, events, policy, artifacts, review/resume runtime
    marketplace/         # Package preview, installer, trust, and SKILL.md compatibility
    persistence/         # Task, governance, RAG, and trace persistence
    providers/           # LLM, MCP, and RAG provider interfaces
    skills/              # Registry, contracts, versions, dependencies, sandbox runtime
    benchmark_runner.py  # LLM/RAG/Workflow/MCP/collaboration benchmarks
  web/                   # React workbench
  scripts/               # MCP launchers and test servers
  docs/                  # Architecture and implementation notes
  examples/              # API request examples
  docker-compose.pgvector.yml

• Implementation timeline
• Workflow production notes

• Add a richer UI for Docker sandbox health and test invocation.
• Add signed external plugin publishing examples and contributor tooling.
• Expand API, workflow compiler, runtime-state, MCP contract, LLM fallback, and benchmark integration test coverage.
• Add conditional branch, parallel node, and richer input/output mapping UX for workflows.

This project is licensed under the MIT License.

Copyright (c) 2026 biheto. When redistributing the project, preserve the original copyright notice and license text.

DevAgent Studio by biheto
https://github.com/biheto/DevAgent-Studio

DevAgent Studio 是一个面向软件项目理解与研发治理的开源多 Agent 工作台。项目基于 FastAPI、LangGraph 和 React 构建，用于帮助团队理解项目结构、识别代码风险、沉淀项目知识、治理 LLM 与工具调用，并执行可追踪的研发工作流。

它不是代码编写 IDE，核心目标是让研发过程更容易理解、审计、评估和持续改进。

• 多 Agent 协作而非单次 Prompt：通过 LangGraph 编排 Planner、项目分析、代码审查、RAG、监督和报告节点。
• Harness Runtime 运行时治理：统一任务上下文、事件时间线、产物、策略、人工审核、持久化与恢复执行。
• 真正可执行的可视化 Workflow：前端拖拽画布会编译成 LangGraph 工作流执行，而不只是展示图。
• LLM 可观测与可治理：可查看 Prompt 版本、调用 Trace、token/cost、fallback、A/B Test，以及按 Agent 配置模型。
• 安全可治理的 Skill 插件体系：Skill 有契约校验、权限分级、严格审批、版本快照、依赖检测、测试用例和执行日志。
• 插件市场与外部兼容：支持内置包、本地路径、URL、GitHub 风格来源，以及外部 SKILL.md 自动转换。
• Docker 代码型 Skill 沙箱：第三方代码可以在禁网、只读、限时限资源的临时容器中运行。
• 受控长期记忆：对话先经 LLM/规则提取为候选记忆，再通过质量评分、冲突检测、人工确认、生命周期策略和 scope 权限边界沉淀到 RAG。
• 遗忘衰减而非无限累积：稳定规则长期保留；普通偏好进入 90 天复核，到期后标记为 expired 并退出检索，但保留审计记录，可由用户刷新、更新或删除，避免陈旧偏好和过时决策持续干扰 Agent。
• RAG 治理与可评测：知识库支持增量索引、文档版本化、ACL 权限过滤、BM25/向量混合检索、可选 LLM Rerank 和 Chunk Gold Set。
• 缤纷 / 清透双主题 UI：顶部可在带黄绿蓝粉彩色环境光柱的“缤纷”主题与 SVG 折射水玻璃“清透”主题之间切换；清透模式支持边缘折射、悬浮阴影、点击涟漪和模式按钮液态连贯切换，并在浏览器本地保存用户选择。

RAG 不再只是“切片后能检索”，而是按项目知识库治理的方式设计：

这让知识库从“能查”升级为“可版本化、可授权、可评测、可持续优化”。

Skill 是可复用能力，例如代码审查、RAG 加工、学习陪练、安全扫描或 Workflow 执行。它可以在 Skills 页面单独测试，也可以加入可视化 Workflow。

已实现的治理能力：

• 契约校验：安装和预览时校验 input_schema、output_schema、permissions、execution_type，避免格式错误的包进入运行时。
• 权限风险分级：使用 safe、project-read、llm、workflow-write、network、filesystem 标识访问能力和风险级别。
• 严格审批：审批键为 skill_code + agent_code。手动测试和工作流执行分别审批，互不放行。
• 版本与回滚：安装升级会保留版本快照，可对比和回滚。
• 依赖声明：Skill 可声明依赖的 MCP 工具、RAG collection、Prompt 版本和 LLM 模型，运行前会检查缺失项。
• 测试用例：插件包可携带测试用例，安装后可一键运行并记录结果。
• 可信来源：记录来源 URL、作者、manifest SHA-256 签名校验、安装次数和本地校验状态。
• Workflow 输入输出映射：前序节点输出可映射到 Skill 节点输入，让 Skill 参与复杂工作流。

如果外部来源没有 plugin.json，但包含 SKILL.md，系统会将其识别为 Prompt Skill：只读取其中的指令文本并交给 LLM，不执行第三方代码。

代码型 Skill 则带有可执行入口，例如 Python 文件。它能力更强，但必须通过权限审批和沙箱限制后执行。

当 .env 中设置 DEV_AGENT_SKILL_SANDBOX=docker 后，代码型 Skill 会在临时 Docker 容器中执行，并使用以下限制：

• 禁止网络访问。
• 容器根文件系统只读，Skill 包只读挂载。
• 限制内存、CPU、进程数和执行超时。
• 移除 Linux capabilities，禁止提升权限。
• 执行结束自动删除容器，同时保留调用结果和失败日志。

Docker 沙箱是隔离层，不代表插件天然可信。安装前仍应检查来源、manifest、权限和代码内容。

Marketplace 支持安装和管理 skill_pack、rag_pack、mcp_pack、prompt_pack、workflow_pack、benchmark_pack 等资源包。

支持的来源：内置资源包、本地目录或 plugin.json、URL/GitHub 风格地址，以及外部 SKILL.md。安装后可查看已安装资源、来源与信任信息、Skill 列表、权限审批、测试调用、添加到 Workflow 和卸载状态。

审批键 = skill_code + agent_code

因此，批准 skill_console 不等于批准 workflow_runner。Skill 必须在实际运行身份下单独获得批准。

环境要求：Python 3.11+（当前项目支持 Python 3.13）、Node.js 18+；如需 pgvector 或 Docker 沙箱，还需要 Docker Desktop。

git clone https://github.com/biheto/DevAgent-Studio.git
cd DevAgent-Studio

python -m venv .venv
.\.venv\Scripts\activate
pip install -e ".[llm,vector,dev]"

cd web
npm install
npm run build
cd ..

copy .env.example .env
.\.venv\Scripts\python.exe -m uvicorn app.main:app --host 127.0.0.1 --port 8100

打开 http://127.0.0.1:8100/ 使用工作台，打开 http://127.0.0.1:8100/docs 查看 API 文档。

可选配置：

# LLM
OPENAI_API_KEY=your_api_key
DEV_AGENT_LLM_MODEL=gpt-4o-mini

# 长期记忆：llm 为 LLM 提取，rule 为仅规则提取
DEV_AGENT_MEMORY_EXTRACTOR=llm
DEV_AGENT_LLM_MODEL_MEMORY_EXTRACTOR=gpt-4o-mini

# pgvector RAG
DEV_AGENT_RAG_STORE=pgvector
PGVECTOR_DATABASE_URL=postgresql://postgres:postgres@localhost:5432/dev_agent_studio
DEV_AGENT_RAG_RERANKER=off
# DEV_AGENT_RAG_RERANKER=llm

# Docker 代码型 Skill 沙箱
DEV_AGENT_SKILL_SANDBOX=docker
DEV_AGENT_SKILL_SANDBOX_IMAGE=python:3.13-slim
DEV_AGENT_SKILL_SANDBOX_MEMORY=256m
DEV_AGENT_SKILL_SANDBOX_CPUS=0.5
DEV_AGENT_SKILL_SANDBOX_PIDS_LIMIT=64
DEV_AGENT_SKILL_SANDBOX_FALLBACK=false

启动 pgvector：

docker compose -f docker-compose.pgvector.yml up -d

可通过 GET /api/v1/skills/sandbox/status 查看 Skill 沙箱配置状态。

.\.venv\Scripts\python.exe -m compileall -q app

.\.venv\Scripts\python.exe -m unittest tests.test_memory_store tests.test_rag_governance -v

cd web
npm run build
cd ..

.\.venv\Scripts\python.exe -c "from app.skills.sandbox import python_skill_sandbox_status; print(python_skill_sandbox_status())"

DevAgent Studio/
  app/
    agents/              # 项目分析、代码审查、RAG、学习、报告逻辑
    api/                 # FastAPI 路由
    graphs/              # LangGraph 图与 Workflow 编译器
    harness/             # 上下文、事件、策略、产物、审核/恢复运行时
    marketplace/         # 资源包预览、安装、可信信息、SKILL.md 兼容层
    persistence/         # 任务、治理、RAG、Trace 持久化
    providers/           # LLM、MCP、RAG Provider
    skills/              # Registry、契约、版本、依赖、沙箱运行时
  web/                   # React 工作台
  scripts/               # MCP 启动和测试脚本
  docs/                  # 设计和实现说明

• 增加 Docker 沙箱状态和测试调用的前端可视化。
• 增加外部签名插件发布示例和贡献工具。
• 补齐 API、Workflow 编译、Harness 状态、MCP 契约、LLM fallback 和 Benchmark 集成自动化测试。
• 增强 Workflow 条件分支、并行节点和输入输出映射交互。

项目使用 MIT License。分发或修改时请保留原版权和许可证文本。

DevAgent Studio by biheto
https://github.com/biheto/DevAgent-Studio