AgentBuilder

AgentBuilder is a generic environment for debugging and running AI agents on the OpenAI Responses API.

It provides a backend, a developer UI, a public chat UI, project skills, private user-created skills, MCP diagnostics, raw request/response logs, and a small set of neutral test tools. It is also a practical starting point for deploying your own AI application: replace the prompts, add domain tools and MCP servers, then ship the simplified public frontend.

What is included

• FastAPI backend with auth, chat sessions, streaming, API logs, skills, and MCP bridge.
• dev_frontend: developer console with chats, raw Responses API exchanges, tool activity, Skills diagnostics, and MCP diagnostics.
• public_frontend: simplified end-user app with login and chats only.
• Local function tool: fetch_page_text.
• Test MCP servers:
  • echo: returns echo/add tool results.
  • test-empty: intentionally returns no tools for diagnostics testing.
• Local skills:
  • create-user-skill
  • summarize-text
  • inspect-mcp
• Private user skills stored in the database and exposed to the model through DB-backed virtual shell skill paths.
• Prompt split under prompts/:
  • AGENT_INSTRUCTIONS.md for compact always-on behavior.
  • ASSISTANT_PROMPT.md for first-turn application context.

Quick start

Backend:

cd D:\Code\AgentBuilder
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
Copy-Item .env_template .env
uvicorn backend.app.main:app --reload --host 127.0.0.1 --port 8101

Developer frontend:

cd D:\Code\AgentBuilder\dev_frontend
npm install
npm run dev

Open http://localhost:5273.

Public frontend:

cd D:\Code\AgentBuilder\public_frontend
npm install
npm run dev

Open http://localhost:5274.

Without OPENAI_API_KEY, chat runs in stub mode. After setting the key, the backend calls client.responses.create, stores previous_response_id, logs raw exchanges, and streams model/tool events to the frontend.

User Skills

Users can create reusable private skills directly from chat. When a user asks to create, save, or update a custom workflow, the agent uses the built-in create-user-skill skill and the create_user_skill tool to store a validated SKILL.md body in the user_skills database table.

At runtime, enabled user skills are exposed only to their owner through the shell skill registry with virtual paths like:

agentbuilder://user-skills/1/meeting-notes

Those paths are not files. When the model reads a virtual user skill path through the shell tool, the backend intercepts the call and returns the skill content from the database. This keeps project skills file-based while keeping user skills private and database-backed.

Configuration

Copy .env_template to .env and adjust:

DATABASE_URL=sqlite:///./agent_builder.db
OPENAI_API_KEY=
OPENAI_MODEL=gpt-4.1-mini
MCP_SETTINGS_PATH=mcp_settings.json
AGENT_SKILLS_DIR=skills
CORS_ORIGINS=http://localhost:5273,http://127.0.0.1:5273,http://localhost:5274,http://127.0.0.1:5274

Repository Layout

backend/          FastAPI app and Responses API agent client
dev_frontend/     Debug UI for agent developers
public_frontend/  Minimal user-facing chat UI
docs/             Setup and extension notes
mcp_servers/      Local test MCP servers
prompts/          Base instructions sent to the model
skills/           Local Codex-style skills

Extending

Use AgentBuilder as a scaffold for your own AI application:

1. Replace prompts/AGENT_INSTRUCTIONS.md and prompts/ASSISTANT_PROMPT.md.
2. Add domain-specific function tools in backend/app/tools.py.
3. Add or connect MCP servers in mcp_settings.json.
4. Add reusable workflows under skills/<name>/SKILL.md.
5. Let users create private reusable workflows from chat with create_user_skill.
6. Keep dev_frontend for debugging and adapt public_frontend for real users.

Read more

• docs/development.md: local development setup, backend/frontend ports, proxy behavior, and important API endpoints.
• docs/mcp.md: MCP configuration, local diagnostic servers, and how AgentBuilder exposes MCP tools to the model.
• docs/prompts.md: how AGENT_INSTRUCTIONS.md and ASSISTANT_PROMPT.md are loaded, when they are sent, and how to edit them safely.
• docs/skills.md: project skills, private user skills, SKILL.md validation rules, registry lifecycle, explicit @skill-name mentions, and shell-tool execution flow.