Build, connect, and orchestrate LLM-powered agents — visually.
Pipelit is a self-hosted workflow automation platform for designing LLM agent pipelines on a drag-and-drop canvas. Wire up triggers, agents, tools, and routing logic — then watch them execute in real time with live WebSocket status updates on every node.
| Visual Canvas | Multi-Trigger | LLM Agents |
| Drag-and-drop React Flow editor with node palette, config panel, and live execution badges | Telegram, webhooks, chat, scheduled intervals, manual — all unified as workflow nodes | LangGraph react agents with tool-calling: shell, HTTP, web search, calculator, datetime, and more |
| Conditional Routing | Scheduled Execution | Real-time Updates |
| Switch nodes evaluate rules and route to different branches via conditional edges | Recurring runs with configurable interval, retry with exponential backoff, pause/resume, and crash recovery | Single global WebSocket pushes node status, execution events, and canvas mutations — no polling |
| Cost Tracking | Conversation Memory | Self-Improving Agents |
| Per-execution token counting and USD cost calculation with Epic-level budget enforcement | Optional per-agent conversation persistence across executions via SQLite checkpointer | Agents can read epics/tasks, spawn child workflows, modify their own graphs, and schedule future work |
- Python 3.10+
- Redis 8.0+ (includes RediSearch natively — see Redis Setup)
- Node.js 18+
git clone git@github.com:theuselessai/Pipelit.git
cd Pipelit
# Backend
python3 -m venv .venv && source .venv/bin/activate
pip install -r platform/requirements.txt
# Frontend
cd platform/frontend && npm install# Generate an encryption key for credential storage
echo "FIELD_ENCRYPTION_KEY=$(python -c 'from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())')" >> .envOr create a .env file in the project root with at minimum:
FIELD_ENCRYPTION_KEY=<your-generated-key>
REDIS_URL=redis://localhost:6379/0Start Redis, then launch everything with a single command:
cd platform && source ../.venv/bin/activate
honcho startThis starts four processes via the Procfile:
| Process | Command | Description |
|---|---|---|
| server | uvicorn main:app --reload |
FastAPI backend on :8000 |
| frontend | npm run dev |
Vite dev server on :5173, proxies /api to :8000 |
| scheduler | rq worker --worker-class worker_class.PipelitWorker workflows --with-scheduler |
1 worker with job scheduler (handles enqueue_in delayed jobs) |
| worker | rq worker-pool workflows -w worker_class.PipelitWorker -n 2 |
2 additional workers for parallel job processing |
The backend auto-creates the database on first startup.
Manual startup (without honcho)
# Terminal 1 — Backend
cd platform && source ../.venv/bin/activate
uvicorn main:app --host 0.0.0.0 --port 8000 --reload
# Terminal 2 — RQ Worker with scheduler
cd platform && source ../.venv/bin/activate
rq worker --worker-class worker_class.PipelitWorker workflows --with-scheduler
# Terminal 3 — Frontend (dev)
cd platform/frontend
npm run dev # http://localhost:5173, proxies /api to :8000Open http://localhost:5173 — the setup wizard will prompt you to create your admin account.
Production: Skip the frontend dev server — run
cd platform/frontend && npm run buildonce, then access the app directly athttp://localhost:8000(FastAPI serves the built SPA).
| Layer | Technologies |
|---|---|
| Backend | FastAPI, SQLAlchemy 2.0, Alembic, Pydantic, RQ (Redis Queue) |
| Frontend | React, Vite, TypeScript, Shadcn/ui, React Flow (@xyflow/react v12), TanStack Query |
| Execution | LangGraph, LangChain, Redis pub/sub, WebSocket |
| Auth | Bearer token API keys, TOTP-based MFA |
Design agent pipelines on an interactive React Flow canvas. Add nodes from the palette, configure them in the side panel, and connect them with typed edges. Live execution badges show running/success/failed status on each node.
Every trigger is a first-class workflow node:
| Trigger | Description |
|---|---|
| Chat | Built-in web chat interface |
| Telegram | Bot integration with webhook support |
| Webhook | Accept external HTTP payloads |
| Scheduler | Recurring interval execution with retry, backoff, and pause/resume |
| Manual | One-click execution from the UI |
| Component | Description |
|---|---|
| Agent | LangGraph react agent with system prompt, tools, and optional conversation memory |
| Deep Agent | Advanced agent with built-in task planning, filesystem tools, and subagent delegation |
| Categorizer | Classifies input into predefined categories |
| Router | Routes messages to different branches based on content |
| Extractor | Extracts structured data from unstructured text |
| Switch | Rule-based conditional routing with per-edge condition values |
| Loop | Iterates over collections, executing a body node per item |
| Spawn & Await | Launches a child workflow and waits for its result |
Agents can call any combination of built-in tools:
run_command | http_request | web_search | calculator | datetime | epic_tools | task_tools | workflow_discover | workflow_create
Reference upstream node outputs in prompts and config fields with {{ nodeId.portName }} syntax. Full Jinja2 filter support. The editor provides a variable picker and syntax highlighting.
Automatic token counting and USD cost calculation per execution. Set token or USD budgets on Epics — the orchestrator gates every node execution against the budget before running.
Create recurring jobs that fire workflow triggers at configurable intervals. Self-rescheduling via RQ enqueue_in() — no external cron needed. Features exponential backoff on failure, pause/resume, and automatic recovery of missed jobs on startup.
- Encrypted credential storage (Fernet)
- TOTP-based MFA with rate limiting and account lockout
- Bearer token authentication
- Agent-to-agent identity verification via TOTP
All endpoints under /api/v1/, authenticated via Authorization: Bearer <key>.
| Resource | Endpoints |
|---|---|
| Auth | POST /auth/token/, GET /auth/me/, POST /auth/setup/ |
| Workflows | GET/POST /workflows/, GET/PATCH/DELETE /workflows/{slug}/, POST .../validate/ |
| Nodes | GET/POST /workflows/{slug}/nodes/, PATCH/DELETE .../nodes/{node_id}/ |
| Edges | GET/POST /workflows/{slug}/edges/, PATCH/DELETE .../edges/{id}/ |
| Executions | GET /executions/, GET .../executions/{id}/, POST .../cancel/ |
| Chat | POST /workflows/{slug}/chat/, DELETE .../chat/history |
| Credentials | GET/POST /credentials/, GET/PATCH/DELETE .../credentials/{id}/, POST .../test/ |
| Schedules | GET/POST /schedules/, GET/PATCH/DELETE .../schedules/{id}/, POST .../pause/, POST .../resume/ |
| Memory | Facts, episodes, procedures, users, checkpoints — all with batch delete |
All list endpoints return {"items": [...], "total": N} with limit/offset pagination.
cd platform
source ../.venv/bin/activate
export FIELD_ENCRYPTION_KEY=$(python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())")
python -m pytest tests/ -vPipelit requires Redis 8.0+ which includes RediSearch natively. Older versions will fail with unknown command 'FT._LIST'.
Installing Redis 8.0+
# Docker (easiest)
docker run -d --name redis -p 6379:6379 redis:8
# Debian/Ubuntu
curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg
sudo chmod 644 /usr/share/keyrings/redis-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main" \
| sudo tee /etc/apt/sources.list.d/redis.list
sudo apt-get update && sudo apt-get install redis
# macOS
brew install redis && brew services start redisVerify: redis-cli MODULE LIST should include the search (ft) module.
Removing older Redis
# Debian/Ubuntu
sudo systemctl stop redis-server && sudo systemctl disable redis-server
sudo apt remove --purge redis-server
# macOS
brew services stop redis && brew uninstall redis
# Docker
docker stop redis && docker rm redisWorkaround without Redis 8
Enable conversation_memory on agent nodes that use spawn_and_await. This switches the checkpointer from RedisSaver to SqliteSaver, bypassing the RediSearch requirement.
MIT
