A node-graph dataflow execution engine designed for LLM agents. Compose data pipelines as {nodes, edges} graphs, POST them to an API, and receive transformed data back.
Built on Nushell for data transformation and Hono for the API.
- Features
- Quick Start
- Prerequisites
- Installation
- Usage
- Development
- Testing
- Environment Variables
- Architecture
- Node Catalog
- Docker Deployment
- Project Structure
- License
- 140+ built-in nodes across 22 categories — from basic transforms to external data sources
- Graph-based execution — define pipelines as JSON/NUON node graphs with typed ports and edges
- Topological execution — automatic dependency resolution and parallelization where safe
- Nushell-powered transforms — all data operations delegate to Nushell subprocesses for robust structured data handling
- LLM-native — first-class
llmandanalyzenodes with support for Anthropic, OpenAI, and local models (LM Studio, Ollama) - Persistent patches — save and reuse named pipeline graphs in SQLite
- Streaming execution — Server-Sent Events for real-time progress on the canvas
- Mermaid diagrams — generate ASCII flowcharts from any saved patch
- Data source integrations — Hacker News, Reddit, Wikipedia, YouTube, GitHub, RSS, SEC filings, FRED, BLS, CoinGecko, and more
# Clone and enter the repo
cd junction-box
# Configure environment
cp .env.example .env
# Edit .env with your API keys (optional for basic usage)
# Run with Docker (recommended)
docker compose up --build
# Or run locally (requires Bun + Nushell)
cd server && bun install && bun run devThe API is available at http://localhost:3001.
| Tool | Version | Purpose |
|---|---|---|
| Bun | latest | JavaScript runtime and package manager |
| Nushell | 0.111.0 | Data transformation engine |
| Docker | (optional) | Containerized deployment |
Nushell must be on your $PATH. The Docker image includes Nushell automatically.
Backend:
cd server
bun install
bun run dev # --watch mode on port 3001# Build and run
docker compose up --build
# Or manually
docker build -t junction-box .
docker run --env-file .env \
--add-host=host.docker.internal:host-gateway \
-p 3001:3001 -v junction-box-data:/app/data junction-box| Method | Path | Description |
|---|---|---|
GET |
/ |
LLM-oriented manifest and quick-start guide |
GET |
/health |
Health check |
GET |
/nodes |
Full node specification array |
GET |
/catalog |
Token-efficient catalog (filter with ?category=) |
GET |
/defs/:type |
Single node definition + example |
GET |
/defs |
All node definitions |
GET |
/patterns |
Pre-built pipeline patterns |
POST |
/exec |
Synchronous graph execution |
POST |
/run |
SSE streaming execution (for canvas) |
POST |
/patch |
Save a named graph patch |
GET |
/patch/:alias |
Retrieve a patch |
DELETE |
/patch/:alias |
Delete a patch |
GET |
/patches |
List all patches |
GET |
/runs/:run_id |
Retrieve run result |
GET |
/runs |
List runs (paginated) |
GET |
/logs |
Raw execution log (runs.jsonl) |
POST |
/parse-nuon |
Parse NUON text to JSON |
GET |
/visualise/:alias |
ASCII Mermaid flowchart |
curl -X POST http://localhost:3001/exec \
-H "Content-Type: application/json" \
-d '{
"nodes": [
{ "id": "a", "type": "const", "params": { "value": "hello" } },
{ "id": "b", "type": "string_op", "params": { "op": "uppercase" } }
],
"edges": [
{ "from": "a", "to": "b", "fromPort": "out", "toPort": "in" }
]
}'More examples are in junction-box-graphs/.
The backend entrypoint is server/index.ts (Hono app). It auto-discovers Nushell primitives from primitives.nu and extensions/*.nu via server/spec.ts.
Key server modules:
| File | Purpose |
|---|---|
server/db.ts |
SQLite schema and CRUD |
server/validate.ts |
Pre-flight graph validation (types, ports, cycles) |
server/execute.ts |
Runtime execution + error normalization |
server/exec-runner.ts |
Execution orchestration |
server/spec.ts |
Dynamic introspection of Nushell primitives |
server/toposort.ts |
Topological sort utility |
server/mermaid.ts |
Mermaid diagram generation |
cd server
bun testTests cover SQLite persistence, schema validation, smoke tests, and per-category node tests (compute, datetime, input, logic, transform).
Copy .env.example to .env and configure:
| Variable | Required | Description |
|---|---|---|
LLM_ENDPOINT |
No | OpenAI-compatible URL or empty for Anthropic cloud |
LLM_MODEL |
No | Default model ID (e.g. claude-sonnet-4-6, gpt-4o) |
LLM_API_KEY |
No | API key for LLM endpoint |
ANTHROPIC_API_KEY |
No | Alternative Anthropic key |
FRED_API_KEY |
No | For FRED economic data nodes |
BLS_API_KEY |
No | For BLS labor statistics nodes |
For LM Studio local models, use http://host.docker.internal:1234/v1/chat/completions.
┌─────────────┐ ┌─────────────┐ ┌─────────────────┐
│ Client │────▶│ Hono API │────▶│ Graph Engine │
│ (Agent/UI) │◀────│ (Bun) │◀────│ (validate + │
└─────────────┘ └─────────────┘ │ toposort) │
└─────────────────┘
│
▼
┌─────────────────┐
│ Nushell 0.111 │
│ (primitives.nu │
│ + extensions) │
└─────────────────┘
Graphs are validated, topologically sorted, then executed by spawning Nushell commands prefixed with prim-. Results are normalized and returned as JSON.
Nodes are organized into categories. Core categories include:
| Category | Examples |
|---|---|
input |
const, env, file_in, fetch |
transform |
filter, map, select, sort, join, table_concat, insert_row |
compute |
math, hash, encode_base64, string_op |
logic |
if, match, try, catch, for, while |
datetime |
date_now, date_add, to_timezone |
output |
return, display, file_out |
external |
llm, analyze |
web |
http_post, http_put, http_delete |
hn |
Hacker News stories, comments, jobs |
reddit |
Subreddit posts, search |
wikipedia |
Article search, content extraction |
youtube |
Video search, transcript |
github |
Repo search, issues, commits |
rss |
Feed fetching and parsing |
market |
Stock quotes, fear/greed index |
coingecko |
Crypto prices, market data |
sec |
SEC filings search |
fred |
FRED economic series |
bls |
BLS labor statistics |
Retrieve the full catalog via GET /catalog.
The Dockerfile uses a multi-stage build on Debian with Nushell and Bun included. docker-compose.yml:
- Exposes port
3001 - Mounts
junction-box-datavolume for SQLite persistence - Fixes
host.docker.internalfor Linux hosts
docker compose up --buildjunction-box/
├── bin/ # Entry scripts and Nushell plugins
├── data/ # SQLite DB, logs, datasets
├── docs/ # Design docs and superpowers
├── extensions/ # Nushell data-source extensions (14 files)
├── junction-box-graphs/ # 50+ example pipeline JSONs
├── patches/ # Legacy patch files
├── primitives.nu # Core Nushell primitive definitions
├── scripts/ # Migration and introspection scripts
├── server/ # Hono backend (TypeScript)
│ ├── index.ts # Main API server
│ ├── db.ts # SQLite layer
│ ├── validate.ts # Graph validation
│ ├── execute.ts # Execution engine
│ ├── spec.ts # Nushell introspection
│ └── ...
├── tests/ # Bun test suites
├── .env.example # Environment template
├── docker-compose.yml
├── Dockerfile
└── ASSESSMENT.md # API assessment report