A single-binary task queue and job runner for local Python services.
Go engine · SQLite WAL · embedded Dashboard · Python SDK (pull worker).
Status: v0.1 — under active development
Managing local Python tasks with crontab or Celery is painful:
| Tool | Problem |
|---|---|
| crontab | No status tracking, no retry, silent failures |
| Celery | Requires Redis/RabbitMQ, broken Flower dashboard, heavy config |
ErmineTQ is a single Go binary. It stores everything in a local SQLite file and serves a built-in dashboard. Python connects in three lines.
# Build from source
git clone https://github.com/peifengstudio/erminetq
cd erminetq
make build
# Run with defaults (creates erminetq.db in the current directory)
./bin/erminetq server
# Run with a config file
cp erminetq.example.toml erminetq.toml
./bin/erminetq serverfrom erminetq import Client, Worker
# Submit tasks
with Client("http://localhost:8080") as client:
task_id = client.submit("send_email", {"to": "alice@example.com"})
# Run a worker
worker = Worker("http://localhost:8080", concurrency=4)
@worker.register("send_email")
def send_email(payload: dict) -> dict:
...
return {"sent": True}
worker.run() # blocks; Ctrl+C for graceful shutdownErmineTQ is configured via a TOML file. Copy the annotated example to get started:
cp erminetq.example.toml erminetq.tomlBy default ErmineTQ looks for erminetq.toml in the working directory.
The file is optional — if it is absent, compiled-in defaults are used.
[db]
path = "erminetq.db" # SQLite file path
[limits]
global = 128 # max total concurrent tasks
[queues.default]
limit = 32 # max concurrent tasks in this queue
[queues.ollama]
limit = 2 # GPU-bound queue, keep low
[task_types.ollama_generate]
queue = "ollama"
limit = 1 # one inference job at a timeSee erminetq.example.toml for the full reference with comments.
ErmineTQ enforces three nested concurrency scopes. All applicable limits must have capacity before a task is dispatched:
global limit
└── queue limit (if configured for that queue)
└── task-type limit (if configured for that type)
A limit of 0 at queue or task-type scope means unlimited at that scope
(global and any other applicable limit still apply).
| Variable | Default | Description |
|---|---|---|
ERMINETQ_CONFIG |
erminetq.toml |
Path to the TOML config file |
ERMINETQ_DB |
erminetq.db |
Path to the SQLite database file |
ERMINETQ_DB overrides db.path in the config file.
Every setting follows the same priority chain (highest wins):
CLI flag > environment variable > config file > compiled-in default
Examples:
# Use a non-default config file
ERMINETQ_CONFIG=./config/prod.toml ./bin/erminetq server
# Override the database path at runtime without editing the config file
ERMINETQ_DB=./data/myapp.db ./bin/erminetq server
# CLI flag takes precedence over everything
ERMINETQ_DB=foo.db ./bin/erminetq server -db bar.db # uses bar.dberminetq <command> [flags]
Commands:
server Start the ErmineTQ server (also applies migrations on startup)
migrate Apply pending database migrations and exit
version Print version information and exit
Flags (server / migrate):
-config string
Path to TOML config file.
Env: ERMINETQ_CONFIG (default: erminetq.toml)
-db string
Override database path.
Takes precedence over config file and ERMINETQ_DB.
- Go 1.25+, Python 3.11+, uv — run
mise installif you use mise golangci-lintfor linting (optional)airfor hot-reload dev:go install github.com/air-verse/air@latest
make deps # download all Go module dependencies
make build # compile → bin/erminetq
make dev # hot-reload server via air (requires air)
make run # run server once via go run
make migrate # apply migrations and exit
make test # run all tests
make test-v # verbose test output
make test-store # run only internal/store tests
make test-cover # generate + open HTML coverage report
make lint # golangci-lint
make fmt # gofmt in-place
make tidy # go mod tidy
make clean # remove bin/ and coverage.out
make clean-db # remove local *.db files
make help # list all targets
# Python SDK examples — server must be running first (make dev)
make example-py-worker # Terminal 2: start Python pull worker
make example-py-submit # Terminal 3: submit example tasks and print results
make example-submit # Go-only: submit Go handler tasksUse env vars with any make target:
ERMINETQ_CONFIG=./config/dev.toml ERMINETQ_DB=./data/dev.db make run
ERMINETQ_DB=./data/dev.db make migrate┌─────────────────────────────────────────────────────┐
│ Clients │
│ Python SDK (httpx) curl / browser │
└────────────┬────────────────────────────────────────┘
│ HTTP JSON API + SSE :8080
┌────────────▼────────────────────────────────────────┐
│ Go Engine │
│ HTTP Router → Task Queue → Worker Pool │
│ └── Go Worker (goroutine) │
│ Pull Worker API (/api/worker/claim + report) │
│ Retry Scheduler + Heartbeat Scanner │
│ Cron Scheduler (schedules → tasks) │
│ SQLite WAL (single file, single writer goroutine) │
└──────────────────┬──────────────────────────────────┘
│ SSE + embed.FS
┌──────────────────▼──────────────────────────────────┐
│ Dashboard (Tailwind + Alpine.js, fully embedded) │
│ Overview · Tasks · Task Detail · Workers · Schedules│
└─────────────────────────────────────────────────────┘
Python SDK worker (separate process, any machine on the same network)
poll → POST /api/worker/claim → execute handler → POST /api/worker/attempts/{id}/succeed|fail
- Go Engine — HTTP API, task queue, worker pool, scheduler, heartbeat scanner
- SQLite WAL — single-file storage, zero external dependencies, single write goroutine
- Python SDK — pull worker: polls
/api/worker/claim, executes handlers, reports results; no Unix socket or persistent connection needed - Dashboard — Tailwind + Alpine.js UI compiled into
dashboard/dist/and embedded into the binary viaembed.FS
See docs/DESIGN.md for full design rationale.
cmd/server/ main entry point — wires everything together
internal/config/ TOML config loader and execution-limit types
internal/store/ SQLite store layer — ALL state transitions go here
internal/queue/ in-memory priority queue + worker pool
internal/api/ HTTP JSON API handlers and types
internal/scheduler/ cron + interval scheduler
internal/dashboard/ HTTP server + SSE broker + embedded static files
dashboard/dist/ frontend build output (embedded into binary)
sdk/python/ Python SDK (erminetq package — Client + Worker)
examples/go/ Go example handlers and task submit script
examples/python/ Python SDK worker and task submit script
docs/ design documents
erminetq.example.toml annotated reference configuration
queued → running → succeeded
→ retrying → queued (backoff via run_at)
→ dead (retries exhausted)
→ halted → queued (resume)
→ cancelled
→ cancelled
restart: original → superseded, new task queued with parent_id
failed is not a task status — it exists only on individual attempts.
The SDK lives in sdk/python/ and is installable as a path dependency:
# via uv (recommended)
uv add erminetq --path ./sdk/python
# via pip
pip install -e ./sdk/python