Repository navigation
Runbooks Index
github-actions[bot] edited this page Oct 30, 2026
·
8 revisions
Canonical source:
docs/11-runbooks/README.md— this page is rendered from it on every publish; edit the source.
Operational runbooks for diagnosing and recovering ClawAI. Start with the symptom → runbook table, then drill into the categorized list.
| Symptom | Start here |
|---|---|
| A curated MCP server must be added or blocked, or a user's server waits for approval | runbook-plugin-admin.md → catalog curation screen, approval queue, rules, audit log |
| A service won't start / crash-loops | runbook-service-crash.md → check Docker logs, DB reachability, migrations |
| A deployed service crash-loops after a release (Nest cannot resolve dependencies) | runbook-prod-service-crash-loop.md → docker logs, module providers, compile spec |
Uploads rejected: antivirus_scan … ECONNREFUSED …:3310
|
runbook-clamav-unreachable.md → clamd died (OOM) inside a still-running container; restart, watchdog prevents recurrence |
| Status page: "Web scraper: Crawl4AI/FlareSolverr/Firecrawl" Down; research degraded | runbook-scraper-sidecar-down.md → enabled row but container/profile not started, missing Crawl4AI token, or OOM |
| Server unreachable / CPU 100% / disk full after deploys |
runbook-server-overloaded-by-builds.md → docker system df, .deploy/history.log; releases no longer rebuild every image (ADR-123) |
| A new API route 404s as HTML in production | runbook-nginx-stale-config.md → nginx is pinned to a stale config inode; recreate the container |
| A deploy refuses to start: "the production .env ... is missing or has unusable values" |
add-a-service-to-production.md → scripts/ensure-prod-env.sh --check, then append what is missing |
| Junk or a flood on the public (no-login) feedback form |
runbook-public-feedback-spam.md → honeypot, Redis + nginx limits, triage source=PUBLIC
|
| Users see "Too many attempts. Try again in N minutes." on sign-in / sign-up | runbook-auth-rate-limits.md → find the policy in the logs, raise a budget, or clear one hashed Redis key |
| A service fails on a symbol its source declares |
runbook-stale-shared-package-dist.md → the container carries an image-baked packages/*/dist; rebuild the image |
A build fails on below the CVE-2026-111497 floor, or you must prove the running Node version |
runbook-node-runtime-floor.md → node tools/security/node-runtime-floor.mjs --running, pull fresh base, raise the floor |
| Requests are slow / timing out | runbook-high-latency.md |
| Status page shows Degraded / Down / no history | runbook-status-page-degraded.md → map the component to its services, then read the raw fan-out or Prometheus |
| A database is corrupt / needs restore | runbook-database-recovery.md |
| The AI "forgot" something earlier in the thread |
context-loss-triage.md → read the receipt's conversation block; it says what was sent and why the rest was not |
| A Thread generation failed or looks stuck | runbook-thread-generation-failed.md → read the attempt log (phase, class, healing steps) |
| Routing picks the "wrong" model | runbook-routing-misclassification.md |
| A chat mode states web facts with no citations, or Consensus copies a fabricated lane | runbook-fabricated-web-facts.md → check whether evidence reached the prompt, then whether grounding fired for that mode |
ollama pull / model download fails |
runbook-model-pull-failure.md |
| Picking an image model always fails |
runbook-image-generation-failure.md → did chat-service redirect it to image-service at all; read image_generations.error_code, never the stored message |
| A video job is stuck or failed |
runbook-video-generation-failure.md → read the video_generations row and docker logs claw-image-service
|
TLS / cert / Hostname doesn't match errors |
troubleshoot-tls.md |
| Local frontier (llama.cpp) issues | frontier-troubleshooting.md · frontier-first-time-walkthrough.md |
| A voice note is never transcribed / "transcription service is busy" |
runbook-voice-note-transcription-failed.md → read the ranked walk in the file-service log; kind= names the cause |
| An answer starts with the model's private notes ("Be concise.") |
rules/56 → a reasoning channel reached content; GLM ends its reasoning with a bare </think>
|
| Chat says "every available AI provider failed" | runbook-provider-call-rejected.md → read the provider's own body; usually a dropped param or no credit |
| Provider connector: sync 500s but the test says OK |
runbook-connector-model-sync-failure.md → the test toast lied; read model_sync_runs.error_message
|
| Workspace connector sync failing | runbook-workspace-automation.md |
| Billing reconciliation is stuck | runbook-billing-reconciliation.md |
| Localized prices are wrong, absurd or all USD | runbook-display-fx-outage.md |
| A billing sweep or delivery job failed | runbook-failed-billing-sweep.md |
| Credit paid for but never granted / nothing meters | runbook-payg-credit.md → boot order, swallowed seed, empty price table; the log is not evidence |
| Anything else | troubleshooting.md · operational-runbooks.md |
Before any runbook, do the standard triage from ../../skills/04-debug-toolkit.md: Docker logs → DB query → API curl → RabbitMQ → nginx. Most incidents are diagnosed in that order. For build/compile failures, see ../08-runtime-devops/build-system.md §7.
- operational-runbooks.md — day-to-day ops
- troubleshooting.md — general troubleshooting
- runbook-service-crash.md
- runbook-plugin-admin.md — curate the plugin catalog, approve or reject users' servers, rules, audit
- add-a-service-to-production.md — a release adds a service with a database or secret
- runbook-clamav-unreachable.md
- runbook-scraper-sidecar-down.md
- runbook-public-feedback-spam.md
- runbook-auth-rate-limits.md
- runbook-server-overloaded-by-builds.md
- runbook-high-latency.md
- runbook-database-recovery.md
- troubleshoot-tls.md
- runbook-routing-misclassification.md
- runbook-model-pull-failure.md
- runbook-image-generation-failure.md
- runbook-connector-model-sync-failure.md
- runbook-provider-call-rejected.md
- runbook-voice-note-transcription-failed.md
- runbook-fabricated-web-facts.md
- runbook-capability-framework.md
- runbook-filesystem-capability.md
- runbook-process-capability.md
- runbook-browser-capability.md
- runbook-screen-capability.md
- runbook-application-capability.md
- runbook-audio-capability.md
- runbook-clipboard-notification.md
- runbook-recipe-engine.md
- runbook-activity-memory.md
- runbook-cross-os-evidence.md
- runbook-desktop-agent-security.md
- runbook-desktop-agent-qa-release-gate.md
- runbook-desktop-agent-release-channels.md
- runbook-tauri-shell-release.md
- runbook-marketplace.md
- billing-operations.md
- runbook-billing-reconciliation.md
- runbook-display-fx-outage.md
- runbook-failed-billing-sweep.md
- runbook-payg-credit.md — deploying, verifying and killing PAYG connector credit
- runbook-video-generation-failure.md — a video job stuck, timed out or failed
- runbook-video-generation-failure.md
- runbook-stale-shared-dist-after-pull.md
ClawAI Wiki · source-controlled under wiki/ · generated/mirrored from the current repository knowledge layer.
ClawAI Wiki
Product
- Product-Vision
- Flagship-Features
- Business-Overview
- Requirements-Register
- Drift-Log
- Runtime-Managed-Connector-Providers
- Threads
- ADR-Index
Architecture
- System-at-a-Glance
- Architecture-Map
- System-Architecture
- Message-Flow
- End-to-End-Data-Flow
- Routing-Engine
- Event-Bus
- Security-Architecture
Platform
- Backend-Services
- Frontend-Architecture
- Shared-Packages
- Database-Reference
- API-Reference
- Provider-Catalog
- Memory-and-Context-Architecture
- File-Support-Architecture
- Research-and-Web-Crawling
Runtime & Ops
- Docker-Guide
- Nginx-Reference
- CI-CD-Pipeline
- Logging-and-Observability
- Operational-Runbooks
- Troubleshooting
Quality
Engineering OS
- AI-Native-Engineering-OS
- Generated-Manifests
- Rules-Catalog
- Skills-Catalog
- Context-Layer
- Engineering-Memory
- Tooling-and-Automation
- Scripts-and-Operations
- ESLint-and-Architecture-Enforcement
- Extended-Work-Skills
Agents
Reference