The Brain

The Brain

shelldon has no model of its own. A Raspberry Pi Zero 2W (416 MB RAM, aarch64) cannot host a usable LLM, so every "thought" is a network call to a remote provider — GLM via Z.ai, native Anthropic Claude, or any OpenAI-compatible endpoint. The component that makes those calls is the broker, and it is deliberately the only part of the system allowed to hold a credential or touch the network.

This page documents the broker: its role as the egress and safety boundary, the provider abstraction, the ordered chain, automatic fallback, retry, the credential vault, native function-calling, and what happens when every provider is down.

Related pages: Architecture · How a Turn Works · The Bus · Self-Coding

All source lives under shelldon/broker/:

shelldon/broker/
├── __init__.py            # "sole credential holder + model/tool egress"
├── provider.py            # the LLMProvider seam + the two error types
├── anthropic_provider.py  # Anthropic-format adapter (GLM via Z.ai + native Claude)
├── openai_provider.py     # OpenAI-compatible adapter (Ollama / OpenAI / OpenRouter / …)
├── chain.py               # config-driven ordered chain builder
├── broker.py              # retry (handle_job) + fallback (handle_job_chain)
├── service.py             # the bus-client loop (run_broker)
└── vault.py               # the broker-only secret-surfacing gate

The broker is the only egress

The architecture rule (AD-2) is blunt: the broker is a separate process and the only holder of credentials and the only code path out to a model or tool. Nothing else in the system — not the core, not the worker, not a plugin — ever holds an API key or opens a socket to a provider.

This is enforced three ways, two of them mechanical:

1. Process separation. The broker runs as its own process and is the privileged actor (the service uid). Other actors talk to it only over the bus.
2. Credentials never travel on the bus. A Job envelope carries the prompt and nothing credential-shaped. The broker injects the key internally when it constructs the provider. There is no key field on any Job, Completion, or Envelope.
3. An import-linter rule keeps core/ LLM-free (AD-1). The [tool.importlinter] "core is LLM-free" contract in pyproject.toml forbids core/ from importing anthropic, openai, google, litellm, zhipuai, or ollama. CI runs uv run lint-imports; if core ever imported a provider SDK the build fails. The provider SDKs are imported only under broker/.

The practical payoff: a bug, a prompt-injection, or a rogue self-written tool in the untrusted "brain" path cannot read your keys or call the network on its own — it has to go through the broker, which is small, auditable, and does no pet-domain parsing.

The provider seam

Every provider, regardless of vendor SDK, sits behind one tiny interface defined in shelldon/broker/provider.py:

@runtime_checkable
class LLMProvider(Protocol):
    name: str  # audit label (chain preset name, e.g. "glm") — never a credential

    async def complete(self, prompt: str) -> str: ...

    async def complete_with_tools(
        self, messages: list[Message], tools: list[ToolDefinition]
    ) -> Completion: ...

Two failure types accompany it, and they are the load-bearing contract of the whole brain:

class TransientProviderError(Exception):
    """Retryable — timeout, rate-limit, or a 5xx."""

class PermanentProviderError(Exception):
    """Non-retryable — bad request, auth, or any other 4xx."""

The rule is absolute: an adapter maps its SDK's exceptions to exactly one of these two types and never lets a raw vendor exception escape. The broker's retry and fallback logic keys only on TransientProviderError / PermanentProviderError, so it stays testable with fakes and never needs to know a vendor SDK exists. A new adapter that leaked a raw SDK exception would break fallback.

There are exactly two adapters, grouped by wire format, not by vendor.

Anthropic-format adapter

shelldon/broker/anthropic_provider.py::AnthropicProvider wraps anthropic.AsyncAnthropic. The same code serves two very different backends — the only difference is config:

• GLM (the default) — point base_url at Z.ai's Anthropic-compatible endpoint (https://api.z.ai/api/anthropic) with a GLM key and a GLM model id.
• Native Claude — leave base_url at the SDK default (api.anthropic.com) with an Anthropic key and a Claude model.

Error mapping (the canonical template every adapter copies):

except (APITimeoutError, APIConnectionError, RateLimitError, InternalServerError) as exc:
    raise TransientProviderError(type(exc).__name__) from exc
except anthropic.APIStatusError as exc:
    if exc.status_code >= 500:
        raise TransientProviderError(type(exc).__name__) from exc
    raise PermanentProviderError(f"status {exc.status_code}") from exc

Note type(exc).__name__, not str(exc). SDK exception messages can embed request headers or keys, and that text would otherwise cross the bus inside Completion.error. Only the exception type name is surfaced; the full detail stays in the chained __cause__ for broker-local logs. A reply with no text and no tool calls raises PermanentProviderError("provider returned no text") — a no-text reply is a failed turn, not a silent empty success.

OpenAI-compatible adapter

shelldon/broker/openai_provider.py::OpenAIProvider wraps openai.AsyncOpenAI. One adapter, many endpoints — base_url is the only thing that changes between Ollama-on-the-LAN, OpenAI, OpenRouter, Groq, Cerebras, NVIDIA, Mistral, GitHub Models, and Gemini (via its OpenAI-compatible endpoint). The error mapping mirrors the Anthropic adapter exactly, so fallback keys on the same two types.

Ollama is the one quirk: it ignores the API key, but the SDK still requires a non-empty one, so the chain passes a placeholder ("ollama").

The ordered chain

shelldon/broker/chain.py::build_chain(env) reads one config variable — PROVIDER_CHAIN — and produces an ordered list[LLMProvider]. It is a comma-separated list of preset names, tried left to right, default "glm":

PROVIDER_CHAIN="glm,ollama,gemini"

Each preset name maps to (adapter class, base_url, key env var, model env var). Most presets are OpenAI-compatible and live as data rows in _OPENAI_COMPAT:

_OPENAI_COMPAT = {
    "openai":     (None,                                "OPENAI_API_KEY",     "OPENAI_MODEL"),
    "openrouter": ("https://openrouter.ai/api/v1",      "OPENROUTER_API_KEY", "OPENROUTER_MODEL"),
    "groq":       ("https://api.groq.com/openai/v1",    "GROQ_API_KEY",       "GROQ_MODEL"),
    "cerebras":   ("https://api.cerebras.ai/v1",        "CEREBRAS_API_KEY",   "CEREBRAS_MODEL"),
    "nvidia":     ("https://integrate.api.nvidia.com/v1","NVIDIA_API_KEY",    "NVIDIA_MODEL"),
    "mistral":    ("https://api.mistral.ai/v1",         "MISTRAL_API_KEY",    "MISTRAL_MODEL"),
    "github":     ("https://models.github.ai/inference","GITHUB_TOKEN",       "GITHUB_MODEL"),
    "gemini":     ("https://generativelanguage.googleapis.com/v1beta/openai/",
                   "GEMINI_API_KEY", "GEMINI_MODEL"),
}

glm, claude, and ollama have small dedicated builders for their non-standard config. Reordering or extending the chain is a config line, never a code change — adding a new OpenAI-compatible provider is one new row in that dict.

Credentials resolve here, and only here, from the broker's own environment. The builders hand the resolved key to the adapters, which are pure (config in, no env reads). The credential's life is: env var → builder → adapter constructor → the vendor SDK client. It never enters a Job, Completion, or Envelope.

build_chain fails fast. An unknown preset name, an empty PROVIDER_CHAIN, or a preset whose required key/model env var is missing raises at startup — a misconfigured chain must never start silently degraded. Blank entries ("glm,,openai") and duplicate presets ("glm,glm", which would just waste a fallback slot on the same provider) are dropped with a warning rather than swallowed silently.

Retry — one transient retry per provider

shelldon/broker/broker.py::handle_job(job, provider) is the per-provider unit. It calls the provider, and on a TransientProviderError it retries exactly once after a small backoff:

_MAX_ATTEMPTS = 2       # first attempt + one retry
_RETRY_BACKOFF_S = 0.5  # pause before the retry so we don't hammer a rate-limited endpoint

The outcomes:

• success → Completion(ok=True, payload=text)
• transient error, then success on retry → success
• transient error twice → Completion(ok=False, ...)
• PermanentProviderError → Completion(ok=False, ...) immediately, no retry (a 4xx won't fix itself in 0.5 s)
• any unexpected exception → caught and converted to Completion(ok=False, error="unexpected provider error: <Type>") — never an exception across the bus

The backoff constant is module-level so tests can zero it; the suite never does a real wall-clock wait.

Fallback — advance through the chain

shelldon/broker/broker.py::handle_job_chain(job, chain) wraps handle_job and is the single thing that makes a dead provider not kill your turn:

async def handle_job_chain(job, chain):
    completion = Completion(ok=False, error="empty provider chain")
    for fallbacks, provider in enumerate(chain):
        completion = await handle_job(job, provider)
        if completion.ok:
            log.info("turn answered by provider %r (after %d fallback(s))", provider.name, fallbacks)
            return completion
        log.warning("provider %r failed, advancing: %s", provider.name, completion.error)
    return completion

Behavior:

• First success wins. As soon as a provider returns ok=True, that Completion is returned and the rest of the chain is never touched.
• Advance on any failure. The loop keys only on completion.ok — it advances on a transient-exhausted failure and on a permanent 4xx. If provider A has a bad key or a wrong model id, provider B might still answer. The goal is a completed turn.
• Sequential, not parallel. Providers are tried in order, one at a time. There is no "race for first answer" (that would multiply spend).
• Audit trail without secrets. On success the answering provider is logged by its preset name ("glm", "ollama") and the number of fallbacks. Every failure is logged at WARNING with its error text. The name is a config label, never a credential, so this audit record is safe.
• Exhaustion returns the last failure Completion. When every provider failed, the last failure is returned. This terminal failure is the handoff to graceful degradation below.

The bus loop

shelldon/broker/service.py::run_broker(socket_path, chain) is the broker's whole runtime. It connects to the bus hub as Actor.BROKER, then loops: read a JOB envelope, run it through handle_job_chain, and write the resulting Completion back. Two details matter:

1. The broker returns a Completion to the WORKER, not a Result to core. The broker is a pure egress boundary — it produces raw model output (text plus any normalized tool calls) and does no pet-domain parsing. The worker is what interprets the reply, runs the tool loop, and emits the Result that core consumes. The turn_id is echoed so the worker/core fence can correlate the answer to the question.

2. It is built to survive. A malformed frame is skipped; a framing error or a vanished hub ends the current connection cleanly. The initial connect is timeout-bounded (_CONNECT_TIMEOUT_S = 5.0) so a hung hub can't block forever, and run_broker(reconnect=True) (the default) wraps connect→serve in a backoff-and-retry loop so a transient hub drop or restart doesn't kill the broker permanently. A deliberate shutdown cancels the task and the loop exits cleanly — cancellation always wins over the reconnect loop.

See How a Turn Works for where this sits in a full turn.

Graceful degradation — reflex-only when the whole chain fails

The whole point of the chain is the v1 pain it fixes: a single GLM 500 or a full network outage used to kill a turn or freeze the pet. Now, when handle_job_chain exhausts every provider, it returns a terminal failure Completion. Core (not the broker) turns that into a degrade:

• Core's _degrade() (shelldon/core/runtime.py) sends DEGRADE_TEXT — a short "…can't think right now…" reply — over the outbound channel, and pushes a FACE_DEGRADED ("cant-think") expression to the E-Ink display.
• The same path fires on a turn timeout, so an offline outage that returns a fast failure degrades promptly — it does not wait out the full turn timeout.
• The process keeps running. The bus loop, core, and arbiter all stay alive, ready for the next message. A full outage makes shelldon quiet, not frozen or crashed.

Recovery is automatic and unlatched. There is no "degraded mode" flag anywhere, no circuit breaker, no health-check timer. After a degrade, the arbiter releases its slot and the turn fence clears. The next message starts a fresh turn that re-attempts the whole chain from the top. The instant any provider answers again, the next turn completes normally with real model text. Degradation is structural — a property of each turn being an independent re-attempt — not a feature with state to manage.

Native function-calling (tool use)

When a Job carries tools, handle_job routes to provider.complete_with_tools(messages, tools) instead of the plain complete(prompt) path. This is how shelldon's self-written tools get invoked (see Self-Coding). The two wire formats express tool use very differently, so each adapter normalizes the provider's native format into one closed contract — a Completion carrying ToolCall structs:

• Anthropic uses tool_use content blocks; tool results ride back on a user turn as tool_result blocks. normalize_anthropic_response joins text blocks into payload and turns each tool_use block into a ToolCall.
• OpenAI uses a tool_calls array on the assistant message, with arguments as a JSON string; tool results come back as role="tool" messages keyed by tool_call_id. normalize_openai_response parses each arguments string into a dict. Malformed model-emitted JSON is mapped to PermanentProviderError (it's bad-request-shaped, not retryable).

Because both adapters emit the same normalized Completion, the worker's tool loop is provider-agnostic — it never sees an Anthropic block or an OpenAI tool_calls array. The error taxonomy is shared too: complete_with_tools raises the same TransientProviderError / PermanentProviderError, so retry and fallback work identically for tool-use turns.

The credential vault

shelldon/broker/vault.py is a related but distinct egress authority. Where the chain holds provider credentials, the vault is the broker-only gate for surfacing the owner's secrets (e.g. a stored value the pet might be asked to recall). Because the broker is the privileged actor, it (and only it) holds the authorized path to read vault/. The worker — the untrusted brain — has no vault-read API, and on a dropped uid it has no OS permission to read the files either.

surface_vault(memory_root, key) is the sole authorized read. It is a read-and-authorize gate, not an LLM-output parser: it gates on a conservative single-segment key (ASCII word chars and - only, via _SAFE_KEY_RE), which forbids path separators and .. by construction so a constructed path can never escape vault/. A rejected key, a missing file, or a corrupt/unreadable file all return None rather than raising into the egress path.

Summary

Concern	Mechanism	File
Sole credential holder + sole egress	separate process, no keys on the bus, import-linter	broker/ (AD-2/AD-1)
Vendor-agnostic interface	LLMProvider Protocol + two error types	provider.py
Two wire formats	Anthropic-format / OpenAI-compatible adapters	anthropic_provider.py, openai_provider.py
Ordered, config-driven chain	PROVIDER_CHAIN → build_chain	chain.py
Retry once on transient error	handle_job	broker.py
Automatic fallback through the chain	handle_job_chain	broker.py
Survive hub drops	timeout + reconnect loop	service.py
Degrade to reflex when all fail	terminal failure Completion → core _degrade()	service.py → core/runtime.py
Native tool use, normalized	complete_with_tools + normalize_*_response	adapters
Owner-secret surfacing	broker-only authorized read	vault.py