Running mycelium on a Raspberry Pi 3B+ — notes on tiny hardware support

[juliarvalenti]

TL;DR

I spent a session trying to get mycelium running on a Raspberry Pi 3B+ (1GB RAM, armv7). It mostly works, with one hard blocker: no ML inference library ships armv7 wheels, so local embeddings are a dead end on 32-bit ARM.

Here are some notes on what would make mycelium more viable on constrained or edge hardware.

---

What worked

• PostgreSQL 15 + pgvector compiled from source ✓
• Python 3.11 via Bookworm ✓
• uv installed fine (armv7 musl binary available)
• uv sync resolves and the lockfile is solid
• Repo clones cleanly, project structure is intuitive

The hard blocker: embeddings

Both sentence-transformers (depends on PyTorch) and fastembed (depends on onnxruntime) have no armv7 wheels. Neither ships 32-bit ARM support. This means memory search is completely unavailable on armv7 hardware.

On arm64 (Pi 4/5 with 64-bit OS), both have wheels — so this is specific to 32-bit.

Ideas for better small-hardware support

1. Pluggable embedding providers

The biggest win would be making the embedding backend swappable in config:

[embeddings]
provider = "openai"  # or "cohere", "local", "none"
model = "text-embedding-3-small"

This lets edge nodes offload embeddings to a remote API and keep the local footprint tiny. The memory filesystem and coordination engine work fine without local inference.

2. Graceful degradation when embeddings are unavailable

Right now if the embedding model fails to load, the server errors. It would be nicer if memory search returned a clear error (search unavailable: no embedding provider configured) while all other commands kept working. memory set/get/ls, rooms, sessions, and coordination don't need embeddings at all.

3. Optional lightweight local option

If local inference is desired on edge hardware, a tiny quantized model via llama.cpp or pre-quantized ONNX could work within 1GB. Not a priority, but worth noting.

Memory footprint without embeddings

With the embedding model not loaded, the stack is surprisingly lean:

• PostgreSQL 15: ~180MB idle
• FastAPI backend (no model): ~80MB
• Total: ~260MB — plenty of headroom on 1GB RAM

The bottleneck is purely the embedding model, not the coordination engine.

---

The core architecture — filesystem memory, pgvector search index, rooms, sessions — is a genuinely good fit for edge/embedded use. The main ask is making the embedding layer optional so the rest of the stack can run anywhere.