Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ irm https://raw.githubusercontent.com/CambrianTech/continuum/main/install.ps1 |

One command -- bootstraps WSL2 + Docker Desktop via winget if missing, auto-toggles the Docker Desktop AI settings (no manual GPU + TCP toggle anymore), drops a `continuum.cmd` on PATH, then hands off to `bootstrap.sh` inside WSL. Works from the default Windows PowerShell 5.1 (it bootstraps pwsh 7 only if needed).

`setup.sh` pulls our forged Qwen3.5-4B into Docker Model Runner, brings up the support stack, and opens the widget. **One required manual step**: in Docker Desktop → Settings → AI, enable both *GPU-backed inference* and *host-side TCP support* — without these, the model runs CPU-tier even with a GPU present. See **[docs/SETUP.md](docs/SETUP.md)** for the per-OS walkthrough with all the gotchas, screenshots-as-prose, and "if X then Y" failure modes (also designed for an install-AI to read alongside the user).
`setup.sh` pulls our forged Qwen3.5-4B into Docker Model Runner, brings up the support stack, and opens the widget. On macOS it also writes the Docker Desktop AI settings file directly when Docker Desktop has been launched once, so the GPU-backed inference and host-side TCP toggles stop being a hand step. See **[docs/SETUP.md](docs/SETUP.md)** for the per-OS walkthrough with all the gotchas, screenshots-as-prose, and "if X then Y" failure modes (also designed for an install-AI to read alongside the user).

<details>
<summary>Development (from source)</summary>
Expand Down
10 changes: 5 additions & 5 deletions docs/INSTALL-ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ How continuum's installers stay maintainable across macOS, Linux, and Windows wi

## Goal

A first-time dev on any supported OS runs **one command** in their default shell and ends up with continuum running locally + a `continuum` command on PATH. Zero manual steps after that one command. No "now also do X in Docker Desktop settings."
A first-time dev on any supported OS runs **one command** in their default shell and ends up with continuum running locally + a `continuum` command on PATH. Zero manual Docker Desktop settings steps after that one command. If Docker Desktop has never been launched on the machine, the installer may ask for that first launch/EULA so the settings store exists.

## The challenge

Expand Down Expand Up @@ -90,10 +90,10 @@ and the small entry-point surface meant the check was cheap.

Today's `setup.bat` + `bootstrap.ps1` together leave these gaps:

- **Docker Desktop AI settings are a manual step.** The README says
"enable GPU-backed inference + host-side TCP support" — every fresh
dev hits this. The new install.ps1 (and install.sh) writes the
settings.json directly + bounces Docker Desktop. Zero manual toggles.
- **Docker Desktop AI settings are auto-written.** The installer writes
the Docker Desktop settings file directly and bounces Docker Desktop.
The only first-run caveat is that Docker Desktop must have launched at
least once so the settings store exists.
- **`setup.bat` infinite `wait_loop`** on widget-server health (no
timeout). Replaced with a bounded wait + actionable failure message.
- **`setup.bat` relative-path quirks** in the WSL handoff (`cp src/...`
Expand Down
38 changes: 13 additions & 25 deletions docs/SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

## What you'll have running

After `curl install.sh | bash` completes (and the per-OS manual steps below):
After `curl install.sh | bash` completes (and any first-time Docker Desktop launch / reboot your OS asks for):

- A continuum widget at `http://localhost:9003`
- Default rooms: General, Pantheon, Code, Factory, Academy
Expand All @@ -26,7 +26,7 @@ If you've used Ollama or LM Studio: continuum is the next layer — multi-person
- [**Linux + Nvidia**](#linux--nvidia) — RTX 30/40/50, native Docker
- [**Linux + AMD / Intel GPU**](#linux--amd--intel-vulkan) — Vulkan path (experimental in this PR scope)

Each section: **prereqs → curl install → required manual steps → success check → if it breaks**.
Each section: **prereqs → curl install → Docker Desktop initialization → success check → if it breaks**.

---

Expand All @@ -48,15 +48,9 @@ curl -fsSL https://raw.githubusercontent.com/CambrianTech/continuum/main/src/scr

Pulls images, pulls the forged Qwen3.5 model into Docker Model Runner, starts the support stack, and launches `continuum-core` natively (Metal for Candle, Bevy, vision, audio).

### Required manual step (one-time, ~30 seconds)
### Docker Desktop initialization

**Docker Desktop → Settings → AI:**

1. Check **Enable GPU-backed inference** (lights up Metal for Docker Model Runner — without this, you get CPU speed and a slow first impression)
2. Check **Enable host-side TCP support** (port `12434`, default — required so the continuum core container can reach DMR on the host)
3. Click **Apply**

Docker Desktop will swap the inference backend to `llama.cpp latest-metal` automatically. **No restart required.**
The installer writes Docker Desktop's AI settings directly once Docker Desktop has been launched at least once and the settings store exists. If this is a brand-new Docker Desktop install, open Docker Desktop once, accept the EULA, then rerun the installer. After that, the GPU-backed inference and host-side TCP toggles are applied automatically.

### Success check

Expand All @@ -70,8 +64,8 @@ Then open `http://localhost:9003`, send "hello" in the General room, and Helper

### If it breaks

- **Personas reply slowly (under 15 tok/s):** the AI toggles weren't applied. Re-check Settings → AI.
- **`docker model status` says `latest-cpu` instead of `latest-metal`:** the GPU-backed inference toggle is off. Toggle it, click Apply, re-check.
- **Personas reply slowly (under 15 tok/s):** Docker Desktop was not initialized far enough for the settings write to land. Launch Docker Desktop once, accept the EULA, rerun the installer, then re-check.
- **`docker model status` says `latest-cpu` instead of `latest-metal`:** the GPU-backed inference toggle did not apply. Re-run the installer after Docker Desktop has a writable settings store.
- **Widget loads but no personas reply:** check `~/.continuum/jtag/logs/system/daemons/AIProviderDaemonServer.log` for routing errors. Most likely the AI provider daemon needs the host-side TCP toggle.
- **Clean reset:** `docker compose down && docker compose up -d` then re-run `curl install.sh`.

Expand All @@ -89,9 +83,9 @@ Then open `http://localhost:9003`, send "hello" in the General room, and Helper
- WSL2 with an Ubuntu distro installed (`wsl --install -d Ubuntu` from PowerShell)
- ~10 GB free disk

### Required manual steps (one-time, ~5 minutes)
### Docker Desktop + WSL initialization

These are not skippable — defaults will leave you running on CPU at ~10 tok/s instead of GPU at ~237 tok/s, or fail to start altogether.
These are not skippable — defaults will leave you running on CPU at ~10 tok/s instead of GPU at ~237 tok/s, or fail to start altogether. The installer writes the Docker Desktop AI settings directly once Docker Desktop has a writable settings store; if Docker Desktop has never been launched on this machine, open it once and rerun the installer after the first-run EULA completes.

#### 1. Configure WSL2

Expand Down Expand Up @@ -121,15 +115,9 @@ wsl --shutdown

WSL will cold-launch with the new config on the next Docker Desktop startup.

#### 2. Enable Docker Desktop AI features

**Docker Desktop → Settings → AI:**

1. Check **Enable GPU-backed inference** (swaps `llama.cpp latest-cpu` → `latest-cuda` automatically — without this, you're on CPU)
2. Check **Enable host-side TCP support** (port `12434` default — required so containers can reach DMR)
3. Click **Apply**
#### 2. Docker Desktop AI settings

Docker Desktop installs the CUDA backend on Apply. **You may see a "WSL integration unexpectedly stopped" dialog with error `Wsl/Service/0x8007274c`** — this is `WSAETIMEDOUT` on the WSL distro initialization. Click **Restart the WSL integration**. If the same error recurs, run `wsl --shutdown` from an admin PowerShell, then click Restart again. The hard reset is sometimes required because the integration restart only re-runs Docker plumbing inside the existing VM, not the VM itself.
The installer writes **Enable GPU-backed inference** and **Enable host-side TCP support** into Docker Desktop automatically once the settings store exists. If Docker Desktop has never been launched on the machine, start it once, accept the EULA, and rerun the installer so the settings file exists. If Docker Desktop shows a "WSL integration unexpectedly stopped" dialog with error `Wsl/Service/0x8007274c`, click **Restart the WSL integration**. If the same error recurs, run `wsl --shutdown` from an admin PowerShell, then click Restart again. The hard reset is sometimes required because the integration restart only re-runs Docker plumbing inside the existing VM, not the VM itself.

### Install

Expand Down Expand Up @@ -166,8 +154,8 @@ While inference runs, you should see GPU utilization spike to 70%+ and memory gr
### If it breaks

- **"WSL integration unexpectedly stopped" loop:** `wsl --shutdown` from admin PowerShell. The Restart-the-WSL-integration button is not the same as `wsl --shutdown` — the latter is the actual VM hard-reset.
- **`docker model status` says `latest-cpu`:** the GPU toggle is off, or Docker Desktop hasn't finished installing the CUDA backend. Re-check Settings → AI, click Apply, wait 60 seconds.
- **Personas reply but `nvidia-smi` shows no activity:** the host-side TCP toggle is off. The container can't reach DMR; it's likely silently routing to a CPU path. Toggle it on.
- **`docker model status` says `latest-cpu`:** Docker Desktop hasn't finished applying the AI settings yet. Re-run the installer after Docker Desktop has a writable settings store, then wait 60 seconds.
- **Personas reply but `nvidia-smi` shows no activity:** the host-side TCP setting did not apply. Re-run the installer after Docker Desktop has a writable settings store.
- **Build fails with apt timeouts:** WSL networking issue, often resolved by `--network=host` or by `wsl --shutdown` to reset DNS. See [docs/infrastructure/WINDOWS-WSL2-INSTALL-GUIDE.md](infrastructure/WINDOWS-WSL2-INSTALL-GUIDE.md) for the full playbook.

---
Expand Down Expand Up @@ -269,7 +257,7 @@ Verifies submodules, IPC sockets, GPU vs CPU backend, scheduler vs llama-server,
If you're a Claude / Codex / similar walking a user through this doc, the failure modes above are written to be pattern-matchable. Specific cues:

- **`Wsl/Service/0x8007274c`** in a Docker Desktop dialog → `wsl --shutdown` from admin PowerShell, then click Restart the WSL integration
- **`predicted_per_second` < 15 on Mac for a 4B model** → AI toggles not applied; have user open Docker Desktop → Settings → AI and check both boxes
- **`predicted_per_second` < 15 on Mac for a 4B model** → Docker Desktop has not been initialized enough for the settings write to land; launch Docker Desktop once, accept the EULA, rerun the installer
- **`docker model status` shows `latest-cpu`** on a Nvidia/Mac box that should have GPU acceleration → same toggle issue
- **`Appears stuck (Nseconds since last success)`** in `AIProviderDaemonServer.log` → most likely a stale-metric warning; verify by sending a chat and confirming the persona replies (the metric is a lagging health probe, not a definitive failure signal)
- **Personas reply with stale provider routing (Candle CPU instead of DMR)** → docker container image is pre-`cfe2a4316`; pull `:pr-891` (or `:latest` post-merge) and restart `docker compose up -d`
7 changes: 3 additions & 4 deletions install.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -106,10 +106,9 @@ Install-WSL2
# ── section: docker desktop AI settings auto-toggle ─────────────────────
# Highest-leverage friction kill. Without these toggles continuum's
# personas run on CPU at ~10 tok/s instead of GPU at ~80-237 tok/s, OR
# the core container can't reach Docker Model Runner at all. Today the
# README has these as a "manual one-time step" and every fresh dev hits
# it. Programmatically write the keys + bounce Docker Desktop so the
# user never has to think about it.
# the core container can't reach Docker Model Runner at all. Write the
# keys programmatically + bounce Docker Desktop so the user never has to
# think about it.
#
# Key reference (from inspecting %APPDATA%\Docker\settings-store.json
# on a real Docker Desktop 4.x install with both toggles set):
Expand Down
52 changes: 48 additions & 4 deletions setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,51 @@ print(' Updated: memoryMiB=${TARGET_MEM_MIB}, cpus=${TARGET_CPUS}')
fi
fi

# ── Enable Docker Desktop AI settings ──────────────────────
# The Windows installer already writes these keys directly. Do the same on
# macOS so the release path doesn't leave GPU-backed inference and host TCP
# to a hand flip in Docker Desktop.
if [ -n "${DD_FILE:-}" ] && [ -f "$DD_FILE" ]; then
AI_SETTINGS_STATUS=$(
python3 -c "
import json, os, shutil
path = os.path.expanduser('$DD_FILE')
with open(path) as f:
cfg = json.load(f)
changed = False
for key in ('EnableDockerAI', 'EnableInferenceGPUVariant', 'EnableInferenceTCP'):
if cfg.get(key) is not True:
cfg[key] = True
changed = True
if changed:
shutil.copy2(path, path + '.continuum-bak')
with open(path, 'w') as f:
json.dump(cfg, f, indent=2)
print('changed')
else:
print('already')
"
)

if [ "$AI_SETTINGS_STATUS" = "changed" ]; then
echo " Docker Desktop AI settings enabled (GPU-backed inference + host-side TCP)"
echo " Restarting Docker Desktop so the toggles apply ..."
docker desktop restart >/dev/null 2>&1 || true
for _ in $(seq 1 30); do
if docker info &>/dev/null 2>&1; then break; fi
sleep 4
done
if ! docker info &>/dev/null 2>&1; then
echo " Warning: Docker Desktop did not come back cleanly after the AI-toggle restart."
fi
else
echo " Docker Desktop AI settings already enabled (GPU + host TCP)"
fi
elif [[ "$PLATFORM" == "mac" ]]; then
echo " Docker Desktop AI settings file not found yet."
echo " Launch Docker Desktop once, accept the EULA, then re-run this script."
fi

# ── Install continuum CLI ─────────────────────────
INSTALL_DIR="${HOME}/.local/bin"
mkdir -p "$INSTALL_DIR"
Expand Down Expand Up @@ -300,10 +345,9 @@ if command -v docker &>/dev/null && docker model --help &>/dev/null 2>&1; then
# DMR runs the model on CPU even with a GPU present — fast machine, slow
# first chat, "Continuum feels broken" review.
echo ""
echo " ℹ️ Manual one-time step: enable GPU acceleration in Docker Desktop"
echo " Settings → AI → ✓ Enable GPU-backed inference"
echo " ✓ Enable host-side TCP support (port 12434)"
echo " Without these, inference runs on CPU. See docs/SETUP.md for details."
echo " ℹ️ Docker Desktop AI settings are auto-enabled when Docker Desktop has"
echo " a settings store to write. If this is a fresh Docker Desktop install,"
echo " launch Docker Desktop once, accept the EULA, and rerun setup."
else
echo ""
echo " ⚠️ Docker Model Runner CLI not available."
Expand Down
Loading