Stabilo Platform

Medical VR rehabilitation system. Patient stands on a motorised balance platform wearing a VR headset. Two USB cameras track body posture. A clinician controls the session remotely from a mobile app over Tailscale. Platform motion is synchronised to the VR game by an ESP32-S3 MCU.

Quick start

1 — Install dependencies (once)

# Python services
cd services/mcu-bridge   && python3 -m venv .venv && .venv/bin/pip install -e ".[dev]" && cd ../..
cd services/pose-tracker && python3 -m venv .venv && .venv/bin/pip install -e ".[dev]" && cd ../..
cd libs/stabilo-bus-py   && python3 -m venv .venv && .venv/bin/pip install -e ".[dev]" && \
                            .venv/bin/pip install websockets && cd ../..
cd services/mocks        && python3 -m venv .venv && .venv/bin/pip install pyzmq msgpack \
                         && .venv/bin/pip install -e ../../libs/stabilo-bus-py && cd ../..

# Flutter apps
cd apps/display-monitor && flutter pub get && cd ../..
cd apps/doctor-mobile   && flutter pub get && cd ../..
cd apps/main-desktop    && flutter pub get && cd ../..

2 — Run (mock mode, no hardware)

cd apps/main-desktop && flutter run -d macos

In the app: Setup → Backend Services → Start All. All 6 services start. Then launch the display and doctor apps in separate terminals:

cd apps/display-monitor && flutter run -d macos
cd apps/doctor-mobile   && flutter run -d macos

3 — Test

# Python unit tests
cd services/pose-tracker && .venv/bin/pytest -q   # 31 tests
cd services/mcu-bridge   && .venv/bin/pytest -q   # 48 tests

# Flutter unit tests
cd apps/main-desktop && flutter test              # 37 tests

# Integration smoke test (requires services running)
libs/stabilo-bus-py/.venv/bin/python services/smoke_test.py

Architecture

[2× USB cameras] ──► pose-tracker ─┐
                                    ├──► ZMQ proxy :5550/5551 ──► zmq_ws_bridge :8765 ──► Flutter apps
[ESP32-S3 MCU] ── USB ──► mcu-bridge ─┘                                              └──► Unity VR

All processing runs on one host machine. Doctor's phone connects over Tailscale. No cloud.

Component	Tech	Role
apps/main-desktop	Flutter macOS	Session orchestrator + Setup GUI
apps/display-monitor	Flutter macOS	Live CoG dot + COCO-17 skeleton
apps/doctor-mobile	Flutter macOS/iOS	CoG dial + session controls
apps/vr-game	Unity + NetMQ	VR rehabilitation game
services/pose-tracker	Python, OpenCV	Stereo 3D pose estimation (60 fps)
services/mcu-bridge	Python, pyserial	ESP32-S3 serial ↔ ZMQ bridge
services/zmq_ws_bridge.py	Python, asyncio	ZMQ → WebSocket for Flutter
firmware/platform-mcu	ESP32-S3 / ESP-IDF	Load cells + motor driver
libs/stabilo-bus-*	Py / Dart / C#	Shared bus client libraries

Key docs

Doc	What
docs/runbook.md	Launch guide, troubleshooting
docs/camera-calibration.md	Camera setup workflow
docs/stabilo-platform-technical.pdf	Full technical reference with architecture diagrams
protocol/	Message contracts (ZMQ topics, MCU serial protocol)

Repository layout

apps/           Flutter + Unity apps
services/       Python backend services + mocks
firmware/       ESP32-S3 firmware (ESP-IDF)
libs/           Shared bus libraries (Python, Dart, C#)
protocol/       Message schemas and contracts
docs/           Runbook, calibration guide, technical reference
infra/          Mosquitto + Tailscale config
legacy/         Archived prototypes — do not edit

Real hardware

• Cameras: follow docs/camera-calibration.md (or use the Setup tab in main-desktop)
• MCU: flash firmware/platform-mcu, then mcu-bridge --port /dev/tty.usbmodem*
• Full technical reference: docs/stabilo-platform-technical.pdf