Zero-install · On-device ML inference · Privacy-first · Real-time biomechanical analysis
A browser-native clinical decision-support tool that runs a deep learning pose estimation model entirely on-device to automatically score sitting balance under the Scale for the Assessment and Rating of Ataxia (SARA), Item 3 — with no data ever leaving the patient's machine.
Webcam feed
│
▼
┌─────────────────────────────────────────────────────┐
│ MediaPipe Pose (BlazePose neural network) │
│ • Two-stage detector + tracker │
│ • 33 anatomical landmarks, sub-pixel precision │
│ • Runs as compiled WebAssembly (WASM) in-browser │
└─────────────────────────────────────────────────────┘
│ landmark stream (x, y, z, visibility per joint)
▼
┌─────────────────────────────────────────────────────┐
│ Biomechanical Feature Extraction │
│ • Bilateral shoulder midpoint trajectory │
│ • Radial displacement time series │
│ • Sway duty cycle · RMS · SD · max deviation │
└─────────────────────────────────────────────────────┘
│ feature vector
▼
┌─────────────────────────────────────────────────────┐
│ Rule-Based Clinical Scoring Engine │
│ • Evidence-based thresholds (SARA 2006) │
│ • Deterministic, fully auditable decision tree │
│ • Produces SARA score 0–4 with reasoning │
└─────────────────────────────────────────────────────┘
│
▼
Score + Research Export (CSV / PNG)
The core of CeMoQu is Google's BlazePose, a state-of-the-art convolutional neural network (CNN) purpose-built for human pose estimation in real-time video streams.
| Property | Detail |
|---|---|
| Architecture | Two-stage CNN: lightweight detector + high-fidelity landmark regressor |
| Landmark output | 33 anatomical keypoints with (x, y, z, visibility) per point |
| Inference runtime | Compiled to WebAssembly (WASM) — runs at native speed inside the browser |
| Hardware acceleration | Automatically leverages WebGL GPU acceleration when available |
| Model complexity | Set to 1 (full-body, balanced accuracy/speed) |
| Temporal smoothing | Enabled — exponential smoothing across frames for stable landmark tracking |
| No server required | All neural network inference executes on the client CPU/GPU |
BlazePose achieves sub-pixel landmark accuracy on upper-body keypoints, making it suitable for clinical biomechanical measurement tasks.
Raw landmark coordinates are transformed into clinically meaningful postural metrics using the following pipeline:
The shoulder midpoint M = (L₁₁ + L₁₂) / 2 — the centroid of landmarks 11 and 12 — serves as the trunk position proxy. This is converted from normalised image coordinates to real-world centimetres using a pixel-per-cm (PPC) calibration factor.
| Metric | Formula | Clinical Meaning |
|---|---|---|
| Max radial sway | max(‖pᵢ − μ‖) |
Peak trunk displacement from mean position |
| Mean radial sway | mean(‖pᵢ − μ‖) |
Average postural offset |
| Sway SD | σ(‖pᵢ − μ‖) |
Variability / irregularity of sway |
| Sway duty cycle | frames with ‖pᵢ − μ‖ > 1 cm / N |
Fraction of time with active sway |
| Max inter-frame shift | max(‖pᵢ − pᵢ₋₁‖) |
Largest instantaneous lurch |
Two rule-based detectors run in parallel during the trial:
- Support event detector — flags a sudden displacement ≥ 8 cm between consecutive frames as a grab/support event (single-frame velocity threshold)
- Tracking loss detector — declares landmark dropout after 15 consecutive frames below the visibility confidence threshold (
visibility < 0.4)
The scoring logic is a deterministic, fully explainable decision tree grounded in the original 2006 SARA publication. Every branch is surfaced in the UI.
Duration achieved AND tracking maintained?
├── No → Score 4 (cannot maintain 10 s)
└── Yes
└── Support event detected OR max sway ≥ 10 cm?
├── Yes → Score 3 (needs support / large deviation)
└── No
└── Max sway < 1 cm?
├── Yes → Score 0 (normal, no sway)
└── No
└── Sway duty cycle ≥ 60%?
├── Yes → Score 2 (constant sway)
└── No → Score 1 (intermittent sway)
| Score | Label | Key Criterion |
|---|---|---|
| 0 | Normal — no sway | Max radial displacement < 1 cm |
| 1 | Slight intermittent sway | Sway duty cycle < 60% |
| 2 | Constant sway, no support | Sway duty cycle ≥ 60% |
| 3 | Needs intermittent support | Support event detected or deviation ≥ 10 cm |
| 4 | Cannot maintain 10 s | Duration not reached or tracking lost |
| Feature | Details |
|---|---|
| On-device ML inference | BlazePose WASM runs fully client-side, zero network calls for AI |
| Real-time landmark tracking | 33-point skeletal model at up to 30 fps |
| Pixel-accurate calibration | Shoulder-width or monitor-geometry PPC estimation |
| Explainable scoring | Live decision tree panel — every threshold visible |
| Research-grade export | Per-frame CSV (x, y, visibility) · summary CSV · sway PNG · score history PNG |
| Privacy by design | No backend, no analytics, no cookies — all compute and storage is local |
| Zero-dependency runtime | Pure browser APIs (Canvas 2D, WebGL, Blob, MediaDevices) |
| Layer | Technology |
|---|---|
| ML inference | MediaPipe BlazePose (TFLite model compiled to WASM) |
| GPU acceleration | WebGL 2.0 via MediaPipe's WASM-GPU backend |
| Rendering | HTML5 Canvas 2D API — mirrored video + landmark overlay |
| Charting | Chart.js 4.4 — score history line chart + trajectory canvas |
| Camera pipeline | MediaDevices.getUserMedia → requestAnimationFrame render loop |
| Export | Blob API + dynamically generated <a download> — no server involved |
| Audio | Web Audio API (OscillatorNode) — countdown beeps synthesised in-browser |
- Chrome or Edge (WebGL + WASM support required)
- Webcam
- Python 3
git clone <repo-url>
cd cemoqusitting
python3 -m http.server 8080Open http://localhost:8080 in your browser.
Note on WASM/CORS: MediaPipe loads its neural network weights and WASM binary via CDN. Browsers enforce CORS on
file://origins, blocking these fetches. A single-line Python server provides a properhttp://origin with zero configuration.
- Start Camera — webcam access is requested; the BlazePose WASM binary downloads and compiles (~5 s first load, cached thereafter).
- Calibrate — enter the patient's shoulder width and click Calibrate for a 3-second landmark-based PPC measurement, or use the monitor-geometry estimator.
- Position the patient — seated, feet unsupported, arms outstretched, eyes open.
- Run the test — 3-2-1 countdown, then 10 seconds of continuous ML-driven pose tracking.
- Inspect the result — SARA score 0–4 with full decision audit. Switch to Research tab for charts and per-frame data export.
index.html Markup only
style.css Design system (CSS custom properties + layout)
js/
config.js Constants, thresholds, shared runtime state
audio.js Web Audio API beep synthesiser
ui.js DOM helpers, tab router
scoring.js Feature extraction + SARA decision engine
draw.js requestAnimationFrame render loop + per-frame handler
camera.js getUserMedia + MediaPipe Pose initialisation
calibration.js PPC calibration (landmark-based + geometric fallback)
test.js Trial lifecycle — start / stop / finalize
dashboard.js Research panel, Chart.js, CSV/PNG export pipeline
guide.js 6-step animated onboarding walkthrough
Schmitz-Hübsch T, et al. (2006). Scale for the assessment and rating of ataxia: Development of a new clinical scale. Neurology, 66(11), 1717–1720.
Bazarevsky V, et al. (2020). BlazePose: On-device Real-time Body Pose Tracking. arXiv:2006.10204
MIT