MiKaPo 3.0 — 60 FPS real-time MMD motion capture
A ground-up rewrite of the solver and capture pipeline, authored by Claude (Fable 5). Headline: 60 FPS rendering with real-time capture — the render loop no longer shares a thread with detection.
Solver
- Data-driven core — one bone-definition table + generic direction/basis/twist solvers replace ~40 hand-written per-bone functions (~1,500 → ~500 lines). Verified bit-equivalent to the 2.0 solver on synthetic landmark suites, 1.7× faster, zero allocations per frame.
- reze-engine math — solver runs on Reze Engine's
Vec3/Quat; matrix inversion replaced by quaternion conjugation; parent chains computed once per frame. Babylon.js remains only in the lazily-loaded debug skeleton preview. - Roll witnesses — forearm/shin direction pins upper-arm/thigh roll, so elbow creases and knee planes orient correctly instead of being left to shortest-arc chance.
- Anatomical finger clamps — swing-twist decomposition per finger with human flexion/spread ranges; noisy landmark frames can no longer bend fingers backward.
- Visibility gating + hold-last-pose — occluded or off-frame limbs hold their pose instead of snapping to identity or chasing garbage.
Capture pipeline
- Web Worker detection — MediaPipe holistic runs off the main thread (frames shipped as transferred
ImageBitmap), so WebGPU rendering holds 60 FPS during capture. - Adaptive motion interpolation — pose tweens sized to the measured detection interval upsample ~30 Hz capture into smooth 60 FPS motion.
- Media-time smoothing — One-Euro filters run on video time; pause/seek resets cleanly instead of warping.
- Solve path runs outside React; per-model calibration happens once at load.
Face
- One-Euro smoothing per channel — fast blinks land at full depth.
- Morph alias resolution against each loaded model's morph list (e.g.
ワ → にっこり → にやり), so more custom models work unmodified.
Live demo: https://mikapo.vercel.app
🤖 Generated with Claude Code