mAI_Coach

Real-time motion capture for strength training.

mAI Coach is a research prototype that brings advanced computer vision to the weight room. By running Google's MediaPipe pose detection models directly on your iPhone, it provides instant feedback on your form—no cloud uploads, no latency, just results.

Accompanying the app is a powerful desktop suite for data curation, helping researchers label video datasets and fine-tune models to match real-world conditions.

Key Features

• Privacy-First AI: All pose estimation runs entirely on-device via MediaPipe. Your workout video never leaves your phone.
• Live Form Coaching: Real-time skeletal overlay with audio coaching cues for grip width, bar tilt, lockout, and depth issues.
• Wrist-Y Rep Detection: Automatic rep counting using streaming peak detection on wrist landmarks—87% accuracy within ±1 rep.
• Centering Guide: Dynamic calibration overlay that auto-dismisses when you're properly positioned.
• Full Workout Tracking: Custom workout plans, templates, exercise reordering, and real-time progress tracking.
• Superset Support: Group exercises into labeled superset blocks with instructions/notes.
• Rest Timer: Automatic rest timer triggered on set completion with configurable durations.
• Personal Records: Auto-detected PRs with trophy badges, confetti, and a dedicated PR board.
• Research-Grade Tooling: A unified PySide6 desktop application for annotating video, trimming clips, and training custom classification models.

Repository Structure

Path	Description
App Core/	The iOS Application. Xcode workspace, SwiftUI views, and MediaPipe inference.
Dev_tools/	The Researcher's Toolkit. Python/Qt app for labeling, visualization, and model training.
tests/	Automated test suite for the Python tools.
temp/	Scratch files (gitignored).
archive/	Archived files (gitignored).

Quick Start

📱 I want to run the iOS App

Building the app requires a Mac with Xcode 15+.

1. Navigate to App Core/.
2. Run pod install.
3. Open mAICoach.xcworkspace and hit Run on your device.

👉 See full iOS instructions

🖥️ I want to run the Developer Tools

The tools require Python 3.10+ and ffmpeg.

1. Run ./scripts/setup_unified_env.sh to build the environment.
2. Run ./scripts/run_unified_tool.sh to launch the GUI.

👉 See full Dev Tools instructions

Maintainers

• Travis Whitney (whitnetr@oregonstate.edu)
• Cole Seifert (seiferco@oregonstate.edu)

License

Proprietary. See LICENSE.md.