Handstand MotionBERT analysis (Mac/CPU)

Static freestanding floor handstand analysis using 2D MMPose + MotionBERT 3D lift, canonical alignment (pelvis→X, hip-center→neck→Z), angle metrics, baseline vs pros, and GPT‑5 feedback.

Web UI quickstart

Use the React site + FastAPI to upload a video and get 2D annotated output, metrics, and AI coaching.

1. Backend API

source .venv/bin/activate
export PYTORCH_ENABLE_MPS_FALLBACK=1
# Choose one or both:
export OPENAI_API_KEY="..." ; export OPENAI_MODEL="gpt-5"
export GOOGLE_API_KEY="..."  # for Gemini, e.g. gemini-2.5-pro
uvicorn server.main:app --reload --host 127.0.0.1 --port 8000

2. Frontend (Vite React)

cd web
npm install
npm run dev

Open http://127.0.0.1:5173 and upload your MP4. Pick LLM (OpenAI/Gemini) in the dropdown.

Troubleshooting:

• 404 from the UI → ensure API runs on 127.0.0.1:8000 (proxy target in web/vite.config.js).
• Black 2D video → ensure ffmpeg is installed; output is H.264/yuv420p.
• LLM errors → verify OPENAI_API_KEY/GOOGLE_API_KEY are set in the API shell.

Quick start

1. Create a virtualenv (Mac, CPU or MPS):

python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
export PYTORCH_ENABLE_MPS_FALLBACK=1

2. Install MotionBERT repo (once), and set MOTIONBERT_DIR:

git clone https://github.com/Walter0807/MotionBERT.git models/motionbert_repo
export MOTIONBERT_DIR="$(pwd)/models/motionbert_repo"

3. Download ~100 pro handstand videos (adjust queries as needed):

python scripts/download_pro.py --max-downloads 120

4. Normalize videos (30 FPS, 720p) into data/*/trimmed:

python scripts/preprocess_trim.py --split pro

5. Extract 2D keypoints with MMPose (HRNet-W32 + RTMDet):

python scripts/extract_2d_mmpose.py --split pro

6. Convert to AlphaPose-like JSON and lift to 3D with MotionBERT:

python scripts/lift_3d_motionbert.py --split pro

7. Align skeletons and compute metrics:

python scripts/align_and_metrics.py --split pro

8. Build pro baseline:

python scripts/build_baseline.py

9. Analyze your clip:

python scripts/preprocess_trim.py --split user
python scripts/extract_2d_mmpose.py --split user
python scripts/lift_3d_motionbert.py --split user
python scripts/align_and_metrics.py --split user
python scripts/analyze_user_clip.py

10. GPT‑5 feedback (requires OPENAI_API_KEY):

python scripts/gpt_feedback.py

Web UI + API

Run the API (FastAPI) and the Vite React frontend.

API

source .venv/bin/activate
uvicorn server.main:app --reload --host 127.0.0.1 --port 8000

Frontend (Vite)

cd web
npm install
npm run dev

Open http://127.0.0.1:5173 — upload a video; the UI calls /api/analyze-sync and shows outputs.

Paths

• Data lives under data/{pro,user}/{raw,trimmed,keypoints2d,poses3d,aligned,metrics}.
• MotionBERT repo expected at models/motionbert_repo (configurable with MOTIONBERT_DIR).