Kinect Exercise Tracker

A MediaPipe Pose exercise/rep counter that uses a Kinect's depth stream, not just RGB, to disambiguate movements that look identical in a 2D skeleton.

Why depth, not just pose angles

MediaPipe Pose gives you 2D joint angles from the RGB frame. That's enough to tell an elbow is bent, but it can't tell you which direction a limb moved in the third dimension — so a press (moving away from the camera) and a raise (moving up, same distance from the camera) can produce near-identical shoulder/elbow angles. This tracker samples the raw Kinect depth value (in mm) at the wrist, shoulder, hip, and knee landmarks to resolve that ambiguity, and uses it to gate 5 of the 9 exercises below.

Two concrete examples from the rule table:

• Barbell Bench Press vs. an angle-only false positive: the elbow angle alone (elbow_angle < 100) is shared with several other bent-arm poses. What actually confirms "press" is the wrist depth moving further from the shoulder than the RGB frame alone would show: wrist_depth > shoulder_depth + 150 (mm). Angle says "arm is bent near the chest"; depth says "and the hand is pushing away from the body," which is what makes it a press.
• Lateral Raises / Front Raises / Dumbbell Flyes: these three have overlapping shoulder-angle ranges (all involve raising the arm). Each requires abs(wrist_depth - shoulder_depth) < 200 (mm) — confirming the wrist stays roughly level with the shoulder in depth, i.e. the arm is moving in a lateral/frontal plane rather than pushing forward — before the angle thresholds are trusted to tell them apart.

Exercises and detection rules

All 9 exercises share one rep-counter state machine: an exercise starts in its "up" (or "down", for Shrugs) state, transitions on the first condition below, and counts a rep when the second condition is met.

Exercise	Start condition	Rep-count condition
Lateral Raises	shoulder angle > 75° (both arms), |wrist_depth − shoulder_depth| < 200mm	shoulder angle < 25° (both arms)
Front Raises	shoulder angle > 85°, elbow angle > 160° (both arms), |wrist_depth − shoulder_depth| < 200mm	shoulder angle < 25° (both arms)
Bicep Curls	elbow angle < 80° (both arms), shoulder angle ≈ 30° ± 20°	elbow angle > 170° (both arms)
Triceps Extensions	elbow angle < 70°, shoulder angle > 160° (both arms)	elbow angle > 170° (both arms)
Barbell Bench Press	wrist_depth > shoulder_depth + 150mm, elbow angle < 100° (both arms)	wrist_depth < shoulder_depth − 150mm (both arms)
Barbell Shrugs	shoulder_depth < hip_depth − 60mm (both sides)	shoulder_depth ≥ hip_depth − 20mm (both sides)
Dumbbell Squats	knee angle < 110° (both legs), hip_depth > knee_depth + 120mm	knee angle > 170° (both legs), hip_depth ≤ knee_depth + 60mm
Dumbbell Flyes	shoulder angle > 110° (both arms), |wrist_depth − shoulder_depth| < 200mm	shoulder angle < 25° (both arms)
Dumbbell Overhead Press	shoulder angle > 160°, elbow angle > 160° (both arms), wrist_depth < shoulder_depth − 100mm	shoulder angle < 80° (both arms)

Once a rep is counted, the tracker locks onto that exercise (up to 12 reps) before it will detect a different one — see Limitations.

Hardware

Kinect v1 (Xbox 360 sensor) only, via the freenect synchronous API (libfreenect). Kinect v2 will not work with this code — v2 requires libfreenect2/pylibfreenect2, a different library with a different API; nothing here is compatible with it.

tools/kinect_tilt_test.py is a standalone hardware test, not part of the tracker — it just nods the Kinect's motorized tilt base up and down a few times to confirm the tilt motor and freenect device connection work. It shares no code or state with kinect_exercise_tracker.py and isn't imported by it.

Setup

pip install -r requirements.txt

freenect (the libfreenect Python bindings) is not on PyPI — it has to be built from libfreenect with its Python wrapper enabled. requirements.txt covers everything else.

Running

python kinect_exercise_tracker.py

Shows an RGB+depth side-by-side window; press q to quit. Per-frame joint angles, detected exercise, and rep counts are logged to exercise_data.csv in the working directory.

Limitations

• Rep-detection accuracy was never measured against ground truth. No accuracy, precision, or false-positive numbers are claimed anywhere in this repo, and none should be inferred — none were measured.
• Single-exercise lock prevents mid-set switching. Once a rep is counted, the tracker locks onto that exercise for up to 12 reps before it will recognize a different one starting — if you switch exercises mid-set, it keeps trying to match the locked one until the count hits 12 or you stop moving in a way that satisfies its start condition.
• exercise_data.csv logging is per-frame, not per-rep — at typical webcam/Kinect frame rates this file grows fast (every processed frame with landmarks writes a row, indefinitely, for as long as the program runs).
• No sets, rest timers, or workout program — this is a raw rep counter, not a guided workout (contrast with the OAK-D repo below).

Relationship to the OAK-D exercise repo

This is not related to, and not a newer version of, the previously published OAK-D-based exercise repo. It's a separate, earlier exploration on different hardware: this one targets Kinect v1 and uses depth to detect 9 exercises with no sets/rest/workout structure; the OAK-D repo targets a different camera, detects 3 exercises using angle data only (no depth), and wraps them in a full guided-workout program (randomized plan, sets, rest countdowns, skip gesture). Treat them as two independent pieces of work, not two versions of the same project.