Add pose landmark-trajectory workflow (_posetools)#353
Merged
Conversation
…n signals New module implementing the array-level pose workflow used across the author's sound-motion studies: video -> tidy per-landmark trajectory arrays/CSV -> derived motion signals. Complements the rendering-oriented MgVideo.pose() pipeline and the per-frame PoseEstimator interface. - extract_pose_landmarks: consolidated MediaPipe extractor (FFmpeg-piped frame reader, fps resampling, resize, model complexity, optional 3D world landmarks, confidence thresholds, NaN on detection failure, detection-rate summary, optional tidy CSV). MediaPipe is imported lazily (optional [pose] extra, already declared in pyproject); both the legacy Solutions API (0.10.14-era wheels) and the Tasks API PoseLandmarker in VIDEO mode (newer 0.10.x wheels) are supported, sharing the model cache with MgVideo.pose(). - midpoint: NaN-propagating trajectory midpoint (e.g. shoulder midpoint as upper-torso proxy). - limb_speed_from_landmarks: confidence-gated central-difference pixel speed with bilateral (element-wise max) limb merge and NaN-aware smoothing; documents the speed-peak-precedes-contact timing bias. - impact_events: candidate impact detection from point trajectories via double-central-difference acceleration magnitude, bilateral max, and an inline relative-threshold peak picker (provisional defaults from the cymbal study; the general picker lives in the sibling _peaks PR). Consolidated/ported from the stillstanding (mp_extract_westney.py, pose_motion.py), Westney-comparisons (concert_mediapipe.py, reh_pose.py, a1_labstage.py) and cymbal-comparison study code/papers (all author-owned analysis code). Multi-view Umeyama pose fusion is deliberately deferred to a phase-2 follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0147io2kMk8M6jcNFNt1G96m
Synthetic ground-truth tests (always run) for midpoint, limb_speed_from_landmarks and impact_events: known constant-velocity speeds, confidence gating, bilateral max merges, Gaussian-bump acceleration impacts recovered after double integration, relative threshold and minimum-interval behaviour, NaN dropout handling. Import-safety is verified in a subprocess with mediapipe blocked (package import and numpy-only helpers work; extract_pose_landmarks raises an ImportError naming the [pose] extra). The extractor itself is covered by an integration test on the bundled dancer example video that skips when mediapipe or the model file is unavailable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0147io2kMk8M6jcNFNt1G96m
…on consistency) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0147io2kMk8M6jcNFNt1G96m
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Complements the existing
MgVideo.pose()rendering path with a landmark-trajectory workflow used across several studies:extract_pose_landmarks— consolidated MediaPipe extractor (video → tidy per-landmark (x, y, visibility) trajectories + optional 3-D world landmarks, NaN dropout semantics, detection-rate summary). Supports both MediaPipe API families (legacy Solutions ≤0.10.14 as a faithful but now-untestable port, and the Tasks-API PoseLandmarker for modern wheels), reusing MGT's landmark names and model cache. Lazy import behind the existingposeextra.midpoint— landmark-group midpoints (e.g. shoulder center).limb_speed_from_landmarks— confidence-gated limb-tip speed with bilateral max-merge (documented speed-precedes-contact bias).impact_events— acceleration-magnitude impact detection from position trajectories (provisional thresholds, parameterized).23 new tests including a real integration test on the bundled dancer.avi (runs, no skips, on mediapipe 0.10.35). No new suite failures. Multi-view Umeyama pose fusion is deliberately deferred to a follow-up PR.
🤖 Generated with Claude Code