Add MediaPipe library integration#136
Merged
Merged
Conversation
Add explanatory docstrings and small style cleanups across the face_recog_stream package. Changes include: adding class/method docstrings and a close() docstring in mediapipe_mesh.py, documenting DetectorConfig/DetectorState and SpeakingDetector.reset/update in speaking_vad.py, finishing a docstring in vvad_speaker.py, and removing stray blank lines in face_tracker.py. Also reflow a long if condition in run.py for readability. These are non-functional, readability and maintenance improvements.
Change vvad track box typing and values from ints to floats to match vvad expectations: vvad_track_boxes now holds Tuple[int, Tuple[float, float, float, float]] and appended box coords are cast with float(...). Also add a "# type: ignore" to the MediaPipe FaceMesh instantiation to silence typing errors from the mediapipe package. These are small type/annotation fixes and do not change runtime behavior beyond ensuring correct types for downstream consumers and the type checker.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces several important updates to the face recognition streaming system, focusing on adapting the configuration for higher-resolution cameras, improving mouth feature extraction, and refactoring the codebase for clarity and extensibility. The main themes are configuration updates for 1920×1080 resolution, the addition of mouth features and MediaPipe integration, and code cleanup.
Configuration updates for 1920×1080 camera:
CAMERA_WIDTHandCAMERA_HEIGHTinconfig.pyto 1920×1080, and adjusted related parameters such asCAMERA_FPS(now 30),TRACK_BUFFER(now 60),SELFIE_MIN_FACE_PX(now 75),AUTO_ENROLL_MIN_FACE_PX(now 65), andSELFIE_MIN_ENGAGEMENT(now 0.0023) to maintain consistent behavior with the new resolution. All documentation and scaling examples have been revised accordingly. [1] [2] [3] [4] [5]Mouth feature extraction and MediaPipe integration:
mouth_features.py, that defines data structures and extraction logic for scale-normalized mouth measurements from face landmarks, supporting future voice activity detection and expression analysis.mediapipe_mesh.py, which wraps MediaPipe FaceMesh for extracting facial landmarks from frames, making it easy to obtain per-face landmark arrays for downstream analysis.Face tracker improvements and refactoring:
face_tracker.pyto remove unnecessary section comments, streamline imports, and update the speaking detector (VVAD) integration: now collects track bounding boxes and pushes them in batch to the VVAD module for each frame, improving modularity and clarity. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12]Minor and auxiliary updates:
run.pyto reflect the new default camera resolution, and removed an unused variable. [1] [2] [3] [4]These changes collectively modernize the system for higher-resolution input, lay groundwork for advanced mouth/voice features, and enhance maintainability.