Repository navigation
- New installer scripts with an interactive detector selection. The installer now uses uv instead of conda and venv for all detector environments. The installation is 4x faster and 22% smaller than in the previous release.
- Faster runs and smaller outputs. On the example sequence, the detectors need 31% less time and generated 7x smaller output than in the previous release.
- Configurable frame extraction with optional hardware-accelerated decoding.
- MediaPipe Face Landmarker (
mediapipe_face_landmarks) for 3D facial landmarks and facial blendshape coefficients. - New person detection algorithms from MMDetection.
- New subject identity system for video and audio. In
dataset_properties.toml, each camera and audio track declares a subject rule (vsplit,hsplit,assign,bbox,audio_range,path,none). All detectors use the subject names of the sequence. See the subject rules wiki. - pyannote (
pyannote) - speaker diarization is now a separate detector that works with any audio transcription model. - New optimized and more stable implementation for MMPose and SAM 3D Body.
- Reworked documentation with new tutorials on configuring detectors and including your own dataset.
- Many other small fixes and changes in different detectors and visualizers.
Breaking changes:
- All detector environments now use uv instead of venv and conda. Please reinstall NICE Toolbox.
dataset_properties.tomluses subject rules instead ofsubjects_descr,sees_subjectsandhears_subjects. Please update it based on the provided example.machine_specific_paths.tomlrequires the newffmpeg_hwaccelfield. Addffmpeg_hwaccel = 'cuda', or'none'to decode videos on the CPU.- The
[io]section ofdetectors_run_file.tomlhas new keys for the new output folder structure. Please update it based on the provided example. - The 2D pose estimation algorithms need
video_subject_detectorand a person detector such asrtmdet_m_personin thealgorithmslist. whisperxno longer runs the speaker diarization itself. Addaudio_subject_detector(andpyannotefor theaudio_rangerule) to thealgorithmslist, or usewhisperx_no_diar.- The
sam_3d_body_dinov3andsam_3d_body_vithinstances replacesam_3d_body. They have new parameters.