Skip to content

v0.3.2

Latest

Choose a tag to compare

@OSLabTools OSLabTools released this 03 Nov 14:00
  • New installer scripts with an interactive detector selection. The installer now uses uv instead of conda and venv for all detector environments. The installation is 4x faster and 22% smaller than in the previous release.
  • Faster runs and smaller outputs. On the example sequence, the detectors need 31% less time and generated 7x smaller output than in the previous release.
  • Configurable frame extraction with optional hardware-accelerated decoding.
  • MediaPipe Face Landmarker (mediapipe_face_landmarks) for 3D facial landmarks and facial blendshape coefficients.
  • New person detection algorithms from MMDetection.
  • New subject identity system for video and audio. In dataset_properties.toml, each camera and audio track declares a subject rule (vsplit, hsplit, assign, bbox, audio_range, path, none). All detectors use the subject names of the sequence. See the subject rules wiki.
  • pyannote (pyannote) - speaker diarization is now a separate detector that works with any audio transcription model.
  • New optimized and more stable implementation for MMPose and SAM 3D Body.
  • Reworked documentation with new tutorials on configuring detectors and including your own dataset.
  • Many other small fixes and changes in different detectors and visualizers.

Breaking changes:

  • All detector environments now use uv instead of venv and conda. Please reinstall NICE Toolbox.
  • dataset_properties.toml uses subject rules instead of subjects_descr, sees_subjects and hears_subjects. Please update it based on the provided example.
  • machine_specific_paths.toml requires the new ffmpeg_hwaccel field. Add ffmpeg_hwaccel = 'cuda', or 'none' to decode videos on the CPU.
  • The [io] section of detectors_run_file.toml has new keys for the new output folder structure. Please update it based on the provided example.
  • The 2D pose estimation algorithms need video_subject_detector and a person detector such as rtmdet_m_person in the algorithms list.
  • whisperx no longer runs the speaker diarization itself. Add audio_subject_detector (and pyannote for the audio_range rule) to the algorithms list, or use whisperx_no_diar.
  • The sam_3d_body_dinov3 and sam_3d_body_vith instances replace sam_3d_body. They have new parameters.