Skip to content

v0.3.1

Choose a tag to compare

@OSLabTools OSLabTools released this 16 Sep 17:51
· 34 commits to main since this release
  • [License Change] To support the open-source community and make NICE Toolbox more accessible, our team decided to change the license of the project from CC BY-NC-SA 4.0 to AGPL-3.0. You can learn more about the new license terms here.
  • UniGaze (unigaze) - a new state-of-the-art gaze direction estimation model.
  • CrisperWhisper (crisper_whisper) - audio transcription model for exact word capture (including verbatims like "[um], [uh], [laughter]").
  • InsightFace (insight_face) - face analysis model, providing face bounding boxes and facial landmarks.
  • Py-Feat (py_feat) facial expression library is updated to the latest version, v2.1.1. The new release adds support for the face_multitask_v2 model, capable of estimating FACS action units, emotions, valence/arousal, gaze, and other modalities in a single model pass.
  • Eye closure detector (eye_closure_ear) using the eye aspect ratio, which estimates an eyelid closure score from 2D facial landmarks.
  • Closed eyes and blinks detector (eye_closure_threshold) with configurable timing thresholds.
  • Audio evaluation metrics: transcription word error rate against reference transcripts, with configurable text normalization and plots.
  • Extension of the ELAN connector for audio transcription and the napari-deeplabcut connector for body joint labeling.
  • New unified detector input/output system with typed array schemas and metadata, applied to both video and audio detectors.
  • Gaze fusion can now use the detected eye state to interpolate over or discard gaze direction when eyes are closed.
  • Rework of the installation - new make commands and a Docker build script that give more control over which model virtual environments are installed.
  • dataset_properties.toml was redesigned to support customization at the individual sequence level, named wildcards and templates.
  • Rerun visualization of SAM 3D Body meshes.
  • Many other small fixes and changes made across different detectors and visualizers.

Breaking changes:

  • Most of the third-party detector venv and conda environments were updated, please reinstall them.
  • The NICE Toolbox Docker image now distributes only the base NICE Toolbox dependencies and environment. Check the updated documentation on how to install specific detectors inside Docker.
  • dataset_properties.toml was redesigned, please update it based on the provided example.
  • The cur_session_ID placeholder is deprecated. The cur_sequence_ID placeholder was renamed to cur_sequence_id (lowercase).
  • The videos section in detectors_run_file.toml was renamed to sequences. sequence_id now supports wildcards (e.g. session_*_take_*).
  • Most of the detectors in detectors_config.toml were updated with new asset links and the new input dependency system.