Repository navigation
v0.3.1
- [License Change] To support the open-source community and make NICE Toolbox more accessible, our team decided to change the license of the project from
CC BY-NC-SA 4.0toAGPL-3.0. You can learn more about the new license terms here. - UniGaze (
unigaze) - a new state-of-the-art gaze direction estimation model. - CrisperWhisper (
crisper_whisper) - audio transcription model for exact word capture (including verbatims like"[um], [uh], [laughter]"). - InsightFace (
insight_face) - face analysis model, providing face bounding boxes and facial landmarks. - Py-Feat (
py_feat) facial expression library is updated to the latest version, v2.1.1. The new release adds support for the face_multitask_v2 model, capable of estimating FACS action units, emotions, valence/arousal, gaze, and other modalities in a single model pass. - Eye closure detector (
eye_closure_ear) using the eye aspect ratio, which estimates an eyelid closure score from 2D facial landmarks. - Closed eyes and blinks detector (
eye_closure_threshold) with configurable timing thresholds. - Audio evaluation metrics: transcription word error rate against reference transcripts, with configurable text normalization and plots.
- Extension of the ELAN connector for audio transcription and the napari-deeplabcut connector for body joint labeling.
- New unified detector input/output system with typed array schemas and metadata, applied to both video and audio detectors.
- Gaze fusion can now use the detected eye state to interpolate over or discard gaze direction when eyes are closed.
- Rework of the installation - new
makecommands and a Docker build script that give more control over which model virtual environments are installed. dataset_properties.tomlwas redesigned to support customization at the individual sequence level, named wildcards and templates.- Rerun visualization of SAM 3D Body meshes.
- Many other small fixes and changes made across different detectors and visualizers.
Breaking changes:
- Most of the third-party detector venv and conda environments were updated, please reinstall them.
- The NICE Toolbox Docker image now distributes only the base NICE Toolbox dependencies and environment. Check the updated documentation on how to install specific detectors inside Docker.
dataset_properties.tomlwas redesigned, please update it based on the provided example.- The
cur_session_IDplaceholder is deprecated. Thecur_sequence_IDplaceholder was renamed tocur_sequence_id(lowercase). - The
videossection indetectors_run_file.tomlwas renamed tosequences.sequence_idnow supports wildcards (e.g.session_*_take_*). - Most of the detectors in
detectors_config.tomlwere updated with new asset links and the new input dependency system.