Towards Intelligibility-Oriented Audio-Visual Speech Enhancement

This paper has been accepted for publication in the Clarity Workshop on Machine Learning Challenges for Hearing Aids

Requirements

• Python >= 3.5 (3.6 recommended)
• PyTorch >= 0.4 (1.2 recommended)
• pytorch-lightning
• tensorboard >= 1.14

You can install all requirements using

pip install -r requirements.txt

Usage

Update config.py with your dataset path Try python train.py --log_dir ./logs --a_only False --gpu 1 --max_epochs 15 --loss stoi to run code.

License

The code in this repository is CC BY 4.0 licensed, as found in the LICENSE file.

Acknowledgements

Parts of the code are adapted from Co-Separation, Lipreading using Temporal Convolutional Networks and VisualVoice.