CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos

CLIPSep - Official PyTorch Implementation

CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos
Hao-Wen Dong, Naoya Takahashi*, Yuki Mitsufuji, Julian McAuley, Taylor Berg-Kirkpatrick
In ICLR 2023. (* corresponding author)

arXiv | OpenReview | Demo

Orgnization

• music: Code for downloading and preprocessing the MUSIC dataset.
• vggsound: Code for downloading and preprocessing the VGGSound dataset.
• clipsep: Code for training and experimenting with the CLIPSep model.

Citation

If you find this work useful for your research, please cite our paper:

@inproceedings{dong2023clipsep,
  title={CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos},
  author={Hao-Wen Dong and Naoya Takahashi and Yuki Mitsufuji and Julian McAuley and Taylor Berg-Kirkpatrick},
  booktitle={Proceedings of International Conference on Learning Representations (ICLR)},
  year={2023}
}