Timbre Transfer using Differentiable Digital Signal Processing based on "DDSP" repository.
Check how it works on Google Colab:
- Russian Language
- Bad English Translation
Attention. Sound recording doesn't work on all browsers. Testing was in Google Chrome and Opera.
Based on: DDSP
Article: Differentiable Digital Signal Processing
Examples: Audio
Differentiable Digital Signal Procressing (DDSP) enables direct integration of classic signal processing elements with end-to-end learning, utilizing strong inductive biases without sacrificing the expressive power of neural networks. This approach enables high-fidelity audio synthesis without the need for large autoregressive models or adversarial losses, and permits interpretable manipulation of each separate model component. In all figures below, linear-frequency log-magnitude spectrograms are used to visualize the audio, which is synthesized with a sample rate of 16kHz.
Result Example:
Citation
@inproceedings{
engel2020ddsp,
title={DDSP: Differentiable Digital Signal Processing},
author={Jesse Engel and Lamtharn (Hanoi) Hantrakul and Chenjie Gu and Adam Roberts},
booktitle={International Conference on Learning Representations},
year={2020},
url={https://openreview.net/forum?id=B1x1ma4tDr}
}