Skip to content
Deep dreams on audio spectrograms get resynthesized for deep-learning generated audio effects.
Branch: master
Clone or download
Pull request Compare This branch is even with markostam:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.

Deep Dreaming on Audio Spectrograms with Tensorflow

This notebook allows imagenet to hallucinate audio effects onto an mp3.

It uses the tensorflow primer on DeepDreaming to adapt Christian Dittmar & Stefan Balke's DeepDreamEffect for Caffe from HAMR 2015 to tensorflow. In addition, the loss function for inducing hallucinations in the convnet is edited such that high energy areas of the spectrogram are ignored, thereby avoiding distortion upon resynthesis and adding musicality to the effect.

In essence, this hack converts audio spectrograms into images, where they can be processed by specific layers of a pre-trained convulutional neural network (Inception v3 trained on ImageNet) , and then re-synthesized into audio.


Python 3.5.x

TensorFlow 0.11 for deep learning.

Librosa 0.4.3 for audio DSP.

numpy 1.11.1

You can’t perform that action at this time.