Skip to content
Deep dreams on audio spectrograms get resynthesized for deep-learning generated audio effects.
Branch: master
Clone or download
Pull request Compare This branch is even with markostam:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
audio
images
.gitignore
LICENSE
README.md
audio_deepdream_function.py
deep_audio_dreams.ipynb
imagenet_comp_graph_label_strings.txt
tensorflow_inception_graph.pb

README.md

Deep Dreaming on Audio Spectrograms with Tensorflow

This notebook allows imagenet to hallucinate audio effects onto an mp3.

It uses the tensorflow primer on DeepDreaming to adapt Christian Dittmar & Stefan Balke's DeepDreamEffect for Caffe from HAMR 2015 to tensorflow. In addition, the loss function for inducing hallucinations in the convnet is edited such that high energy areas of the spectrogram are ignored, thereby avoiding distortion upon resynthesis and adding musicality to the effect.

In essence, this hack converts audio spectrograms into images, where they can be processed by specific layers of a pre-trained convulutional neural network (Inception v3 trained on ImageNet) , and then re-synthesized into audio.

Dependencies:

Python 3.5.x

TensorFlow 0.11 for deep learning.

Librosa 0.4.3 for audio DSP.

numpy 1.11.1

You can’t perform that action at this time.