Generate ASMR audio file using WaveGAN

Since audio file is time series data, it has lots of frames per second(44100/sec) and each frame has a huge scope of amplitude(2^16). Furthermore, sound in real life has a cycle, which needs substantial length of frames, therefore it is required to handle the long range of data. As a result, the existing LSTM or RNN model in machine learning libraries are not proper to train the music data.

To overcome these problems, the large receptive fields have to be applied. I could found some methods that use this technic, Google Deepmind's Wavenet and WaveGAN. Wavenet is based on CNN and WaveGAN used transformation of DCGAN.

Of the two options, I chose the latter one. And I could generate asmr files by referencing the paper and the writer's Github code.

If you process the whole procedures in your local environment, it'll take a huge amount of time for each file. To train multiple dataset fastly, I used on-premise MLass NSML. By referencing NSML Document and NSML examples, you could port your code on NSML then it'll allocate GPU resources to you.

Details

• PPT

List

Non-ASMR

1. Chopin - Ballades(Krystian Zimerman)

   Dataset

   Output

2. Bach - Goldberg Variations(Glenn Gould)

   Dataset

   Output

3. Sergei Rachmaninov - Piano Concerto No.1 1st mov

   Dataset

   Output

4. Lala Land OST(insturments)

   Dataset

   Output

5. Philip Glass - Opening

   Dataset

   Output

6. Antonio Carlos Jobim - Wave

   Dataset

   Output

ASMR

1. Autumn leaves

   Dataset

   Output

2. Bird

   Dataset

   Output

3. La Seine

   Dataset

   Output

4. NYC

   Dataset

   Output

5. Cutting

   Dataset

   Output

6. Tropical wave

   Dataset

   Output

7. Barber shop

   Dataset

   Output

8. Trafalgar square

   Dataset

   Output

9. Fry

   Dataset

   Output

10. Le Café

    Dataset

    Output

11. Yamanote Line

    Dataset

    Output

Referneces

• https://github.com/chrisdonahue/wavegan

• https://n-clair.github.io/vision-docs/_build/html/ko_KR/index.html