Workshop on music visualization with p5.js from Eyeo '15, previously MozFest '14, NYU ITP
JavaScript HTML
Pull request Compare This branch is even with therewasaguy:gh-pages.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
demos
lib
music
output
template/my_name_-_sample_viz
README.md

README.md

Visualizing Music with p5.js

This session is for anyone who would like to explore music, visuals and creative coding for the web. We'll demonstrate types of data we can get from digital signal processing using interactive sketches in p5.js and the p5.sound library that builds upon the Web Audio API. We'll explore various methods to map this data onto meaningful visuals that enhance our experience of music.

Slides

Intro Slides

Code Slides

Demos:

1. Amplitude

Hello Amplitude | Source Code

Amplitude over time

Amplitude Threshold

Simple Beat Detection

Draw Full Waveform w/ Playhead

Time Domain (Oscilloscope) | Source Code

2. Frequency

FFT - Fast Fourier Transform

FFT Spectrum Drag, Drop 'n Analyze | Source Code

FFT Spectrograph

FFT Particle System | Source Code

Scaling the FFT

FFT Scale by Neighbors

FFT Scale by 1/3 Octave

FFT Unknown Pleasures

FFT Particle System Scaled / Seek (Fireworks)

FFT Peak Detect | Source Code

3. Pitch

autocorrelation in the time domain to detect fundamental frequency

Autocorrelate Time Domain | Source Code

Autocorrelation Circle | Source Code

Pitch Track | Source Code

4. Musical Timing

sync music to timestamped lyrics

Display Lyrics | Source Code

pre-analyze file with the Echo Nest API, and visualize the result

Pre-rendered Analysis (via Echo Nest - Beat + Pitch) | Source Code

Pre-rendered Analysis (via Echo Nest) - Pitch Segment 1 | Source Code


Libraries

Participants may use whatever tools they wish, but the demos in this repo use the following libraries:

p5.js is a JavaScript library that starts with the original goal of Processing, to make coding accessible for artists, designers, educators, and beginners, and reinterprets this for today’s web.

p5.sound.js is an addon library that brings the Processing approach to the Web Audio API.

p5.dom.js is an addon library that helps us manipulate the DOM.

Setup

Inspiration:


p5.sound Classes That We'll Use For Music Visualizations:

p5.AudioIn - microphone! documentation | source code

p5.SoundFile - load and play .mp3 / .ogg files. documentation | source code

  • loadsound() creates a SoundFile using a Web Audio API buffer. Use during preload(), or with a callback, or with drag and drop.

p5.PeakDetect - detect beats and/or onsets within a frequency range documentation }| source code

p5.Amplitude - Analyze volume (amplitude). documentation | source code

  • .getLevel() returns a Root Mean Square (RMS) amplitude reading, between 0.0 and 1.0, usually peaking at 0.5
  • .smooth()

p5.FFT - Analyze amplitude over time / frequency. documentation | source code

  • .analyze() returns amplitude readings from 0-255 in the frequency domain.
  • .waveform() returns amplitude readings from -1 to 1 in the time domain. demo | source

Music included in the demos/repo:


Dig Deeper into Web Audio Music Visualization...

Deeper into DSP

Music Visualization Inspiration...

Notation

Interactive

Audio

Data Sonification

Musical Form

Lyrics