Skip to content

Visuals

Jump to bottom
Yohan Chalier edited this page Jul 5, 2023 · 1 revision

By default, no visualizer is attached to the tracker. It simply prints dots to stdout when a beat occurs. You may override this behavior by specifying the name of a handler after beat tracking arguments:

python -m beatviewer <Beat Tracking Args> <Handler Name> <Handler Args>

Possible handler names are beep, fireworks, fluid, galaxy, rectangles, socket, tunnel, warp and waves.

Contents

Beep On Beat

Handler name: beep

Make sounds when beats and onsets are detected. If you are on Windows, it uses the winsound library. If you are on Linux, it uses and external package called beep.

beatviewer beep [-h] [--beats-only] [--onsets-only]
Argument Default Description
beats-only False Only beep when beats occur
onsets-only False Only beep when onsets occur

Warp

Handler name: warp

This visualisation synchronizes a video by warping time so that checkpoints in the video (visual beats) correspond to audio beats. The checkpoints can be set automatically using a preprocessing step (see Directogram Computation), or manually (see Video Annotator). Once set, you simply must keep the video file and the accompanying JSON file in the same folder, and call beatviewer:

python -m beatviewer warp [-h] [--rewind REWIND] [--timing TIMING] [--before BEFORE] [--after AFTER] [--margin MARGIN] [--size SIZE] [--virtual-cam] [--prev-threshold PREV_THRESHOLD] [--next-threshold NEXT_THRESHOLD] [--jumpcut] video_path
Argument Default Description
video_path Path to the video file
rewind 0 Probability of going back in time at each checkpoint
timing linear Timing function and arguments, see below
before 120 Number of frames stored in the buffer before its cursor
after 120 Number of frames stored in the buffer after its cursor
margin 120 Number of frames above the default buffer size to trigger the cleanup
size 900 Length of the largest video edge
virtual-cam False Send video output to a virtual camera instead of a window
prev-threshold 0.1 Duration (in seconds) above which the timing is updated (see below). Increase to reduce flickering.
next-threshold 0.1 Duration (in seconds) above which the timing is updated (see below). Increase to reduce flickering.
jumpcut False Jump to a random checkpoint when a beat occurs instead of warped playback

Timing functions. Time warping between checkpoints is, by default, linear. However, it often looks better to emphazise the visual beats, so several warping easing functions are available. Possible choices are linear, power or bezier. The last two require arguments, e.g. --easing "power(1.5)" or --easing "bezier(0.3,0,0.7,1)".

Function Argument Behavior
linear - Linearly interpolates between previous and next checkpoints
power float If t is the progression between the previous and next checkpoints, it gets raised to the specified power. A power of 1 means linear interpolation. A power greater than 1 starts slowly and accelerates. A power greater than 1 starts fast and decelerates.
bezier x1, y1, x2, y2 Uses a cubic Bezier interpolation, given two control points. Behaves as CSS bezier interpolation.
ease-in - Same as bezier(0.42,0,1,1)
ease-out - Same as bezier(0,0,0.58,1)
ease-weak - Same as bezier(0.3,0,0.7,1)
ease-medium - Same as bezier(0.42,0,0.58,1)
ease-strong - Same as bezier(0.75,0,0.25,1)

Beat timing distance thresholds. Warping is achieved by interpolating a playback cursor between the previous and the next beat instant given a previous and a next video checkpoints. The next beat instant is predicted as the previous beat instant plus the estimated beat period, i.e. the BPM. When a new beat occurs, one of the two instants is updated: the (temporally) closest one is chosen. The temporal distance threshold is used to prevent any update when the detected beat occurs relatively close to the predicted one. Larger threshold reduces flickering, as the program will wait for the lag to get large before updating.

It also works with GIF files. Here are some that you might want to try:

Filename Checkpoints
alien.gif 0 9 18 27 36 45 54 63 72 81 90
catjam.gif 5 19 31 44 56 68 79 90 103 115 128 140 152
charlie-brown.gif 4 12
kirua.gif 2 8
mickey.gif 5 13
steps.gif 2 16
walk.gif 13 27
zoom.gif 15

JSON File Format. You can manually write the JSON file that goes with the video, to manually specify the checkpoints. Here is a sample file:

{
    "path": "catjam.gif",
    "frame_count": 157,
    "frame_rate": 24.0,
    "width": 640,
    "height": 640,
    "checkpoints": [5, 17, 29, 42, 54, 66, 77, 89, 101, 113, 126, 138, 150]
}

Pygame Visuals

The following visuals rely on Pygame 2D drawing features.

Tip: Hide Pygame support prompt

You may disable Pygame's support prompt by setting the environment variable PYGAME_HIDE_SUPPORT_PROMPT to 1.

Galaxy

Handler name: galaxy

This draws a number of dots, slowly coming towards the screen, that are pushed around when a beat occurs.

beatviewer galaxy [-h] [--fps FPS] [--width WIDTH] [--height HEIGHT] [--fullscreen] [--n N] [--bw] [--z0 Z0] [--dz DZ] [--dxy DXY] [--vmax VMAX] [--blur BLUR]
Argument Default Description
fps 60 Framerate
width 1280 Window width
height 720 Window height
fullscreen False Make the window fullscreen
n 200 Number of dots
z0 0.1 Z-index (between 0 and 1) of the camera plane. Lower values increase the FOV.
dz 0.015 Dot speed along the Z axis.
dxy 0.02 Acceleration amplitude for the XY axes. Greater values will make beat impacts stronger.
vmax 0.05 Dot speed limit on the X and Y axis. Both range from 0 to 1.
bw False Disable colors
blur 2 Dot blurring factor. Dots get sharper as the value increases.
random-dz 0 Z-axis speed deviation probability
base-radius 1 Dot radius

Rectangles

Handler name: rectangles

This draws rectangles when beats occur, along with color changes.

beatviewer rectangles [--fps FPS] [--width WIDTH] [--height HEIGHT] [--fullscreen] [--palette {bw,rgb,dark,chromakey,hippy,contrast}] [--scale-curve SCALE_CURVE] [--duration DURATION] [--beat-breadth BEAT_BREADTH] [--onset-breadth ONSET_BREADTH] [--onsets]
Argument Default Description
fps 60 Framerate
width 1280 Window width
height 720 Window height
fullscreen False Make the window fullscreen
palette dark Color palette, possible values are bw, rgb, dark, hippy, chromakey and contrast
scale-curve 0.3 Power of non-linear scale progression. Values smaller than 1 produce an ease-out effect.
duration 5 How long a rectangle takes to go off screen, in seconds
beat-breadth 0.97 Relative size of inner beat rectangles. Smaller values create thicker rectangles.
onset-breadth 0.99 Relative size of inner onset rectangles. Smaller values create thicker rectangles.
onsets False Also handle onsets

Tunnel

Handler name: tunnel

This draws a tunnel that comes closer to the screen in sync with the beat.

beatviewer tunnel [-h] [--width WIDTH] [--height HEIGHT] [--n N] [--z0 Z0]
Argument Default Description
fps 60 Framerate
width 1280 Window width
height 720 Window height
fullscreen False Make the window fullscreen
n 20 Number of tunnel parts
z0 0.05 Z-index (between 0 and 1) of the camera plane. Lower values increase FOV.

Waves

Handler name: waves

This draws waves propagating through ropes along with the occuring beats.

beatviewer waves [-h] [--fps FPS] [--width WIDTH] [--height HEIGHT] [--fullscreen] [--line-count LINE_COUNT] [--line-offset LINE_OFFSET] [--dot-count DOT_COUNT] [--dot-offset DOT_OFFSET] [--amplitude AMPLITUDE] [--sigma SIGMA] [--offset OFFSET] [--abs-distance] [--buffer-size BUFFER_SIZE]
Argument Default Description
fps 60 Framerate
width 1280 Window width
height 720 Window height
fullscreen False Make the window fullscreen
line-count 20 Number of lines (or ropes)
line-offset 0.001 Time differnce between two lines, in seconds
dot-count 50 Number of dots per line
dot-offset 0.0007 Time difference between two dots, in seconds
amplitude 100 Gaussian perturbation amplitude, in pixels
sigma 0.1 Gaussian perturbation width
offset 0 Static perturbation offset, in seconds
abs-distance False Use absolute distance instead of Euclidean norm for spacing lines and dots
buffer-size 20 Gaussian buffer size

Socket

Handler name: socket

This handler acts as a websocket server. Any connected socket will receive a message whenever a beat, onset or BPM change event occurs. Messages are simple strings: BEAT, ONSET and BPM <float>.

beatviewer socket [-h] [--web] [--host HOST] [--port PORT] [--mute-beats] [--mute-onsets] [--mute-bpm]
Argument Default Description
web False Use the WebSockets protocol (for the web simulations)
host localhost Server host
port 8765 Server port
mute-beats False Do not send BEAT messages
mute-onsets False Do not send ONSET messages
mute-bpm False Do not send BPM messages

Regular socket server is used to communicate with the OBS script.

Web Visuals

Web visuals are HTML pages with a JavaScript code that acts as a socket client. Under the hood, the program uses the socket handler, and then starts a basic HTTP server to serve an HTML file (those files can be seen in the web folder).

You could simply start beatviewer with the socket handler and open the HTML files within your browser, it would do the same. Also, it should be possible (untested at this day) to make this work remotely.

Fireworks

Handler name: fireworks

Each beat triggers a new firework, with a random color and a random number of branches.

Parameter Default Description
uri ws://localhost:8765 Socket server URI
p 0.5 Geometric law power for generating the number of branches for a firework.
palette green Color palette. Possible values are green, colors, dark.
line_width 40 Line width.

Fluid Simulation

Handler name: fluid

Each beat adds smoke particules with a random velocity inside a Eulerian fluid simulator, adapted from the work of Matthias Müller for Ten Minute Physics.

Some GET parameters can be changed:

Parameter Default Description
uri ws://localhost:8765 Socket server URI
velocity 20.0 Maximum air flow velocity when opening the valve.
palette red Smoke particules color palette. Possible values are red, green, blue, dark and hippy.
iters 40 Number of simulation iteration for each frame
dt 0.01667 Simulation time step
over_relaxation 1.9 Simulation over relaxation (see Müller video)

OBS script

This repository contains an OBS script called BeatViewer, that communicates with the regular socket visualizer.

Installation

If this is the first time you are using a Python OBS script, you will have to set Python path for OBS, and add the script to OBS:

  1. In OBS, go to Tools > Scripts
  2. Go to the Python params tab
  3. Set Python path to the folder containing Python's executable
  4. In OBS, go to Tools > Scripts
  5. Click on the plus sign, and open obs_beatviewer.py

Usage

  1. Start beatviewer with the socket handler (--web flag should be off, here we use regular sockets)
  2. Open OBS
  3. Go to Tools > Scripts, find the obs_beatviewer.py script, and under the Socket Server option panel, click the Connect button. Now OBS should be connected to the socket server, which should have logged this
  4. Check one or more of the available visuals

If you provide a source for the Debugging Text Source option, the provided Text Source will be used to display current tracker status, such as connection state and last BPM value.

Blinking Slideshow

Change the source path of an Image Source when a beat occurs and briefly toggle source visibility.

Option Default Description
Source name BlinkingSlideshow The name of the Image Source which the source will be changed
Images Folder — Path to a folder containing the images to display (ordered by name)
Blinking Duration 100 Duration of a blink in milliseconds (absolute duration) or in percent of beat duration (relative duration)
Duration Relative to BPM False Switch between absolute and relative duration
Random Index Jumps False Go through images in a random order
Automatic Scaling True Automatically scale the source when a image is loaded

fftools

Dedicated assets must be prepared for this effect to work. You might want to look at the Average or Shots tools from my set of FFmpeg Tools for this: you can either split a video into multiple shots, or create long-exposure type images from videos to decompose movements.

Seek On Beat

Automatically seeks within a media source when a beat occurs, causing a jump cut effect.

Option Default Description
Source name SeekOnBeat The name of the Media Source which the source will be changed
Clone this wiki locally