Shady is a nifty CLI tool for rendering GLSL fragment shaders for easy development and hacking.
To build this version
go get -u github.com/billtraill/shady/cmd/shadyCurrently working on webservice interface to imu (Inertial Measurement Unit) or motion sensor. See also https://github.com/billtraill/whichwaysup which provides an example webservice for Raspberry Pi using an Adafruit BNO085. May change this to be a generic webservice json interface to be able to inject any unifom into 'glsl' shaders.
To get the upstrean version
go get -u github.com/polyfloyd/shady/cmd/shadyThe basic setup is a single fragment shader, like a regular fragments shader, calculates the color for each pixel. But instead of receiving vertex and normal and transformation information from the vertex shader, it defines it's own algorithm for shapes.
The best supported format/environment for shaders is that of Shadertoy.com. Example:
void mainImage(out vec4 fragColor, in vec2 fragCoord) {
vec2 uv = fragCoord.xy / iResolution.xy;
fragColor = vec4(uv.x, uv.y, 0, 1);
}
Currently, the iTime, iTimeDelta, iFrame, iDate, iMouse, and
iResolution, iChannelResolution uniforms are supported. Other uniforms are
defined but not initialized.
See also https://www.shadertoy.com/howto for info on how to write shaders for Shadertoy.
To include another GLSL file, you may use the directive below:
#pragma use "path/to/file.glsl"This allows you to use functions and such from other GLSL files so it becomes possible to create libraries. There is no namespacing or generation of forward function declarations, it just takes a source file and dumps it in the place of this directive much like C does. However, it does prevent including the same file more than once in recursive inclusion.
File paths are resolved relative to the source file that declared the include directive.
It is possible use resources like images, videos and audio from shaders in
this environment by using the iChannelX samplers. On the website, one can
select this resource input mapping using dialogs. This implementation requires
these mappings to be specified in the shader source. The respective uniform is
declared automatically.
Mappings can refer to other files such as images, videos and audio. Relative paths are resolved relative to the GLSL file that declared them.
Mappings are declared in a special directive that is parsed by shady. These are typically inserted at the top of the file. Its format is:
#pragma map <uniform name>=<loader>:<value>Mappings can also be set on the command line when invoking shady by using the
-map flag. The format is the same as described earlier. Mappings set on the
command line take precedence over mappings declared in source files. There are
no hard requirements for the namespace being the same as in the shader, so it
is possible to e.g. override an image with a video. Example:
shady -i thing.glsl -ofmt x11 -g 1366x768 -f 60 -map myTexture=video:myVideo.mkv
uniform name is the name of the sampler uniform that is inserted into the
source of the fragment shader. Unlike shadertoy.com, which names all samples as
iChannelX, the name can be of any value as long as it is a valid GLSL
variable name.
loader specifies how value should be interpreted.
There are a couple of loaders that you can choose from:
The builtin loader gives access to some of the presets that can be found on
Shadertoy. Accepted values for builtin are:
Back Buffer: creates asampler2Dcontaining the previously rendered image.RGBA Noise Small: creates asampler2Dtexture with pseudo-random noise. The randomness is deterministic.RGBA Noise Medium: the same as above, but bigger.
Example: Enable the sampler named iChannel0 as a noise texture:
#pragma map iChannel0=builtin:RGBA Noise MediumSetting the loader to image interprets the value as a path to an image file
and creates a sampler2D containing a static texture containing the RGBA data
of the image file.
Supported formats are JPEG, PNG and GIF, the latter using the first frame of
the animation for the texture.
For each mapped image, an additional vec3 uniform is created with the
original size of the image named ${uniform name}Size. The Z component of this
vector is reserved.
Example:
#pragma map myTexture=image:yoloswag.pngAudio files can be loaded as a texture with a size of 512x2. Row 0 contains the FFT of the current window and row 1 contains the actual sound wave. For regular audio files, The playback rate is determined by the duration and framerate flags. For realtime audio, the window is the most recently produced audio, skipping information if rendering can not keep up.
If the value is just a file, this file is used as audio. FFmpeg is invoked to decode the file, so any format supported by FFmpeg can be played.
For realtime audio, only raw PCM pipes are supported. The filename must be
followed by the PCM format settings as ;<rate>:<channels>:<encoding>.
encoding is the sign as s or u followed by the number of bits per sample
and then the endianness as le or be, e.g. s16le.
Example: Map audio to the audio of an MP3 file:
#pragma map music=audio:whatever.mp3Using videos as textures is very similar to images, there is a sampler2D
uniform containing the current video frame and a ${uniform name}Size vector
for the resolution. There is an additional {uniform name}CurTime float which
is the current time in seconds in the video. This value is the same as iTime,
but wraps when the video is restarted from the beginning.
The sound of the video is not available, although this may be implemented in the future.
Example:
#pragma map video=video:party.mkvIt is possible to map another shader as a texture by using the buffer loader.
This is equivalent of just calling the mainImage function of this other
shader and using the calculated color as texel. However, buffers have a
separate resolution and Back Buffer. Because the render output of a buffer in
raster format, the size of the texture should be specified in the mapping by
appending ;WxH to the shader filename.
Like videos, the buffer is declared as a sampler2D along with a
${uniform name}Size vector.
Example:
#pragma map thing=buffer:other-shader.glsl;512x512NOTE: Buffer support is not very well tested, your mileage may vary.
If Shady was compiled using the kinect build tag, it is possible to use a
Kinect's RGB and depth image in shaders. Just pass -tags kinect to go build
when building and use #pragma map kinect=kinect:on to create a sampler2D of
the Kinect's video stream. The alpha channel holds the depth image.
Internally, libfreenect is used which only supports the earlier Kinect versions for the XBox 360.
Work in progress...
This provides an interface to a imu via a webservice. It should be easly to adapt a webservice for any imu. The uniforms that are added are prefixed with the map name.
These are (assuming the name is imu):
- uniform vec3 imuAcceleration
- uniform vec3 imuGyro
- uniform vec3 imuMagnetometer
- uniform vec4 imuQuaternione
- uniform bool imu Shake
Example: Map imu to a webservice:
#pragma map imu=imu:http://localhost:8080/May change this to be a generic uniform injector from a webservice.
Currently the webservice is polled every 1/10 second. TODO make this configurable.
Ledcat is a program that can be used to control lots of LEDs over lots of protocols. Shady can be combined with Ledcat to bring the fireworks to your LED-displays!
It can be installed like this when you have the Rust Language:
cargo install ledcatTo aid development, Ledcat can be used to simulate a display in a terminal like this:
# LEDCAT_GEOMETRY is a special env var that Ledcat and Shady use to set the
# display size. It is also possible to use the -g flag on both programs.
export LEDCAT_GEOMETRY=128x128
shady -i example.glsl -ofmt rgb24 -f 20 | ledcat -f 20 showFFmpeg may be used to render to video files:
# Render at 1024x768 at 20 fps and show it, the same as using `-ofmt x11`:
shady -i example.glsl -ofmt rgb24 -g 1024x768 -f 20 \
| ffplay -f rawvideo -pixel_format rgb24 -video_size 1024x768 -f 20 -
# The same, but render 12 seconds to an MP4 file
shady -i example.glsl -ofmt rgb24 -g 1024x768 -f 10 \
| ffmpeg -f rawvideo -pixel_format rgb24 -video_size 1024x768 \
-framerate 10 -t 12 -i - example.mp4
Visualising the output of MPD is possible by adding the following to your MPD config:
audio_output {
type "fifo"
name "FIFO"
path "~/.mpd/mpd.fifo"
format "22000:16:1"
}
And then creating a PCM audio mapping like this:
#pragma map music=audio:~/.mpd/mpd.fifo;22000:1:s16leSome shaders can really ask a lot from a system, in these cases it may not be possible to animate real time. If it is acceptable to have the animation be of finite length and restart after a while, write a series of frame to a file, and load them in a loop.
# Render a 20 second loop to a file:
shady -i example.glsl -g 64x64 -f 60 -n $((20*60)) -ofmt rgb24 -o ./my-animation.bin
# Play the animation repeatedly:
while true; do
cat ./my-animation.bin | ledcat -g 64x64 -f 60 show
doneOptionally, you could use something like gzip to reduce the file size.
Headless rendering is possible. If $DISPLAY is unset because X11 is not
running, try running shady with the EGL_PLATFORM env var set to surfaceless
or drm.
If you still are not able to get shady to run headless, animate to a file and play from that file in real time. See above.
Error compiling fragment shader:
0:2(1): error: syntax error, unexpected NEW_IDENTIFIER
The above error could be caused by a precision mediump float; being present.
Because this is an OpenGL ES directive, it is not supported. Try removing it or
wrapping with a preprocessor macro:
#ifdef GL_ES
precision mediump float;
#endif2021/06/02 16:15:37 initEGL display.CreateContext: failed to call eglCreateContext
2021/06/02 16:15:37 Couldn't initialize engine: failed to call eglCreateContext
On a Raspberry Pi 4 this error is caused by not having env variable MESA_GL_VERSION_OVERRIDE set, try
export MESA_GL_VERSION_OVERRIDE=3.3
