GLSL Shader API

[mitchmindtree]

No description provided.

[maxdee]

+1 :)

[mitchmindtree]

Seeing as we're currently switching to Vulkan #208 this API will involve compiling GLSL to SPIR-V. There are already multiple solutions for this, but there are a few things important to nannou that are worth investigating:

• Is it currently possible/easy to compile and load GLSL at run-time when using Vulkan via vulkano.
• What versions of GLSL are supported by the GLSL to SPIR-V compiler? Is version support platform dependent like it used to be? Or is support uniform across platform now that it just compiles down to SPIR-V anyway?
• Are there limitations due to the GLSL -> SPIR-V process or is everything supported?

[JoshuaBatty]

Adding in some thoughts for the functionality the shader API should enable.

Loading
Typically users will have .vert and .frag files for their shader stored in the /assets folder of their project. So we should allow the user to load external files stored either in the assets folder or elsewhere on disk.

Here might be some common use cases for loading external shader files.

1. external .vert and .frag files are loaded
2. external .vert, .geom and .frag files are loaded for shaders using a geometry shader
3. same as above but with a tessellation shader
4. load a compute shader

A lot of times when you are only concerned with using a fragment shader for post processing of doing all your graphics inside the frag shader you end up using a passthrough vertex shader. It would be awesome if the user in this instance could define just a frag shader and then nannou loads a default passthrough vertex shader behind the scenes for them.

Finally, it would be worth allowing users to write out the shader file as a string in the rust code that can get passed in for loading.

Setting Uniform Variables
Below are the common uniform variable types that nannou should be able to pass in for the shader.

• int
• float
• bool
• vec2, vec3, vec4
• mat2, mat3, mat4
• samplerXX

Each uniform variable is referenced by it's variable name defined in the shader, and then the appropriate data type is provided by nannou to update that variable.

Uniform Buffer Objects
Another really nice way is making use of Uniform Buffer Objects (UBO's). This allows you to define a struct in your shader like so.

struct Particle
{
  vec3 position;
  vec4 color;
  mat3x3 rotation;
  .....
}; 

Then in nannou, you could have a data type that matches the type defined in the shader like so.

struct Particle
{
   position: pt3,
   color: Rgba,
   rotation: Matrix3,
}

This way you can update a group of variables each frame and pass just a single type into the shader. Sometimes this could be considered slower, but there are cases when this technique is very useful.

Passing in Textures to the Shader
There a quite a number of Sampler types supported in GLSL, see here. The most common one is sampler2d

I've had problems in the past have to bind and unbind textures surrounding a shader pass to get access to them in the shader. This can quickly become an absolute nightmare, especially when you have 2 or more textures that you need access to inside the shader. We should be able to pass is a vector of textures and index into them inside the shader.

Default uniforms
Similar to ShaderToy, it would be nice if there was some defaults for time, frame, date passed in for the user.

Also, when passing in a mesh, inside the vertex shader there should be defaults for accessing texcoords, position, colour, normals.

Passing a Mesh to Draw onto.

• Default plane (2x triangles)
• 3d Model
• 3d mesh from nannou (cube, sphere, icosahedron etc...)

Reading and Writing Data
Currently the way to do this is write information into a colour channel of a texture. For example, writing the speed of a particle into the Red channel of a texture which is read from another shader. This can get quite confusing and isn't straight forward at all. It would be great if we could have a type stored in GPU memory that could be written to by say the compute shader, and then read from a vertex of frag shader elsewhere in the pipeline. Working with types this way is a lot more intuitive than reading from colour channels. Not sure what the most idiomatic way of doing this in Vulkan is but worth really trying to get this correct.

Handy Shader Functions Built Into Nannou
It would be great if nannou came with some super handy shader files with methods for doing common things. For example, there could be a Signed Distance Function file that could be referenced and used in a shader. This would allow you to create a new frag file, and have instant access to the time, frame, date and also SDF functions like, sphere, cube ... basically any 2d and 3d SDF for drawing instantly.

@mitchmindtree & @freesig anything else you can think of that would be nice?