p5.js Shader generation API

[lukeplowden]

Addresses #7188
Example sketch

Changes:

This PR extends the Shader Hooks p5.Shader.modify() API by adding a way to write shaders in JavaScript which will generate corresponding GLSL code. It simplifies the process of writing shaders, especially for people new to them.

The API currently looks like this:

  myShader = baseMaterialShader().modify(() => {
    const greenValue = uniformFloat(() => myCol());

    getFinalColor((col) => {
      col.x = 1;
      col.y += greenValue;
      return col;
    });
  }, { parser: true, srcLocations: false });

And the generated hook would be:

  vec4 getFinalColor (vec4 color) {
     vec4 temp_0 = color;
     temp_0 = vec4(1.0000, temp_0.y + greenValue, temp_0.z, temp_0.w);
     vec4 finalReturnValue = temp_0;
     return finalReturnValue;
   }

There are two modes for the API. In the JS code above, the second parameter is an optional options object. Currently, since parser: true is set (which is the default option), things like operator overloading and automatic naming of uniforms are allowed. With the (acorn) parsing stage enabled, it is transpiled to an intermediate stage which looks like:

    const greenValue = uniformFloat('greenValue', () => myCol());
    getFinalColor(col => {
        col.x = 1;
        col.y = col.y.add(greenValue);
        return col;
    });

Optionally users can write in this style as well.

A lot of new functions are provided, including create*() and uniform*() where * is a GLSLnative type. There are also the hook callbacks like getFinalColor() above.

Blocking issues

Right now there are a couple of blocking issues before 2.0's release:

• Input Structs need to work properly. I will add the struct inputs this week, the system is already set up for them to be added.
• Built in Functions: I am working on generating the built-in GLSL functions. One issue with this will be about functions which share names with p5 methods, like cos() etc. I can set a flag for these which overrides them, but it depends on which order they're imported. One potential fix is to add these to the global scope and then remove them after setup().

Longer term

And some remaining, medium to long term issues:

• FES: Parameter checking
• Currently, the transpiler is not super smart about operator overloading, and it will attempt to do something like 33.mult(time). This should be a small fix. It should also not transpile anything in uniform default value callbacks (as in uniformFloat('greenValue', () => myCol()); above) as these won't be able to assess at runtime.
• Adding ternary operators
• Variables passed between shaders
• There is only .xyzw, not .rgba or .stpq. RGBA would be especially important as conflating position/ colour data can be difficult for new shader programmers to wrap their head around. See the proxy issue later.
• Discard and proper if statements (non returning)
• Detecting uniforms mid hook. i.e. this could automatically make a uniform for () => millis() :

getWorldInputs((inputs) => {
	inputs.pos.x += sin(millis)
})

• Full GLSL compatibility with types and built in functions. Next thing to consider might be matrices.
• Swizzling can be done through making proxy objects for vectors, or just custom getters and setters for them. Any attempt to access a combination a set of the following could be remapped to xyzw. Including myVec4.yxyz etc.

const swizzles = [
	['x', 'y', 'z', 'w'],
	['r', 'b', 'g', 'a'],
	['s', 't', 'p', 'q'],
],

• Potential to add our own implementations of some common functions e.g. noise
• Currently, if you forgo the acorn walk transpile stage, we get line numbers from the user's sketch output into the generated code. This can be helpful for debugging. Acorn parse gives the option to get line locations. We could use this to retain line numbers with the transpile stage too:

const ast = parse(code, { ecmaVersion: 2021, locations: true });

• The parser is definitely not JavaScript complete, which it needn't be, but it will be worth deciding on and documenting the rules very well. For example, scope is tricky to wrap your head around.
• Would be nice to transpile line 1 to line 2.

   let negativeUV = -UV;
   let negativeUV = createNode(0)-UV;

• When texture() is called, splice in the getTexture definition into shaders that don't already have it (e.g. they didnt branch off of a base material shader)

PR Checklist

Not currently done.

• npm run lint passes
• [Inline reference] is included / updated
• [Unit tests] are included / updated

[davepagurek]

This is looking so good! I was poking around in preview/global seeing how easily I could break things and it's feeling really stable!

[davepagurek]

Since not all hooks will be able to use this (e.g. in a vertex shader hook before it's assigned) are we able to just rely on texture coords in hook inputs?

[limzykenneth]

@davepagurek @lukeplowden Are we ready to have this merged? If so I would like to include it in the next beta release so it can be tested more widely, if not that's fine as well and I'll just release a beta without this for now.

[davepagurek]

I think this is good to merge! if we notice any other little bugs that need fixing we can make another PR.