[p5.strands] Bridging p5.strands and p5.js functions

[lukeplowden]

Increasing access

By aiming to make the learning curve for p5.strands (therefore shaders) as easy as possible when coming from p5.js. As much as possible, p5.js functions should work in p5.strands, unless there is good reason for them not to.

Most appropriate sub-area of p5.js?

• Accessibility
• Color
• Core/Environment/Rendering
• Data
• DOM
• Events
• Image
• IO
• Math
• Typography
• Utilities
• WebGL
• Build process
• Unit testing
• Internationalization
• Friendly errors
• Other (specify if possible)

Feature enhancement details

Currently, some p5.js functions will not work properly in p5.strands. This issue aims to improve compatiblity, reduce confusion, and make it easier to mix standard p5.js code in with p5.strands shaders.

Background

p5.strands ports a large number of GLSL's built in functions. For example:

• trigonometric functions: sin, cos, etc
• maths functions: abs and mix
  In some cases, these are duplicates of p5.js functions. The p5.strands compiler temporarily overrides duplicates, so they can be used with p5.strands objects.

p5.strands operates a bit differently than the rest of the library. In a way, it emulates namespacing by overriding p5.js functions, but it does so implicitly. The compiler lets use use JavaScript values like Numbers and Arrays, but internally they are converted to objects representing nodes in a kind of Abstract Syntax tree (e.g. FunctionCallNode, VectorNode, etc). So when we try to use p5.js functions, they can fail if we pass these nodes as objects. At the same time, p5.strands currently disables the Friendly Error System, which further obfuscates the problem.

(Note: the disabling of FES is a separate Issue which I will post soon and update the link to soon)

Apart from documentation and other learning materials, some work is needed to improve the interoperability of p5.js and p5.strands functions. The aim is to keep the node based abstraction hidden, and provide more analogues for p5.js functions. This should make p5.strands more integrated with the rest of the library, making it more accessible.

p5.js Math functions

p5.js' math functions, such as map, don't always work as expected in p5.js. Take the following code for example:

  a = baseColorShader().modify(()=> {
    const time = uniformFloat(()=> millis());
    getWorldInputs(inputs => {
      let r = 1;
      let b = r / 0.2;
      let g = map(50, 0, 100, sin(time), 1);
      inputs.color = [r, 0, g, 1];
      return inputs;
    });
  });

This currently throws a generic JS 'NaN' error, as map expects a number but receives a FunctionCallNode object. This should be allowed behaviour, and p5.js functions should as much as possible be usable within p5.strands to reduce the mental load of remembering what can and can't be done.

There may be cases apart from the Math functions, but that's the most obvious example. This part of the issue requires some discussion about what should be ported to p5.strands, and how it could be implemented.

Mix / lerp

There is a special case for mix, which is GLSL's version of lerp in p5.js. There has been some discussion about creating an analogue for lerp in p5.strands, so that either of the following would be possible:

myShader = baseColorShader().modify(() => {
	getFinalColor(col => {
		let red = [1, 0, 0, 1];
		let blue = [0, 0, 1, 1];
		
		// Currently possible:
		col = mix(red, blue, 0.5);
		
		// Not currently possible:
		col = lerp(red, blue, 0.5);
	})
})

In this case, it's a small amount of code to fix, and could be done separately.

[davepagurek]

• I think lerp is ready to be worked on if anyone is interested! (See [p5.strands] Aliasing GLSL's mix as lerp #7875)
• For map, the challenge will maybe be deciding how to manage the more complicated function. One option could be to inject a new map function into a shader hook (an undocumented feature of shader.modify() is that you can write a function of your own the same way you'd write a hook, and then you can use that function from within your hooks.)
• Not discussed above, but random() could be a good one to add. The thing to figure out would be how to keep track of the random seed (maybe we make a global variable that we initially set to something pixel or vertex dependent, and update it on each random() call?)
• Also not discussed above, but noise() could also be good. This might be implemented the same way as map since it doesn't need state like random does

[LalitNarayanYadav]

Hi! 👋

I'm interested in contributing to this issue, especially the lerp alias for mix. However, I’m having trouble locating the relevant file or module where mix is currently defined, or where lerp should be added within the p5.strands structure.

Could you please point me to the correct file or folder in the repo?

Thanks in advance!

[davepagurek]

Thanks @LalitNarayanYadav! Currently, mix is created here, where we loop over the GLSL functions we want to make JavaScript equivalents of (and the rest of p5.strands is in this file too):

p5.js/src/webgl/ShaderGenerator.js

Lines 1564 to 1567 in 2606c21

	'mix': [
	{ args: ['genType', 'genType', 'genType'], returnType: 'genType', isp5Function: false},
	{ args: ['genType', 'genType', 'float'], returnType: 'genType', isp5Function: false},
	],

Later on in the file, it loops through the things in this list, creating a new p5.strands function out of them. The array of args options specify the different overloads the function has, and genType in this case gets filled in with a specific vector type (e.g. vec2, vec3, vec4) based on what a user provides.

Currently, the part later on that generates the functions assumes the name in the list (fnName) is both the name the user can call it by, and also the name of the GLSL function it should translate to:

p5.js/src/webgl/ShaderGenerator.js

Lines 1598 to 1620 in 2606c21

	Object.entries(builtInGLSLFunctions).forEach(([functionName, properties]) => {
	const isp5Function = Array.isArray(properties) ? properties[0].isp5Function : properties.isp5Function;
	if (isp5Function) {
	const originalFn = fn[functionName];
	fn[functionName] = function (...args) {
	if (GLOBAL_SHADER?.isGenerating) {
	return fnNodeConstructor(functionName, args, properties)
	} else {
	return originalFn.apply(this, args);
	}
	}
	} else {
	fn[functionName] = function (...args) {
	if (GLOBAL_SHADER?.isGenerating) {
	return new fnNodeConstructor(functionName, args, properties);
	} else {
	p5._friendlyError(
	`It looks like you've called ${functionName} outside of a shader's modify() function.`
	);
	}
	}
	}
	})

I could see us solving this maybe by manually creating a similar function to the snippet above, but outside of the loop, that sets fn.lerp to the same thing that mix got assigned to?

[LalitNarayanYadav]

Thanks so much, @davepagurek , that makes it a lot clearer!

Just to confirm my understanding: since the function creation loop uses the same name for both the JS function and GLSL mapping, you're suggesting that I manually define fn.lerp in the same way as mix, but outside the loop, basically assigning fn.lerp = fn.mix so users can call either one?

Also, would it be okay to add a short comment there explaining the purpose of the alias for clarity?

Happy to take this up if it sounds good!

[davepagurek]

It'll be a bit more complicated because setting fn.lerp updates the lerp function in all contexts, not just in p5.strands. The code that currently overrides p5 functions (the isp5Function branch in the snippet above) handles that by getting a reference to the original function, and then overriding it. Inside the overridden function, it only calls the new implementation if we are inside a p5.strands callback (GLOBAL_SHADER?.isGenerating), and otherwise just calls the original function. So I think you'd want to replicate that block of logic by either calling fn.mix if inside of strands, or calling the original lerp implementation otherwise.

[davepagurek]

Also, would it be okay to add a short comment there explaining the purpose of the alias for clarity?

That would be great, thanks!

[LalitNarayanYadav]

Thanks for the clarification, @davepagurek!

So to implement the lerp alias correctly, I should:

• Save a reference to the original lerp function,
• Override fn.lerp with a function that checks if we're inside a p5.strands callback (GLOBAL_SHADER?.isGenerating),
• If yes, call the mix implementation,
• Otherwise, call the original lerp function.

I’ll also add a clear comment explaining the aliasing purpose as you suggested.

Please let me know if this plan sounds good, and I’ll start working on it!

[davepagurek]

That sounds great, thanks!