p5Shader - Is there a way to specify custom attributes?

[JetStarBlues]

How would this new feature help increase access to p5.js?

Potential to expand capability of p5's WebGL mode.

Most appropriate sub-area of p5.js?

• WebGL

Feature enhancement details:

Is there a way to specify custom attributes when working with shaders?

From what I've gathered from this contributor_doc, these are the available attributes (let me know if this assumption is incorrect):

• attribute vec3 aPosition
• attribute vec3 aNormal
• attribute vec2 aTexCoord
• attribute vec4 aDirection
• attribute vec4 aVertexColor

Suppose I want to pass some custom attributes to a shader. (For example, I want to specify a cube (or other custom geometry such as a 3D model or something generated dynamically) by passing in some vertices as one attribute, and some vertex colors as another attribute). How can I go about this? Is there a setAttribute equivalent to setUniform? If not, is there a way to do this using private p5 methods?

---

Sidenote:
It would be great if the shader documentation on p5js.org/reference was a bit more fleshed out. For example:

• list all available attributes, and give a brief explanation of how they are generated
• give an example of how to use each attribute. (For example the use of beginShape and endShape to specify custom geometry (aPostion), and vertex colors (aVertexColor) (as seen here Enabling lights() in WEBGL breaks vertex colors.  #4274))

---

Related issues:

• Ability to specify vertex normals when drawing 3d shapes with beginShape and vertex #5022
• Allowing per-vertex fill / stroke changes significantly decreases performance #2022

[JalokimFreelance]

I believe you are looking for this:
https://itp-xstory.github.io/p5js-shaders/#/ - site about shaders in p5
https://itp-xstory.github.io/p5js-shaders/#/./docs/using-texture-coordinates - attributes

[JetStarBlues]

@JalokimFreelance Thanks for the link! I'm looking for the ability to define custom attributes. The examples in the linked articles use the predefined ones.

[JetStarBlues]

The key thing I'm hoping for is a setAttribute equivalent to setUniform.
At present, it seems like,

• if you want to specify custom values for aPosition, you can
  • use beginShape, vertex, endShape in the draw loop
  • or, use loadModel to load an STL or OBJ model created with other software
• if you want to specify custom values for aVertexColor
  • the only option is to use beginShape, fill/stroke, vertex, endShape

If a setAttribute method existed, then you could in theory just pass these elements as an array. E.g.

• setAttribute( "a_customVerts", [ [ 0, 0, 0 ], [ 1, 0, 0 ], [ 0.5, 1, 0 ] ] )
• setAttribute( "a_customVertColors", [ [ 1, 0, 0 ], [ 0, 1, 0 ], [ 0, 0, 1 ] ] )
• setAttribute( "a_fancyCustomThing", [ ... ] )

The default attributes (aPosition etc)* would remain as they are, so that everything continues working as it currently does. But if someone wants to get more from p5 and shaders, the option is available to specify custom attributes.

_ _ _.
*which seem to be determined by p5Geometry and p5RendererGL.

[aferriss]

Hi @JetStarBlues

I don't believe such a function exists currently, although I think it could make a good addition to the webgl api. Binding attributes is typically a bit more complex than uniforms and most of the time people rely on the ones provided by the library. You might be able to pack your data into another buffer that you're not using, for instance if you don't need normals you could put your data in there.

I'd also look around https://github.com/processing/p5.js/blob/main/src/webgl/p5.RenderBuffer.js as I think that's where the attribute binding functions are.

If you'd like to take a crack at adding a setAttribute function to the api, you'll also need to make a case for why it increases accessibility. I think there may be a case to make in that it lowers the barrier of entry for beginners in webgl / graphics programming wanting to use attribute buffers, and expands the usefulness of the library on par with some others.

Hope that helps!

[JetStarBlues]

@aferriss Thank you for the detailed reply!

If you'd like to take a crack at adding a setAttribute function to the api, you'll also need to make a case for why it increases accessibility. I think there may be a case to make in that it lowers the barrier of entry for beginners in webgl / graphics programming wanting to use attribute buffers, and expands the usefulness of the library on par with some others.

I'm still learning many of these concepts, but I'll try to give it a shot. If anyone more experienced wants to give it a go, please feel free to do so.

You might be able to pack your data into another buffer that you're not using, for instance if you don't need normals you could put your data in there.

How would one do this? It seems that the various geometry constructors fill the attributes with values. How would one then overwrite these values? (ex one, two)

[aferriss]

Well, I think with the default shapes (rect, box, ellipse, etc) you won't be able to because there's no way to hold a reference to the shapes (I could be wrong about this but this is how I remember). This is actually something that could be nice to have too, so that you could loop over the shape's vertices, uvs, normals, etc. It's kind of outside of the processing / p5 paradigm of declaratively drawing shapes and maybe more similar to something like three.js.

However, if you use a mesh with loadModel() generated in blender / c4d / maya, you can put whatever you want in the texture coordinates or normals channels. So it would just be a matter of packing that data in your 3d software of choice.

This is definitely an area where the webGL api needs some work. I think there were some tasks related to allowing per vertex colors floating around at some point. Searching through the closed issues might bring them up.

[JetStarBlues]

Sorry for the late reply!
Thank you for the explanation!
Haven't had time to sit down and tackle this, but will update accordingly.

[JetStarBlues]

Hi @aferriss

It doesn't look like it is possible to render a model using a custom shader.

I copied the loadModel example and created the following shaders:

v.vert

attribute vec3 aPosition;
attribute vec4 aVertexColor;

varying vec3 vVertexColor;

void main ()
{
	// vVertexColor = aVertexColor.rgb;
	vVertexColor = vec3( 1.0, 0.0, 0.0 );  // red

	vec4 positionVec4 = vec4( aPosition, 1.0 );

	positionVec4.xy = positionVec4.xy * 2.0 - 1.0;

	gl_Position = positionVec4;
}

f.frag

varying vec3 vVertexColor;

void main ()
{
	gl_FragColor = vec4( vVertexColor, 1.0 );
}

The main program looks like this,

let octahedron;
let shader1;

function preload ()
{
	octahedron = loadModel( 'octahedron.obj' );

	shader1 = loadShader( "v.vert", "f.frag" );
}

function setup ()
{
	createCanvas( 100, 100, WEBGL );

	// Disable line shader ??
	// noStroke();

	// Disable fill shader ??
	// noFill();
}

function draw ()
{
	shader( shader1 );

	clear();
	rotateX( frameCount * 0.01 );
	rotateY( frameCount * 0.01 );
	model( octahedron );

	// rect( 0, 0, 0, 0 );
	// ellipse( 0, 0, 0, 0 );
	// triangle( 0, 0, 0, 0, 0, 0 );
}

Expected result, the octahedron is red. However, the model seems to be rendered by the default shaders (for material and lines). Setting noFill() does nothing.

When the rect( 0, 0, 0, 0 ) is selected instead of the model, a red plane appears as (somewhat 😕) expected. (Similarly for ellipse( 0, 0, 0, 0 ) and triangle( 0, 0, 0, 0, 0, 0 )).

It seems there's no clean way to hack the existing interface to use custom attributes.

[JetStarBlues]

I spoke too soon.
Using the undocumented(!) default uniforms seen in your example here (uniform mat4 uProjectionMatrix and uniform mat4 uModelViewMatrix), I was able to get the octahedron to render red.

v.vert

attribute vec3 aPosition;
attribute vec4 aVertexColor;

uniform mat4 uProjectionMatrix;
uniform mat4 uModelViewMatrix;

varying vec3 vVertexColor;

void main ()
{
	// vVertexColor = aVertexColor.rgb;
	vVertexColor = vec3( 1.0, 0.0, 0.0 );  // red

	vec4 positionVec4 = vec4( aPosition, 1.0 );

	// positionVec4.xy = positionVec4.xy * 2.0 - 1.0;

	// gl_Position = positionVec4;

	gl_Position = uProjectionMatrix * uModelViewMatrix * positionVec4;
}

[aferriss]

It might not be appearing because you aren't using any of the matrices for projection. Without looking more closely I can't say for certain, but a typical vertex shader for a 3d object looks like:

attribute vec3 aPosition;
attribute vec4 aVertexColor;

uniform mat4 uModelViewMatrix;
uniform mat4 uProjectionMatrix;

varying vec4 vColor;

void main(void) {
  vec4 positionVec4 = vec4(aPosition, 1.0);
  gl_Position = uProjectionMatrix * uModelViewMatrix * positionVec4;
  vColor = aVertexColor;
}

You can see a couple more of the shaders here: https://github.com/processing/p5.js/tree/c1640a7cb11c25145e9b9dd561ba27f7b49fcde4/src/webgl/shaders

You might also need to look into the obj loader to ensure that it is properly reading color from the mesh as well.

I'm afraid you're nudging up against the boundaries of what has been implemented in p5. You might try checking out three.js if p5 isn't able to get you where you're trying to go. It's much more fully featured when dealing with 3d and webGL.

[JetStarBlues]

I'm afraid you're nudging up against the boundaries of what has been implemented in p5. You might try checking out three.js if p5 isn't able to get you where you're trying to go. It's much more fully featured when dealing with 3d and webGL.

Ah, I see.

For posterity, here's a link to the final working code I put together. There are three variants:

• sketch.js uses loadModel

  • relies on undocumented behaviour (p5.Geometry default buffer properties (name, content type), default attribute names)

• sketch2.js directly invokes a new instance of p5.Geometry

  • relies on undocumented behaviour (p5.Geometry default buffer properties (name, content type), default attribute names)

• sketch3.js uses gl commands

   let renderer = createCanvas( 400, 400, WEBGL );
   let gl = renderer.GL;

  • relies on undocumented behaviour (p5.Shader.bindShader())

[aferriss]

@JetStarBlues These are great examples! Thanks for taking the time to share them. It seems like you were able to get pretty close to what you were aiming for. It's great to see the variety of approaches, and I think definitely helps illuminate areas in the webGL api that could use more work.

One area where help is always welcome is documentation, so if you see anything that you think should be updated, or was never documented in the first place, I think those would be very welcome PR's.

I've never seen anyone directly with the p5.Geometry class like that either so contributing those examples could be very valuable for other newcomers to the 3d api.

[JetStarBlues]

Oh! Happy to hear =D! (Thanks for taking the time to reply to these sporadic posts!)

One area where help is always welcome is documentation, so if you see anything that you think should be updated, or was never documented in the first place, I think those would be very welcome PR's.
I've never seen anyone directly with the p5.Geometry class like that either so contributing those examples could be very valuable for other newcomers to the 3d api.

Noted. Will give it a try.

[JetStarBlues]

@aferriss Came up with one more implementation idea.
Like sketch2.js, sketch4.js creates a new instance of p5.Geometry. However instead of overriding existing buffers, it creates new ones. I've added it to the gist. This seems to be the cleanest of the four.

let m1;
let shader1;
let renderer;

function preload ()
{
	shader1 = loadShader( "vert3.glsl", "frag.glsl" );
}

function setup ()
{
	renderer = createCanvas( 400, 400, WEBGL );

	m1 = new p5.Geometry();
	m1.gid = "uniqueName1";

	m1.vertices = [

		new p5.Vector(      0,   0,     40 ),
		new p5.Vector(   22.5,   22.5,  0  ),
		new p5.Vector(   22.5, - 22.5,  0  ),
		new p5.Vector( - 22.5, - 22.5,  0  ),
		new p5.Vector( - 22.5,   22.5,  0  ),
		new p5.Vector(      0,   0,   - 40 ),
	];

	m1.faces = [

		[ 0, 1, 2 ],
		[ 0, 2, 3 ],
		[ 0, 3, 4 ],
		[ 0, 4, 1 ],
		[ 5, 4, 3 ],
		[ 5, 3, 2 ],
		[ 5, 2, 1 ],
		[ 5, 1, 4 ]
	];

	// Custom attributes
		m1.custom_vertexColors = [

			1, 0, 0,
			0, 1, 0,
			0, 1, 0,
			0, 1, 0,
			0, 1, 0,
			1, 0, 0,
		];

		// https://github.com/processing/p5.js/blob/374acfb44588bfd565c54d61264df197d798d121/src/webgl/p5.RendererGL.js#L151
		renderer.retainedMode.buffers.fill.push(

			new p5.RenderBuffer(

				3,                            // number of components per vertex
				'custom_vertexColors',        // src
				'custom_vertexColorsBuffer',  // dst
				'aCustomVertexColor',         // attribute name
				renderer                      // renderer
			)
		);
}

function draw ()
{
	shader( shader1 );

	clear();
	rotateX( frameCount * 0.01 );
	rotateY( frameCount * 0.01 );

	model( m1 );
}

[JetStarBlues]

It will always create buffers of type Float32Array, but I think that's good enough for most cases.

[aferriss]

This is great! I really didn't know that you could use p5 to make procedural geometry in this way. I definitely think we should consider documenting this workflow. @stalgiag what do you think about this?

I moved some of your examples to the editor and simplified them as well:

Simple Triangle
Simple Plane
Textured Plane
Custom Particle System

The last example of custom particles, I was able to render 100,000 particles on my macbook at 60fps!

[JetStarBlues]

Cool examples!
The id attribute reminds me of the vertexId attribute in the vertexshaderart demos. Will try porting one and see how it goes. I think we have most of what's needed to do so... (minus the ability to specify the mode argument of drawElements/drawArrays).

[stalgiag]

This is really great. I know it isn't very in line with the rest of the API but I often find myself wishing that p5.Geometry was made with the intent that users would make their own. This could be accomplished with changes as simple as automatically generating the uniqueID and adding a function that adds a p5.Geometry object to the buffers.

[JetStarBlues]

I think one other thing to consider for this approach, is making the way that values are specified consistent. For the default buffers, vertices and vertexNormals are specified as p5.Vectors whereas vertexColors and uvs are specified as a list. For an end user, this is an arbitrary distinction (arbitrary rule to remember).

[aferriss]

@JetStarBlues Completely agree. I actually got briefly stumped by this when I was making my examples.

@stalgiag agreed! It feels like all the major pieces are already there and it would just take a little push to make it friendlier.

[JetStarBlues]

Here is an example illustrating how access to the uv values can be used for uv mapping.

[JetStarBlues]

@aferriss It occurred to me that a similar technique could be used to "use a shader as a texture". The current example in the documentation uses createGraphics. Here is the same example tweaked to skip this step. I'm planning to create an issue on the p5.js-website repo where I cover in more detail the changes involved. But I wanted to share it quickly here before calling it a day.

[JetStarBlues]

An update.

Found out how to update the buffer values "on the fly".
_prepareBuffer() (called via model() > drawBuffers()), checks the following to see if it should update the buffer values:

• the buffer does not exist
• or model.dirtyFlags[this.src] !== false

It looks like _prepareBuffer() was designed to support updates! In a sketch it looks something like this:

function draw() {
  // modify vertices
  ...

  // mark vertices as dirty
  m.dirtyFlags.vertices = true

  // draw the model. Internal call to _prepareBuffer() will update buffers
  model(m);
}

Here is a simple live version of this in action. An additional hack is required to propogate the vertex updates to the default stroke shader.

Here is another demo. This one is based on Coding Challenge: 3D Supershapes. In index.html, you can switch between using Immediate Mode and Retained Mode geometry. I am not sure if there is a performance difference between the two approaches.

(I find it surprising that the immediate mode version can handle more vertices (via nSubdivisions e.g. 50) than the retained mode... Maybe it has to do with the vertices being stored as p5.Vectors and thus needing to be flattened every frame when _prepareBuffer() is called)

[somecho]

Hi! This might be unrelated to this issue, but is there a way using this method to also define the primitive rendering mode like LINES or TRIANGLE_STRIP?

[lakshz]

If this open, can I take this?

[davepagurek]

This is open, but needs a bit of discussion about how it should be solved before we get to the implementation. Do you have an idea of how you want to approach the problem, like what the API would look like for users?

[Qianqianye]

Adding WebGL custom attributes contributor @lukeplowden @davepagurek in this discussion.

[davepagurek]

Closing this now that #7276 is merged in! This will be coming in p5 2.0 🙂