Closed
Description
Most appropriate sub-area of p5.js?
- AccessibilityColorCore/Environment/RenderingDataDOMEventsImageIOMathTypographyUtilitiesWebGLBuild processUnit testingInternationalizationFriendly errorsOther (specify if possible)
p5.js version
1.6.0 onwards
Web browser and version
No response
Operating system
No response
Steps to reproduce this
I've implemented shadow mapping in p5.js v1.5.0 using p5.Graphics
and the p5.treegl library. While this works seamlessly in v1.5.0, I'm facing challenges with versions from p5.js v1.6.0 onwards. I am planning to transition to using p5.Framebuffer
once this issue is resolved.
Steps to Reproduce:
- Run the shadow mapping implementation in p5.js v1.5.0 here.
- Attempt the same using p5.js v1.9.3 here.
I've verified that the p5.treegl commands function identically in both versions. Notably, p5.js v1.6.0 introduced significant updates to shader-related features. I'm unsure if this issue qualifies as a bug or if I should report it or ask for help at the p5.js forum.
@davepagurek, could you possibly take a look and provide some guidance?
Metadata
Metadata
Assignees
Type
Projects
Status
DONE! 🎉
Activity
davepagurek commentedon May 9, 2024
Thanks for catching this! After #5923, textures are unbound when the shader is unbound to prevent some other bugs (keeping textures bound when not using them causes a lot of other downstream bugs). However, we unbind the shader immediately after each draw, so only the first shape has the depth texture bound, and then subsequent shapes have an empty texture. Right now, it means I guess you'll have to re-apply the uniform before each shape you draw.
To fix this in p5, maybe we need to keep track of what textures shader had bound the last time it was used so that we can re-bind those values when you use the shader again. That way textures will work like other uniforms.
nakednous commentedon May 14, 2024
Thanks for the suggestion! I can confirm that reapplying the depth uniform before rendering each shape serves as a workaround.
[-]shadow mapping[/-][+]Image uniforms get reset after each draw using a shader[/+]davepagurek commentedon May 21, 2024
If anyone is interested in taking this on, I think what we would need to do is adjust
setUniform
to store the last set texture on the shader:p5.js/src/webgl/p5.Shader.js
Lines 957 to 960 in bcf9134
...and then re-apply the last used texture value, if it exists, when you bind the shader again in
_setFillUniforms
and_setStrokeUniforms
:p5.js/src/webgl/p5.RendererGL.js
Lines 2071 to 2072 in bcf9134
One complication: since this gets called after each draw finishes, this call below would end up storing an empty image when we really just want it to unbind the current image:
p5.js/src/webgl/p5.Shader.js
Lines 573 to 577 in bcf9134
So maybe
unbindTextures
should just do this bit instead of callingsetUniform
?p5.js/src/webgl/p5.Shader.js
Lines 959 to 960 in bcf9134
JordanSucher commentedon May 22, 2024
I tried to recreate #5923 so I could understand the issues that arise if we don't unbind a texture, but I found that when I commented out the unbindTextures() call, it didn't seem to cause any problems - tried the sketch from that issue, fill renders as expected.
Is it possible we don't need to have that call anymore?
red.cube.mov
davepagurek commentedon May 22, 2024
I made a new sketch on the most recent p5 here: https://editor.p5js.org/davepagurek/sketches/pwIAXyF0X
With the implementation of
unbindTextures
empty, the fills still stop working for me and I get the same error about framebuffer feedback in the console.JordanSucher commentedon May 22, 2024
very strange - that sketch works fine for me across chrome, safari, arc
nakednous commentedon May 22, 2024
I tested this on Arch Linux.
It fills the texture in both Chromium and Falkon, but in Firefox it only does it the first frame (mousePress to reveal it).
davepagurek commentedon May 22, 2024
It's a weird situation because it's left up to the driver developers to determine what's considered feedback or not, so on some platforms it works. Unfortunately we mostly have to program to the lowest common denominator, and for now that means cleaning up our textures when not using them.
nakednous commentedon May 22, 2024
weirdly enough leaving
unbindTextures
empty, solves the shadow mapping issue here even in firefox.3 remaining items
davepagurek commentedon Aug 26, 2024
While it does solve the issue here, it also would mean that while some textures remain bound, others do not, which could lead to confusion for users. To keep consistency between texture types, I think we'd still need to record what used to be bound before we unbind it so that it can be re-bound.
Forchapeatl commentedon Aug 26, 2024
@davepagurek , please for an example of this scenario . Why would we re-bound after unbinding ?
Forchapeatl commentedon Aug 28, 2024
Hi , @davepagurek . Please, I am still struggling to understand this logic
Forgive me if I sound silly or ignorant but , it seems like every texture gets bounded here. From my logs i can see that textures get bounded. My question is , If textures are bounded already there is no need to unbind them ( the code works when
unbindTextures
function is empty ) , why do we have to track them ?davepagurek commentedon Aug 28, 2024
I think there are two parts here, first being why we unbind things at all. This issue #5921 happens if you leave certain framebuffers bound. Essentially: if you have a framebuffer bound as an input (as a uniform to a shader), many WebGL implementations prevent you from also using it as an output (drawing contents to it.) In that issue, the code isn't actually trying to both read from and draw to the framebuffer at the same time, it only reads from the framebuffer within the
push
/pop
where it uses the shader, but if p5 leaves the uniforms bound, then the browser doesn't know we won't read from it while we're writing. So by unbinding after we're finished drawing something that reads from it, we avoid that issue.The problem that comes up, then, is if you try to draw two different things in a row with a texture bound. Because we unbind after a draw, this code will no longer work as expected:
This isn't the only possible solution, but my suggestion was to separate the concepts of (1) what textures are currently bound, and (2) what the user last passed into
setUniform
. The idea would be,setUniform
just records what the user wants to be set. When we draw an object, we first bind that recorded texture, then draw, then unbind it. That would update the flow for that code:Hopefully that example makes sense, let me know if I can clarify anything!
jujpenabe commentedon Aug 28, 2024
I'm trying to test a system to store and reapply the previous texture in shaders. Currently, I have the following code in the
p5.Shader
:I've added the _prevTexture property to the `p5.Shader class:
However, I'm unsure about when and how to reapply
_prevTexture
. I've tried assigning the texture inp5.RendererGL.js
afterbindShader
like this:@davepagurek
Any guidance is appreciated
davepagurek commentedon Aug 28, 2024
Thanks for taking a look! I think maybe the places that would make sense are
bindShader
andunbindTextures
. I'm imagining:bindShader
, you could loop throughthis.samplers
and re-applyuniform.texture
for each oneunbindTextures
, you can set them all back to empty textures, but update the way it does it: right now I think it does it in a way that will overwriteuniform.texture
, so we'd want to make it do everything it currently does except that bit.I'm suggesting continuing to store it as
uniform.texture
because you could have multiple of these per shader, not just one, and we already have something that works close to what we want, so it might be a smaller change to adapt that existing system.Forchapeatl commentedon Aug 30, 2024
Finally solved the issue. It tracks previous boundedTextures / previousBindings.
The complete code
May I get started on the PR ?
jujpenabe commentedon Aug 31, 2024
@Forchapeatl Hi, I tested your code and it's working fine with the OP sketch.
But, for me it's not clear when the
boundedTextures
Set is used:Maybe just the
previousBindings
Map manages all the logic to solve this bug?Fix #7030
davepagurek commentedon Nov 6, 2024
I've added code to the 2.0 branch that should resolve this issue, so I'm going to mark this as closed. Expect a fix in the 2.0 release early next year!