Skip to content

Image uniforms get reset after each draw using a shader #7030

Closed
Forchapeatl/p5.js
#1
@nakednous

Description

@nakednous
Contributor

Most appropriate sub-area of p5.js?

  • Accessibility
    Color
    Core/Environment/Rendering
    Data
    DOM
    Events
    Image
    IO
    Math
    Typography
    Utilities
    WebGL
    Build process
    Unit testing
    Internationalization
    Friendly errors
    Other (specify if possible)

p5.js version

1.6.0 onwards

Web browser and version

No response

Operating system

No response

Steps to reproduce this

I've implemented shadow mapping in p5.js v1.5.0 using p5.Graphics and the p5.treegl library. While this works seamlessly in v1.5.0, I'm facing challenges with versions from p5.js v1.6.0 onwards. I am planning to transition to using p5.Framebuffer once this issue is resolved.

Steps to Reproduce:

  1. Run the shadow mapping implementation in p5.js v1.5.0 here.
  2. Attempt the same using p5.js v1.9.3 here.

I've verified that the p5.treegl commands function identically in both versions. Notably, p5.js v1.6.0 introduced significant updates to shader-related features. I'm unsure if this issue qualifies as a bug or if I should report it or ask for help at the p5.js forum.

@davepagurek, could you possibly take a look and provide some guidance?

Activity

davepagurek

davepagurek commented on May 9, 2024

@davepagurek
Contributor

Thanks for catching this! After #5923, textures are unbound when the shader is unbound to prevent some other bugs (keeping textures bound when not using them causes a lot of other downstream bugs). However, we unbind the shader immediately after each draw, so only the first shape has the depth texture bound, and then subsequent shapes have an empty texture. Right now, it means I guess you'll have to re-apply the uniform before each shape you draw.

To fix this in p5, maybe we need to keep track of what textures shader had bound the last time it was used so that we can re-bind those values when you use the shader again. That way textures will work like other uniforms.

nakednous

nakednous commented on May 14, 2024

@nakednous
ContributorAuthor

Thanks for the suggestion! I can confirm that reapplying the depth uniform before rendering each shape serves as a workaround.

changed the title [-]shadow mapping[/-] [+]Image uniforms get reset after each draw using a shader[/+] on May 21, 2024
davepagurek

davepagurek commented on May 21, 2024

@davepagurek
Contributor

If anyone is interested in taking this on, I think what we would need to do is adjust setUniform to store the last set texture on the shader:

case gl.SAMPLER_2D:
gl.activeTexture(gl.TEXTURE0 + uniform.samplerIndex);
uniform.texture =
data instanceof p5.Texture ? data : this._renderer.getTexture(data);

...and then re-apply the last used texture value, if it exists, when you bind the shader again in _setFillUniforms and _setStrokeUniforms:

_setFillUniforms(fillShader) {
fillShader.bindShader();

One complication: since this gets called after each draw finishes, this call below would end up storing an empty image when we really just want it to unbind the current image:

unbindTextures() {
for (const uniform of this.samplers) {
this.setUniform(uniform.name, this._renderer._getEmptyTexture());
}
}

So maybe unbindTextures should just do this bit instead of calling setUniform?

uniform.texture =
data instanceof p5.Texture ? data : this._renderer.getTexture(data);

moved this to Bugs with Solutions (No PR or test) in p5.js WebGL Projectson May 21, 2024
JordanSucher

JordanSucher commented on May 22, 2024

@JordanSucher
Contributor

I tried to recreate #5923 so I could understand the issues that arise if we don't unbind a texture, but I found that when I commented out the unbindTextures() call, it didn't seem to cause any problems - tried the sketch from that issue, fill renders as expected.

Is it possible we don't need to have that call anymore?

red.cube.mov
davepagurek

davepagurek commented on May 22, 2024

@davepagurek
Contributor

I made a new sketch on the most recent p5 here: https://editor.p5js.org/davepagurek/sketches/pwIAXyF0X

With the implementation of unbindTextures empty, the fills still stop working for me and I get the same error about framebuffer feedback in the console.

JordanSucher

JordanSucher commented on May 22, 2024

@JordanSucher
Contributor

very strange - that sketch works fine for me across chrome, safari, arc

nakednous

nakednous commented on May 22, 2024

@nakednous
ContributorAuthor

I tested this on Arch Linux.
It fills the texture in both Chromium and Falkon, but in Firefox it only does it the first frame (mousePress to reveal it).

davepagurek

davepagurek commented on May 22, 2024

@davepagurek
Contributor

It's a weird situation because it's left up to the driver developers to determine what's considered feedback or not, so on some platforms it works. Unfortunately we mostly have to program to the lowest common denominator, and for now that means cleaning up our textures when not using them.

nakednous

nakednous commented on May 22, 2024

@nakednous
ContributorAuthor

weirdly enough leaving unbindTextures empty, solves the shadow mapping issue here even in firefox.

3 remaining items

davepagurek

davepagurek commented on Aug 26, 2024

@davepagurek
Contributor

While it does solve the issue here, it also would mean that while some textures remain bound, others do not, which could lead to confusion for users. To keep consistency between texture types, I think we'd still need to record what used to be bound before we unbind it so that it can be re-bound.

Forchapeatl

Forchapeatl commented on Aug 26, 2024

@Forchapeatl
Contributor

I think we'd still need to record what used to be bound before we unbind it so that it can be re-bound.

@davepagurek , please for an example of this scenario . Why would we re-bound after unbinding ?

Forchapeatl

Forchapeatl commented on Aug 28, 2024

@Forchapeatl
Contributor

Hi , @davepagurek . Please, I am still struggling to understand this logic

I think we'd still need to record what used to be bound before we unbind it so that it can be re-bound.

  bindTextures() {
    const gl = this._renderer.GL;

    for (const uniform of this.samplers) {
      let tex = uniform.texture;
      if (tex === undefined) {
        // user hasn't yet supplied a texture for this slot.
        // (or there may not be one--maybe just lighting),
        // so we supply a default texture instead.
        tex = this._renderer._getEmptyTexture();
      }
      gl.activeTexture(gl.TEXTURE0 + uniform.samplerIndex);
      tex.bindTexture();
      console.log(tex);
      tex.update();
      gl.uniform1i(uniform.location, uniform.samplerIndex);
    }
  }

Forgive me if I sound silly or ignorant but , it seems like every texture gets bounded here. From my logs i can see that textures get bounded. My question is , If textures are bounded already there is no need to unbind them ( the code works when unbindTextures function is empty ) , why do we have to track them ?

davepagurek

davepagurek commented on Aug 28, 2024

@davepagurek
Contributor

I think there are two parts here, first being why we unbind things at all. This issue #5921 happens if you leave certain framebuffers bound. Essentially: if you have a framebuffer bound as an input (as a uniform to a shader), many WebGL implementations prevent you from also using it as an output (drawing contents to it.) In that issue, the code isn't actually trying to both read from and draw to the framebuffer at the same time, it only reads from the framebuffer within the push/pop where it uses the shader, but if p5 leaves the uniforms bound, then the browser doesn't know we won't read from it while we're writing. So by unbinding after we're finished drawing something that reads from it, we avoid that issue.

The problem that comes up, then, is if you try to draw two different things in a row with a texture bound. Because we unbind after a draw, this code will no longer work as expected:

shader(myShader)
myShader.setUniform('someTexture', myFramebuffer)

// This one works, the framebuffer is still bound
model(modelA)
// After the draw, it gets unbound immediately.

// Since it got unbound, the framebuffer still won't be set here, even though it looks
// like it should be present.
model(modelB)

This isn't the only possible solution, but my suggestion was to separate the concepts of (1) what textures are currently bound, and (2) what the user last passed into setUniform. The idea would be, setUniform just records what the user wants to be set. When we draw an object, we first bind that recorded texture, then draw, then unbind it. That would update the flow for that code:

shader(myShader)
myShader.setUniform('someTexture', myFramebuffer)
// Here we have recorded that we are filling the someTexture slot with myFramebuffer,
// but it is not yet bound.

// Before draw, we bind myFramebuffer
model(modelA)
// After the draw, it gets unbound, but we still know that that's what should be in someTexture

// Before draw, we bind myFramebuffer again
model(modelB)
// After the draw, we unbind again, and still keep it recorded

// Now we finally update it to something else when setUniform is called again:
myShader.setUniform('someTexture', myFramebuffer2)

Hopefully that example makes sense, let me know if I can clarify anything!

jujpenabe

jujpenabe commented on Aug 28, 2024

@jujpenabe

I'm trying to test a system to store and reapply the previous texture in shaders. Currently, I have the following code in the p5.Shader:

unbindTextures() {
  for (const uniform of this.samplers) {
    const tex = uniform.texture;
    if (tex) {
      // Store the texture as the previous texture
      this._prevTexture = this._prevTexture !== tex ? tex : this._prevTexture;
    }
    this.setUniform(uniform.name, this._renderer._getEmptyTexture());
  }
}

I've added the _prevTexture property to the `p5.Shader class:

constructor(rederer, vertSrc, fragSrc) {
  // other properties
  this._bound = false;
  this._prevTexture = null;
  this.samplers = [];
}

However, I'm unsure about when and how to reapply _prevTexture. I've tried assigning the texture in p5.RendererGL.js after bindShader like this:

fillShader.bindShader();
// Reapply the last texture that was bound if not null
if (fillShader._prevTexture != null) {
  console.log('Applying previous fill textures: ');
  this._tex = fillShader._prevTexture;
}

@davepagurek

  1. Should this be implemented inside bindShader instead?
  2. Or should this logic only be applied to _setFillUniforms and _setStrokeUniforms?

Any guidance is appreciated

davepagurek

davepagurek commented on Aug 28, 2024

@davepagurek
Contributor

Thanks for taking a look! I think maybe the places that would make sense are bindShader and unbindTextures. I'm imagining:

  • In bindShader, you could loop through this.samplers and re-apply uniform.texture for each one
  • In unbindTextures, you can set them all back to empty textures, but update the way it does it: right now I think it does it in a way that will overwrite uniform.texture, so we'd want to make it do everything it currently does except that bit.

I'm suggesting continuing to store it as uniform.texture because you could have multiple of these per shader, not just one, and we already have something that works close to what we want, so it might be a smaller change to adapt that existing system.

Forchapeatl

Forchapeatl commented on Aug 30, 2024

@Forchapeatl
Contributor

Finally solved the issue. It tracks previous boundedTextures / previousBindings.

p5.Shader = class {
    ...
    this.previousBindings = new Map();  // Map to store previous bindings
    this.boundedTextures = new Set();
  }
bindTextures() {
    const gl = this._renderer.GL;
    this.previousBindings = new Map();  // Map to store previous bindings

    for (const uniform of this.samplers) {
        let tex = uniform.texture;
        if (tex === undefined) {
            tex = this._renderer._getEmptyTexture();
        }

        const textureUnit = gl.TEXTURE0 + uniform.samplerIndex;
        gl.activeTexture(textureUnit);

        // Store the previous binding for this texture unit
        const previousTexture = gl.getParameter(gl.TEXTURE_BINDING_2D);
        this.previousBindings.set(textureUnit, previousTexture);

        tex.bindTexture();
        tex.update();
        this.boundedTextures.add(tex);

        gl.uniform1i(uniform.location, uniform.samplerIndex);
    }
}
unbindTextures() {

    const gl = this._renderer.GL;
    for (const uniform of this.samplers) {
        const textureUnit = gl.TEXTURE0 + uniform.samplerIndex;

        if (this.previousBindings.has(textureUnit) === false){
        this.setUniform(uniform.name, this._renderer._getEmptyTexture());
       }
    }
}

The complete code
May I get started on the PR ?

jujpenabe

jujpenabe commented on Aug 31, 2024

@jujpenabe

@Forchapeatl Hi, I tested your code and it's working fine with the OP sketch.
But, for me it's not clear when the boundedTextures Set is used:

    this.boundedTextures = new Set();

Maybe just the previousBindingsMap manages all the logic to solve this bug?

added a commit that references this issue on Nov 3, 2024
2d10678
davepagurek

davepagurek commented on Nov 6, 2024

@davepagurek
Contributor

I've added code to the 2.0 branch that should resolve this issue, so I'm going to mark this as closed. Expect a fix in the 2.0 release early next year!

moved this from Bugs with Solutions (No PR or test) to DONE! 🎉 in p5.js WebGL Projectson Jun 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status

    DONE! 🎉

    Milestone

    No milestone

    Relationships

    None yet

      Development

      Participants

      @nakednous@davepagurek@JordanSucher@jujpenabe@Forchapeatl

      Issue actions

        Image uniforms get reset after each draw using a shader · Issue #7030 · processing/p5.js