TextureMatrix not functioning correctly

[ajayns]

I've tried reproducing the issue in a simpler environment but was unfortunately unable to do so in the first attempt, so I'm hoping you have so insight just by reading the details of my problem itself.
This is how the layout looks like (it has no error but almost similar thing shows up an error in my local env): https://codepen.io/ajayns/pen/XWmQLNL

The plane's image unlike how it's expected to be (like object-fit: cover) it's showing up in stretched dimensions or zoom in and other glitches. Any input on how I can correct the uTextureMatrix0 uniform?

I have a feeling it was something to do with the layout but then I recreated that in the codepen in which it seems to work fine, now I suspect it's got something to do with the shader and that's why I'm looking for ideas.

[martinlaxenaire]

Hey @ajayns!

Are you by any chance instancing your plane before the window load event actually occurs? Because this could lead to some wrong plane sizes calculations.

You could try to use the resize() method of your texture (or even of your plane) in the window load event and/or your plane onLoading callback and see if it solves it.

[ajayns]

I did try using the resize() method for plane and texture to no avail. It seems like the plane has some issues getting the right coordinates from the dom plane element.

Since I really don't have much else to provide to showcase the issue, do you have any insights into how I could generate the matrix manually or do something similar based on the size of the plane dom element? Hopefully, I might be able to find the cause of the problem when I try that.

[martinlaxenaire]

Well, that's weird!
Have you tried comparing both plane.getBoundingRect() and plane.htmlElement.getBoundingClientRect() values to see if there's any difference?
Also have you checked your texture _size property to see if it's correct?

You could still pass your plane and texture width and height values as two vec2 uniforms to your vertex shader, and calculate the ratios in there and apply them to your aTextureCoord attribute. Or do it directly in js and send them as uniform afterwards.

I'd still like to find out what could go wrong there!

Cheers,

[ajayns]

You could still pass your plane and texture width and height values as two vec2 uniforms to your vertex shader, and calculate the ratios in there and apply them to your aTextureCoord attribute. Or do it directly in js and send them as uniform afterwards.

Ahh that looks like it could solve the issue for my case even though it is a hacky fix.

I'd still like to find out what could go wrong there!

Same it's a pretty weird bug where even I'm unsure what's going wrong even with my going all around the problem and attempting a simplified reproduction.
The tips you've mentioned to debug does look handy, I'll check that out properly and get back to you on this.

Thanks a bunch for the help :)

[ajayns]

It gets stranger, the values seem to be calculated right:

Getting hacky, what seems to work for this case is:

  vTextureCoord = vec2(.86, 1., 0., 1.) * aTextureCoord;

(this isn't the exact calculated value nor does it place the image in the center, but I got this by fiddling on dat.gui to match aspect ratio - it implies that the calculations are off somewhere)
As for the vTextureCoord we get by using TextureMatrix and TextureCoord, I can't seem to find any multiplication or so that fixes it.

Also texture matrix I'm using in case that's relevant:

[martinlaxenaire]

So to sum up, all the sizes values are correct, so there's no reason the texture matrix calculations could be wrong either.

Actually I've just made a quick test locally with a plane and texture of the exact same sizes. Got the same texture matrix, and the rendered texture looks just fine.

I honestly don't know what could be wrong here.
Maybe the issues' coming from something you're doing inside your shaders?

[ajayns]

I doubted that myself and so I ran it using the most basic vertex and fragment shaders which basically just passing the varying vTextureCoord and then using that the image is displayed. Still, the issue persisted.

From what I see I believe there's something going wrong with the calculations itself as I'd mentioned the uTextureMatrix seems to mess it up while me using a custom ratio to aTextureCoord seem to get the image to display.

But yes, I still have no clue what exactly is going wrong either, so I'll put this on hold, try other things, and eventually maybe I could spot what's happening here.

[martinlaxenaire]

Allright, then maybe the texture matrix uniform is not being updated somehow?
Silly question: is the bug still there when you resize your window?

[ajayns]

Sheesh, I found and resolved the error, that was quite the headache. I'm sorry for all the trouble but you were right, it had to do with the shaders.

In my local repo, I've added a couple of textures to the plane and I'm switching between them like your async textures loading example. Since I was using a bunch of images of the exact same size, I'd assumed I could just use the texture matrix of just one of them for all the images as the size and fit were the same. And that's where I went wrong haha, I added the individual matrices for each texture and that fixed the issue.

Also, any clue why my assumption was wrong? If textures are of exact same size, shouldn't the matrices be similar? Sorry, if this is a dumb question and I'm completely unaware of something

[ajayns]

Closing the issue, and I'm sorry again for all the trouble!

[martinlaxenaire]

Glad you found the bug :)

If the textures all have the exact same sizes, they should indeed have the same texture matrices as well.
Now that you know the correct texture marix values, you can use them to debug what matrix is wrong tho, and maybe find out why it was buggy in the first place.

[ajayns]

Yes, I'll take some time to look into that as well. Now knowing that my assumption was right, it is indeed a very weird bug.