FLIP_Y usage

[lexaknyazev]

Why UNPACK_FLIP_Y_WEBGL is false here?
https://github.com/KhronosGroup/glTF-WebGL-PBR/blob/b8128dd1bc8cbd6e165631efbf5f629d3a5b2407/scene.js#L451

glTF 2.0 spec says that it should be true for WebGL:
https://github.com/KhronosGroup/glTF/tree/master/specification/2.0#images

[javagl]

I had seen this hint, and wondered where it came from. My first guess for the answer here would be: "When it is set to true, then the textures are upside down".

But to understand this correctly: Does this change between 1.0 and 2.0 mean that all textures of all sample models will have to be flipped?

[lexaknyazev]

glTF 1.0 implied WebGL convention (HTML images start with upper left corner). So all other backends had to flip Y. glTF 2.0 uses lower left corner as origin, so only WebGL-based engines must flip textures.

If our 2.0 sample models don't follow this, we must fix something (tools, spec, or models?) ASAP.

[javagl]

This flipping is a distressingly common issue, always. At least it once was also an issue at CesiumGS/obj2gltf#13 , but I think that OBJ does not specify anything regarding the texture coordinates. I also stumbled over this in javagl/gltfTutorialModels#2 , and even added information about the texture coordinates in the texture image of https://github.com/javagl/gltfTutorialModels/tree/2.0/SimpleTexture , but now, I'm (once more) not sure whether this is right. (Fortunately, this model is not part of the official sample models right now, so I'm tempted to lean back and watch how this is being resolved...)

[bghgary]

From glTF 2.0 image section:

First image pixel corresponds to the lower left corner of the image.

After investigating this a bit, I think this statement is not correct and we should change it.

All exporters that I’m aware of all export images with the first pixel corresponding to the top left corner of the image. And all WebGL client implementations that I know of do not flip Y on load. For the sake of simplicity, I think we should update the spec to match the client implementations and samples.

Edited: client implementations --> WebGL client implementations

[lexaknyazev]

Looks like I got this issue wrong in a comment above. Only OpenGL (ES) uses lower left corner as an origin, while others (D3D, Vulkan, Metal) expect upper left corner to have the first pixel.

So the spec is not correct and we must fix that section.

[bghgary]

Only OpenGL (ES) uses lower left corner as an origin, while others (D3D, Vulkan, Metal) expect upper left corner to have the first pixel.

This part is confusing. This is my understanding.

When you say origin, you are strictly talking about how to access the GPU texture in memory when using texture coordinates. In other words, a texture coordinate of (0, 0) is the bottom-left of the GPU texture in memory for OpenGL as opposed to top-left for DirectX.

When textures are stored to disk formats, the first pixel is the top-left of the image for typical image viewers/loaders.

According to WebGL texImage2D documentation:

The first pixel transferred from the source to the WebGL implementation corresponds to the upper left corner of the source. This behavior is modified by the UNPACK_FLIP_Y_WEBGL pixel storage parameter, except for ImageBitmap arguments, as described in the abovementioned section.

That means the default behavior is that WebGL will treat the first pixel (0, 0) as the top-left of the image, which in texture coordinates is (0, 1). This essentially means that texImage2D flips the texture by default when using an image source.

[bghgary]

So the spec is not correct and we must fix that section.

+1

[lexaknyazev]

WebGL's default behavior doesn't match OpenGL. When UNPACK_FLIP_Y_WEBGL is true, WebGL behaves like OpenGL.

[bghgary]

WebGL's default behavior doesn't match OpenGL

I think you are talking about the difference between the functions WebGL texImage2D and OpenGL glTexImage2D.

OpenGL documentation for glTexImage2D:

The first element corresponds to the lower left corner of the texture image. Subsequent elements progress left-to-right through the remaining texels in the lowest row of the texture image, and then in successively higher rows of the texture image. The final element corresponds to the upper right corner of the texture image.

So, yes, that is true. But texture coordinates (0,0) is still the bottom-left of the texture regardless of what texImage2D or glTexImage2D is doing, right?

[lexaknyazev]

So, yes, that is true. But texture coordinates (0,0) is still the bottom-left of the texture regardless of what texImage2D or glTexImage2D is doing, right?

Sorry, I'm not following.

Vertex buffer (two tris):

[-1.0, -1.0, 1.0, -1.0, 1.0, 1.0, -1.0, -1.0, 1.0, 1.0, -1.0, 1.0]

Vertex shader:

attribute vec2 pos;
varying vec2 v_tc;
void main() {
  v_tc = pos * 0.5 + 0.5;
  gl_Position = vec4(pos.x, pos.y, 0.0, 1.0);
}

Fragment shader:

precision highp float;
varying vec2 v_tc;
uniform sampler2D tex;
void main() {
  gl_FragColor = texture2D(tex, v_tc);
}

texture2D(tex, vec2(0.0, 0.0)) gives upper left pixel and an image appears upside-down by default. The reason is that viewport's Y axis is pointing upwards.

[bghgary]

Sorry, I'm still trying to understand it completely. I think we just need to specify what texture coordinate maps to the first pixel of the image. (My understanding for PNGs and JPGs is that the first pixel is the top-left of the image when viewed in a standard image viewer.) Two separate parts can affect this:

1. Loading the image into a GPU texture.
2. Texture coordinate convention.

Flipping Y on either part will flip the Y. Flipping both will cancel each other out.

For the implementations/samples that are out there right now, texture coordinate (0,0) maps to the first pixel of the image.

[bghgary]

For example, I created a Quad in Unity and mucked with the shader so that it showed UV.y as a gradient. Brighter = larger values. Here the first pixel of the image has UV(0,1)

Then I exported the Quad into a glTF using my exporter and put it into BabylonJS and mucked with the shader in a similar way. Here the first pixel of the image has UV(0,0)

[javagl]

So when applying the texture from https://raw.githubusercontent.com/javagl/gltfTutorialModels/2.0/SimpleTexture/glTF/testTexture.png to a quad, using the texture coordinates that are shown in the image, is it then rendered properly, or flipped?

[lexaknyazev]

Just keep in mind, that on-screen OpenGL/WebGL coordinates are:

EDIT: initial version of image above showed positions normalized to [0; 1]. That isn't correct for gl_Position output.

[javagl]

Right, that's the positions. For the "SimpleTexture" model, I originally used these positions, and the same values for the texture coordinates (just because it seemed nice, intuitive and consistent).

But this caused some confusion, because the initial rendering then appeared to be upside down (see CesiumGS/cesium#4808 (comment) ). I finally flipped the texture coordinates to

0,0     1,0

0,1     1,1

to let it look right. I'm still not sure whether this is only necessary for WebGL-based implementations and/or depending on whether the specific WebGL-based implementation by default assumes FLIP_Y or not.

Just for reference, some related issues from the glTF repo: KhronosGroup/glTF#674 , KhronosGroup/glTF#736

[lexaknyazev]

So OpenGL treatment of texture data is:

The image itself (pointed to by data) is a sequence of groups of values. The first group is the lower left corner of the texture image. Subsequent groups fill out rows of width width from left to right; height rows are stacked from bottom to top forming the image.

Counting from zero, each resulting N_th texel is assigned internal integer coordinates (i; j), where
i = N mod width
j = floor(N / width) mod height

That means that tex coords (0, 0) always point to lower left corner of the image in OpenGL.

But in WebGL, the same tex coords (0, 0) will point to the upper left corner.

---

See KhronosGroup/glTF#1005.