Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
[WebGL] Reconcile the global matrix #637
To start, recall that 3D rendering typically uses three matrices; called "model matrix", "view matrix", and "projection matrix". Working backwards, melonJS needs the projection matrix to transform our pixel coordinates to WebGL cube space coordinates using orthographic projection; and a view matrix is used for nested containers and other object that need relative drawing operations. The model matrix is unnecessary because the majority of entities drawn in melonJS are quads (exactly two triangles). A model matrix is more useful with large meshes, which have a greater number of vertices.
The scaling and rotation values can be streamed to the GPU as part of the vertex attribute array. These can be packed into 2 floats (a 3x3 matrix requires 6 floats for full precision) assuming 16 bits for each of the scaling coordinates. Additionally, the rotation angle and texture index can be packed into the same float. The anchor point will also be required, which is a 3rd float. These attributes can be applied within the vertex shader, since the order of operations is static; translation can be added to the destination coordinates before calling
Entities that need a model matrix can implement one with a custom compositor and shader. Spine is an example that will benefit from a model matrix.
To reconcile the global matrix, we need to remove most of the matrix transformation usage.
With that part done, each view matrix transformation will be treated as change in WebGL state; it will be sent to the GPU as a uniform variable, so changes to the matrix require flushing the compositor. This will be pretty efficient, assuming view matrix transformations only occur on
Additional TODO items:
Attribute Packing: E.g. texture coordinates can be packed into a single float, since each axis will never be bigger than 65,536 pixels (fits into 16-bits). The packing algorithms will provide less than 16-bits of precision, probably 15-bits, which is still plenty for texture coordinates. Another candidate is colors, and much bigger win: pack a
@krojew i see that you have been working yourself on a webgl library, any chance you want to help us here optimize/complete our webgl renderer ? We still have a lots of improvements listed in this ticket but as well #720 and #593.
@parasyte can also provides further explanation on our implemetation :)
@obiot I would be glad to help, but I have very limited time atm. I've not been able to even complete the typings yet.
But, I can give some quick tips for now from skimming over the description:
now that the WebGL renderer is enabled by default with the platformer, I noticed that Firefox is also complaining a bit about it :
Error: WebGL: texImage2D: Incurred CPU pixel conversion, which is very slow. melonJS.js:17367:17 Error: WebGL: texImage2D: Chosen format/type incurred an expensive reformat: 0x1908/0x1401 melonJS.js:17367:17 Error: WebGL: texImage2D: Incurred CPU-side conversion, which is very slow. melonJS.js:17370:17 Error: WebGL: texImage2D: Incurred CPU pixel conversion, which is very slow. melonJS.js:17370:17 Error: WebGL: texImage2D: Chosen format/type incurred an expensive reformat: 0x1908/0x1401 melonJS.js:17370:17
which leads to the two following lines :
@parasyte is this somehting you would maybe have the time/be willing to look at ?