Extension: Static lightmap textures

[stevenvergenz]

An initial pass on an extension to provide precomputed lighting and global illumination to glTF materials. I'm going to attempt to implement this in the UnityGLTF project, but I want to run it by this group too. Thoughts?

[lexaknyazev]

Since glTF is a data format, it's fine to not provide exact implementation details, but some basic data properties should be provided to enable interoperability.

So, could you elaborate a bit more about texture data:

• grayscale or RGB;
• alpha channel;
• sRGB or linear colorspace.

[stevenvergenz]

Updated

[stevenvergenz]

After thinking about how this extension will work with a texture atlas (my primary use case), I'm considering moving this extension out of the material, and onto the node. From a data economy point of view, this makes the most sense as every node with a lightmap is going to have a separate texture from an otherwise identical but differently-placed node. This may force some engines to do a fair bit of duplication (Three.js and Blender come to mind), but it's nothing that model authors or loader contributors wouldn't have to do anyway. Lightmaps are hard in those engines.

Any objections?

[pjcozzi]

Sounds OK to me.

[donmccurdy]

...every node with a lightmap is going to have a separate texture from an otherwise identical but differently-placed node. This may force some engines to do a fair bit of duplication (Three.js and Blender come to mind)...

I haven't used lightmaps before, but curious what you mean by this. In three.js, can you just use the material.lightMap property in the same way that other textures are set?

[stevenvergenz]

Exactly. If light maps are unique to the object, it means that sharing materials is impossible (different lightMap texture), unless you're using an atlas, in which case sharing the texture object is impossible (different tile settings).

On Mon, Jun 19, 2017, 7:16 PM Don McCurdy ***@***.***> wrote: ...every node with a lightmap is going to have a separate texture from an otherwise identical but differently-placed node. This may force some engines to do a fair bit of duplication (Three.js and Blender come to mind)... I haven't used lightmaps before, but curious what you mean by this. In three.js, can you just use the material.lightMap property <https://threejs.org/docs/#api/materials/MeshStandardMaterial.lightMap> in the same way that other textures are set? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1017 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABy5CKlNuMq6JNc-hPZvwOtKuli4kQMjks5sFyuQgaJpZM4N4F3d> .

-- --- Steven Vergenz, AltspaceVR steven@altvr.com

[donmccurdy]

One side effect of moving the light maps to the node: a mesh may contain more than one primitive, and the primitives may use different materials. It will not be possible to apply the lightmap to one primitive's material but not another's.

Also, while the idea of sharing materials but allowing different lightmaps makes good sense, will editors/exporters support this? In Maya at least, lightmaps are associated with a material, so practically it seems unlikely that you would merge those materials at time of export.

EDIT: On the other hand, if environment maps will be an extension to the node (#946 (comment)) then it would not be so inconsistent to do that here.

[stevenvergenz]

I'm not sure about implementation across engines, but I put the lightmap data on nodes because I couldn't imagine a scenario where it would be reused. Regardless of engine, that produces the most compact gltf in this case. And it sort of worked out that Unity does its lightmaps that way.

On Fri, Jun 23, 2017, 7:41 PM Don McCurdy ***@***.***> wrote: One side effect of moving the light maps to the node: a "mesh" may contain more than one primitive, and the primitives may use different materials. It will not be possible to apply the lightmap to one primitive's material but not another's. Also, while the idea of sharing materials but allowing different lightmaps makes good sense, can editors/exporters support this? In Maya at least, lightmaps are associated with a material, so practically it seems unlikely that you would merge those materials at time of export. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1017 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABy5CFyeB85RNt-3MELTLsGAcg18wTixks5sHHdAgaJpZM4N4F3d> .

-- --- Steven Vergenz, AltspaceVR steven@altvr.com

[pjcozzi]

@McNopper OK to close as duplicate with #947 and #945?

[pjcozzi]

Sorry, wrong PR.

[electrowolff]

Is this still being worked on? I feel like having the light map on the node is incorrect. Why would the light map be handled any different from the occlusion map, which is specified at the material level?

[stevenvergenz]

I think extension work had been put on the back burner while the 2.0 spec was being finalized. Hopefully we'll see some more activity here soon?

Regarding lightmap location justification, see my previous responses. Basically, occlusion is constant for all instances of a mesh, but the lightmap is highly dependent on position/orientation/scale, which is only defined for nodes.

[electrowolff]

Maybe I'm missing something about occlusion maps, but it's not constant across mesh instances? Take a contrived example of two instances of a mesh, one out in the open and one fully inside a box. The one inside the box would be fully occluded, while the one outside is not occluded at all. Maybe what I'm saying is the occlusion map should be on the node too?

[stevenvergenz]

You're technically right, but I understand that "ambient occlusion" is usually implictly self-occlusion only. So position relative to the rest of the scene wouldn't matter.

[Kupoman]

Ambient occlusion usually takes global lighting into account, which is what you might see in a light map. Occlusion maps for PBR are also sometimes called cavity or cavity occlusion maps, and are just based on the geometry.

[Ben-Mack]

+1 for lightmap. It especially helpful for static scene and being used a lot in games, hoping it will be implemented soon 🙏

[AnneKitsune]

👍 Anything going on here?

[donmccurdy]

I see lightmaps have already been mentioned there, but for others in this thread, note that we are looking for feedback on the next iteration of glTF materials: #1442

[pjcozzi]

@donmccurdy there's a lot of interest in lightmaps, e.g., #1017 (comment), does it make sense to keep this extension PR open or keep this conversation as part of next-gen PBR, #1442, potentially influenced by this PR?

[donmccurdy]

I think lightmaps are orthogonal to next-gen PBR. If there’s enough interest and supporters to push the extension it could even be created before next-gen PBR, and (with some care) should be forward- and backward-compatible.

I don’t use lightmaps in my own workflow and can’t do this myself right now, but if interested parties want to identify both engines and authoring tools where lightmaps are supported and this extension can likely be implemented, that would be good progress.

I also think that more detail in this section is necessary:

The details of how this lightmap is applied will depend on the lighting model in use by the consumer.

With AO we’ve found some similar questions, and if those details (or the range of possibilities) can be clarified early on it will benefit implementers.

[AnneKitsune]

Currently I am using blender to bake the lightmap directly onto textures before exporting to glTF. The issue with that is that the whole baking process in blender needs to happen for each object present in the scene, one at a time, using a custom script (no built-in way to do multiple objects at once). A glTF lightmap extension would be great!

[idorurez]

Hi all, looking to see if we can start spec'ing lightmaps. Thinking of proposing FB_materials_lightmap or some extension of sort

[donmccurdy]

@idorurez Would you be willing to open a new Pull Request and help address feedback/questions there? I think this PR stalled on time/availability. I believe a lightmap extension would still be a welcome addition to glTF, although more detail on the client implementation will be necessary. Even if some clients handle lightmaps differently today, we should do the work of understanding the most common implementations.

[idorurez]

Sure. Though, we're getting things figured out here first, and I'll submit a pull request once we have more details on our implementation.

…

On Sat, Jul 20, 2019 at 1:55 PM Don McCurdy ***@***.***> wrote: @idorurez <https://github.com/idorurez> Would you be willing to open a new Pull Request and help address feedback/questions there? I think this PR stalled on time/availability. I believe a lightmap extension would still be a welcome addition to glTF, although more detail on the client implementation will be necessary. Even if some clients handle lightmaps differently today, we should do the work of understanding the most common implementations. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1017?email_source=notifications&email_token=ADX6SJACTQODRTIH622BWF3QAN3UBA5CNFSM4DPALXO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2NV3RQ#issuecomment-513498566>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADX6SJFEBMTYHR26P4DFRRLQAN3UBANCNFSM4DPALXOQ> .

[idorurez]

Opened up a pull request, draft for a vendor spec out

#1658

[donmccurdy]

Moving this discussion to #1658 – please add feedback, needs, and comparison of implementations there!