Proposal to align documentation to X3D and glTF standards for IfcSurfaceTexture

[Moult]

Note: related to #155. Textures affect the lighting model, so they must be read together.

Credits to Ilkin, mentored by Jakob Beetz, and Michalis from X3D who have helped research, implement, and come to this proposal. However, I have just typed out this proposal, and they have not yet read it yet so do not assume they support this just yet :)

Further reading: https://community.osarch.org/discussion/877/understanding-how-textures-and-shaders-work-in-ifc

Current situation

To our knowledge we're the first people not only to implement this in IFC... but also one of the few people who care about it in the X3D world. IFC seemed to have chosen one of the most obscure and buggily implemented aspects of the X3D spec to reference. That said, referencing X3D is awesome because they can do all the domain research and we can follow them. For example, X3D has since heavily simplified the texture systems to match the behaviour shown in glTF, so the current situation in IFC is already outdated, but we can catch up pretty quickly.

It's difficult to summarise the full extents of the technical implications of the current situation, but here's an attempt to put it into terms that most 3D artists would appreciate.

• Out of the box, IFC supports a diffuse texture. Other maps: specular, normal, are not supported.
• You cannot simply specify "use this image texture". A texture has implicit default blending modes in a stack of zero or more textures.
• Your texture can be made out of multiple textures, combined one after another using blending modes.
• Out of the box, officially not all blending modes are supported (e.g. multiply is, but overlay is not).
• Out of the box, officially not all blending mode intensities are supported (e.g. mix with a factor of 0.2 is supported, but add or multiply cannot have a factor of 0.2, it is locked to 1 or 0)
• Out of the box, you can blend with a flat colour, but only if that is the very first thing you do in your stack of textures. You cannot do it midway.
• Maybe, with a very ugly hack I don't like, we can blend with a flat colour anywhere in the stack by defining a 1px texture in the IFC code. Nasty but in theory it works.
• Out of the box, X3D (and so IFC) actually allows you to apply blending modes after lighting is calculated, not prior. This means that not only the IFC support mixing textures prior to rendering, it inadvertently also supports describing compositing. However spec only allows for you to composite on an image rendered with obsolete techniques like a Gouraud shader. So in practice this is impossible to implement with modern tools.
• Out of the box, the rules about how blend modes are applied are not intuitive to artists. This is not necessarily a bug, as the implementation allows for the usual implementation we would expect in software like the GIMP / Photoshop / Krita / Blender / etc, but I highly suspect other implementers will get confused and we might get buggy implementations.
• Out of the box, X3D v3.2 also has confusing rules about treating grayscale images differently by default from RGB textures. Again, not strictly a bug, but very likely implementers will get confused and we will get buggy implementations. This has been fixed in X3D v4.
• Movie textures are not supported in IFC, but are supported in X3D. Just an FYI.
• Procedural textures (noise, brick, musgrave, voronoi) are not supported
• Some other modifiers like some forms of vector math and colour invert is supported.
• Other common modifiers like hue / saturation, colour curves, RGB to BW, etc, are not supported.

Proposal description

There are three proposals which solve the limitations of the current situation, shown in order of priority.

1. With an unofficial convention inspired from X3D, we can nominate other types of maps: specular, normal, etc. The usual stuff. Highest priority - artists need maps.
2. With an unofficial convention inspired from X3D, we can remove ambiguities on grayscale processing and alpha blending to prevent buggy implementations that X3D has suffered from. These are exciting toys but not critical, and in fact "sneakily hidden" in X3D due to their lack of popularity.
3. With an unofficial convention we can support all blending modes and support factor intensities for all blending modes. These are exciting toys but not critical, and in fact "sneakily hidden" in X3D due to their lack of popularity.

Proposal 1: add back texture maps types

This existed in IFC2X3 and was removed for some reason. X3D v3.2 doesn't support texture maps but X3D v4 does. The solution is simple: a fifth parameter is added to the Parameter attribute list designating which texture map type the IfcSurfaceTexture is part of. This solution is similar o the X3D v4 approach. The fifth parameter must include the type of texture map chosen from the following list. Note that we have based the names off IFC2X3, but updated it to match X3D / glTF naming.

Map	PHONG	NOTDEFINED (Physcial)	FLAT (Unlit)
AMBIENT	Yes	No	No
NORMAL (upgraded from BUMP)	Yes	Yes	Yes
EMISSIVE (called SELFILLUMINATION in IFC2X3)	Yes	Yes	Yes
METALLICROUGHNESS	No	Yes	No
OCCLUSION	Yes	Yes	No
SHININESS	Yes	No	No
SPECULAR (called REFLECTION in IFC2X3)	Yes	No	No
DIFFUSE (called TEXTURE in IFC2X3)	Yes	Yes	No

For example:

#2=IfcImageTexture($,$,'REPLACE',$,('','','1 1 1','1','DIFFUSE'),'data/pattern.png')

This sentence needs to be added to the description of the Parameter attribute:

Parameter[5] shall hold the type of map this corresponds to, chosen from the following list: AMBIENT, NORMAL, EMISSIVE, METALLICROUGHNESS, OCCLUSION, SHININESS, SPECULAR, DIFFUSE. For more information on which texture maps apply in different lighting models refer to the X3D specification.

Proposal 2: Prevent buggy implementations by making blending and grayscale handling unambiguous and more intuitive

First, to fix the grayscale bug, a few lines need to be added below this existing documentation:

Texture are defined by 2D images that contain an array of colour values describing the texture. The texture values are interpreted differently depending on the number of components in the texture and the specifics of the image format. In general, texture may be described using one of the following forms:

Intensity textures (one-component)
Intensity plus alpha opacity textures (two-component)
Full RGB textures (three-component)
Full RGB plus alpha opacity textures (four-component)

The lines to be added at the bottom are:

A grayscale texture is exactly equivalent to using an RGB texture with all 3 components (red, green, blue) equal.

A texture without an alpha channel is exactly equivalent to using a texture with an alpha channel filled with 1 (indicating opaque).

Now to fix the blending ambiguity, X3D modes used by IFC by default apply to all channels: RGBA. X3D lets you specify the blending mode separately for RGB and A. To fix the ambiguity, we should enforce the implementer to always specify the blending mode for RGB and A separately. To fix this, this description should be added to the "Mode" attribute of IfcSurfaceTexture:

The value chosen for the mode field must also specify the blending mode for the RGB channel separately from the blending mode for the alpha channel separated by whitespace. For example "MODULATE REPLACE" specifies Color = (Arg1.color × Arg2.color, Arg1.alpha) as per the X3D specification.

For example:

#1=IfcSurfaceStyleWithTextures((#2,#3))
#2=IfcImageTexture($,$,'MODULATE REPLACE',$,('','','1 1 1','1'),'data/pattern.png')
#3=IfcImageTexture($,$,'SUBTRACT ADD',$,('','','1 1 1','1'),'data/squirrel.png')

Proposal 3: support modern blending modes

I assume most implementers would stick to a single image, because even in X3D it is a very niche topic and the consensus in X3D seems to be moving towards a simplified "single" texture input just like glTF, instead of stacks of blended textures.

However, given that the current scenario is referencing multitextures (i.e. blended stacks of textures) we could easily extend the list of modes. Some ones already exist, like MODULATE, and ADD, so here is a list that could be added:

• OVERLAY
• DARKEN
• LIGHTEN
• SCREEN
• DIFFERENCE

See https://en.wikipedia.org/wiki/Blend_modes . Also any mode can have the suffix "FACTORALPHA" so that the blending may be applied with a user customised factor intensity. This is a feature artists would expect.

To include this in IFC4.3 it is as simple as adding to the "Mode" attribute description of IfcSurfaceTexture:

In addition to the X3D modes, the following modes are allowed: OVERLAY, DARKEN, LIGHTEN, SCREEN, DIFFERENCE. Modes may be suffixed with FACTORALPHA to apply an factor based on the alpha parameter.

[Moult]

Note: further reading on the exact details of this proposal is documented by Ilkin here (though I have made some more adjustments since this PDF was written):

A proposal for how multitexturing should work in IFC4.pdf

[Moult]

So after some discussion with @aothms and @TLiebich it seems as though taking a step back the fact that nobody in 15 years in the X3D world have really appreciated multitextures, and given that they haven't garnered interest in it, it is unlikely to provide much value in IFC. Also, once you start stacking nodes, there is a blurred line between usefulness and when node stacking becomes non-interoperable between rendering engines.

So as a result, I'd like to revise the proposal to follow the direction of X3D and glTF and remove the multitexture support. So no more blending modes. No more texture stacks. No more "list of 4 string parameters where you need to read the X3D spec that even X3D people never reads to fully understand". This will hopefully encourage implementers and not scare them by forcing them to read obscure X3D spec portions to understand blending modes and edge cases. The primary goal is to encourage implementers to build texture support.

Here is the revised proposal (note the original proposal 2 and 3 are now obsolete):

Proposal description

There are two proposals which solve the limitations of the current situation, and heavily simplify textures to follow the direction of X3D and glTF.

1. With an unofficial convention inspired from X3D, we can nominate other types of maps: specular, normal, etc. The usual stuff. Highest priority - artists need maps.
2. Deprecate multitexture support and revert to a maximum of 1 texture per texture type

Proposal 1: add back texture maps types

This existed in IFC2X3 and was removed for some reason. X3D v3.2 doesn't support texture maps but X3D v4 does. The solution is simple: convert the mode attribute into an attribute which designates which texture map type the IfcSurfaceTexture is part of. This solution is similar to the X3D v4 approach. The mode attribute will be chosen from the following list. Note that we have based the names off IFC2X3, but updated it to match X3D / glTF naming.

Map	PHONG	NOTDEFINED (Physcial)	FLAT (Unlit)
AMBIENT	Yes	No	No
NORMAL (upgraded from BUMP)	Yes	Yes	No
EMISSIVE (called SELFILLUMINATION in IFC2X3)	Yes	Yes	Yes
METALLICROUGHNESS	No	Yes	No
OCCLUSION	Yes	Yes	No
SHININESS	Yes	No	No
SPECULAR (called REFLECTION in IFC2X3)	Yes	No	No
DIFFUSE (called TEXTURE in IFC2X3)	Yes	Yes	No

For example:

#2=IfcImageTexture($,$,'DIFFUSE',$,$,'data/pattern.png')

This is the new description of the Mode attribute:

Mode shall hold the type of map this corresponds to, chosen from the following list: AMBIENT, NORMAL, EMISSIVE, METALLICROUGHNESS, OCCLUSION, SHININESS, SPECULAR, DIFFUSE. For more information on which texture maps apply in different lighting models refer to the X3D specification.

Proposal 2: Remove multitexture support

Multitexture support is previously implemented in IFC using the mode attribute and the parameter attribute. Simple solution: deprecate the Parameter attribute.

Also replace this sentence:

If multiple surface textures are included in the IfcSurfaceStyleWithTextures applying them to a geometric item, a mode and optional parameters can be included that blending operations.

... with ...

Only a single texture may be supplied per texture Mode type (DIFFUSE, SPECULAR, etc).

[Moult]

Marking as decided since this was already discussed without issues :)

[aothms]

Can we summarize the schema and doc changes that need to occur?