Serialize lights into gfx-replay json format.

[0mdc]

Motivation and Context

This changeset adds light serialization to the gfx-replay format.

Lights are serialized incrementally - they are only written to the json file when lighting changes occur. We assume that the keyframes are always read sequentially.

`GfxReplayTest::testLightIntegration` Replay File

{
    "keyframes": [
        {
            "lightsChanged": true,
            "lights": [
                {
                    "vector": [
                        1.5,
                        2.0,
                        2.5,
                        1.0
                    ],
                    "color": [
                        1.5,
                        2.0,
                        5.0
                    ],
                    "model": "global"
                },
                {
                    "vector": [
                        -10.0,
                        4.25,
                        10.0,
                        1.0
                    ],
                    "color": [
                        5.0,
                        5.0,
                        0.0
                    ],
                    "model": "camera"
                }
            ]
        },
        {
            "lightsChanged": true,
            "lights": [
                {
                    "vector": [
                        0.0,
                        1.2,
                        -4.0,
                        1.0
                    ],
                    "color": [
                        4.0,
                        4.0,
                        4.0
                    ],
                    "model": "object"
                }
            ]
        },
        {},
        {
            "lightsChanged": true,
            "lights": [
                {
                    "vector": [
                        0.1,
                        0.2,
                        -0.3,
                        0.0
                    ],
                    "color": [
                        0.0,
                        0.0,
                        1.0
                    ],
                    "model": "global"
                }
            ]
        },
        {
            "lightsChanged": true
        }
    ]
}

How Has This Been Tested

• New unit test (GfxReplayTest::testLightIntegration).

Types of changes

• Docs change / refactoring / dependency upgrade
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have completed my CLA (see CONTRIBUTING)
• I have added tests to cover my changes.
• All new and existing tests passed.

[eundersander]

Looking good so far. I left a few comments.

[eundersander]

I'm a little torn on serializing our entire quirky LightInfo struct (including LightPositionModel). I talked in the design doc about wanting gfx-replay to be agnostic to the renderer, and most renderers IMO won't offer "object-local" or "camera-local" lights. I think we only need to support lights specified in global space. Also, I don't think anybody is using object-local or camera-local lights in Habitat. I don't think we'll bother implementing them in the new batch renderer.

See also this discussion here: https://cvmlp.slack.com/archives/G0131KVLBLL/p1660649432149979 .

It is probably fine to leave this as-is, but let's beware introducing unneeded complexity into the gfx-replay format in general.

[0mdc]

I completely agree.

I feel like it doesn't hurt to leave the LightPositionModel field as-is for now. We should look into specifying a standard light data layout as we integrate them with the batch renderer.

I am still unfamiliar with the batch renderer, but if we are following a pure ECS approach, we might want to separate the transform component from the light data.

I think we only need to support lights specified in global space.

Object-local lights may be occasionally useful (e.g. if the simulated agent has a light). However, I don't think that we are doing anything like that at the moment.

I don't know how camera-local lights would be useful.

[Skylion007]

Vector3 stores floats right? Why do we keep calling getDouble? Are we also serializing it as a double? These storing floats as doubles add up.

[mosra]

JSON's only numeric type is doubles, yes. Not sure if rapidjson even has 32-bit-float APIs, usually there isn't any significant difference in parsing as a float or a double.

[Skylion007]

Oh darn, didn't know that about JSON. Yet another reason to avoid using it for our dataset.jsons in Habitat-Lab. 0-0 You can set the max decimal places in RapidJSON globally it appears, but other than that no. :(

[mosra]

Corrade's Utility::Json has an option to switch between floats / doubles (and 32/64bit integer types), but not even there it makes any difference memory-usage-wise (it abuses the IEEE floating-point layout to store integers inside a double representation, and in all cases it's stored directly in parsed token memory that just has to be of certain size), and until I have my own optimized floating-point parser that has a dedicated code path for 32bit floats, it won't make any difference for parsing speed either. It's there purely for convenience.

But limiting decimal places (or excessive line wraps / indentation for arrays) could make a difference in subsequent parsing time, yes.

[eundersander]

Related: see usePrettyWriter and maxDecimalPlaces in writeJsonToFile. I added this in a half-hearted attempt to minimize the filesize of gfx-replay files.

[eundersander]

LGTM!