Colour space and bit depth

[PLAN8VR]

What is the maximum colour gamut and bit depth brush can handle?

Ie, is there any point supplying 16/32 bit pngs in rec2020 etc, or should the images to process just be flattened down to srgb and 8bit?

What gains/losses are had with higher quality colour spaces?

[ArthurBrussee]

In theory the gaussian splatting formulation could be extended to handle higher bit depths. If you have a test dataset you should give it a try, it might already +- work, but no gaurantees haha

[PLAN8VR]

Well, actually it was because Supersplat was showing banding typically associated with bit depth issues, turns out that's what it was, but to do with multiple layers of thin splats being stacked and the 8bit interpretation not being good enough to display colours correctly. But it got me thinking about the value of supplying higher bit depth images for processing.

I'll do some tests with 8,16,32 bit colour and see what happens.

But I guess the main question is, can Brush only cope with 8 bit? If so, it would be diminishing returns to supply higher bit depth images I would assume.

But thinking purely from a photographic viewpoint, it is a serious point to consider. On very high end production, which GS is of course heading towards with AR VR etc, bit depth handling is quite an important consideration for end result imagery.

[PLAN8VR]

Also, it would be good to clarify the colour space. I am assuming srgb is the the biggest space that can be used?

Again, thinking of the future of AR VR usage, headsets that might be able to display higher colour spaces, as well as film production workflows, this is also an important point to consider.

At present, mostly splats are being displayed on web, so web safe SRGB makes sense. But in photography work flow, it's generally best to stay in the higher colour spaces until the final output depending on what the intended purposes are (print, web, film, games etc)

So in pure workflow terms, where does the requirement stop for colour space interpolation? Seems to me that the Gaussian splat processing software should be working with higher colour space and bit depth to preserve fidelity, and at export this is where the flattening should be applied - option in export settings for ply as 8/16/32 and srgb/adobe srgb/rec2020 etc etc depending on delivery intentions.

In simple terms, Gaussian Splat processing is for all intents and purposes, an extension of the photographic workflow and should preserve the highest level of photographic dynamic range possible. The delivery format is where the colour intent should be compressed or preserved.

[simonbethke]

Here is an (ugly) test-data set with 16bit PNG files. https://drive.google.com/file/d/1ZED7j8iipnvQYDoB4KJIaYjsx8JsDAEV/view?usp=sharing

If brush was actually training with the full 16bit, it would produce more details on the dark side:

[PLAN8VR]

I did some inconclusive tests comparing 8/16/32 bit files in png and exr format. I was surprised to discover that the exr is accepted by brush, however the largest colour space Brush appears to accept it srgb - I tried Adobe rgb, which looked ok, but I'm not 100% convinced that it's not clipping out the bandwidth. If EXR / HDR workflow is to be included, I think the colour space is quite an important consideration, as the higher bit depth range can utilise this data and provide a better tonal range.

So, I trained a 16bit exr in adobe rgb. and the export seemed to have maintained the dynamic range beyond 1 when testing the result in supersplat. However, I'm not sure that visually there was much difference than training on an 8bit png as mentioned above by Simon. It looks like the dynamic range is being clipped or something.

Having said that, I don't know the technicalities of what's going on under the hood, so these are more or less blind tests. However, it does seem that there is a good case for pushing further into higher dynamic and colour ranges, it seems that Postshot already does this, and for obvious pipeline reasons, it would be great to push Brush to its maximum pipeline potential quality.

The following video is a test training with 27 stacked, 6 stop HDR images from three bracketed exposures. The exported format was 16bit EXR, using Adobe RGB colour space. With the help of @mvaligursky_59448 we identified in Supersplat that Brush is exporting HDR ranged ply, as identified by the histogram showing colour data above the 1 cnt value.

2025-12-04.08-50-33.webm

[PLAN8VR]

This is the stacked HDR raw image in Darktable demonstrating the available dynamic range.

2025-12-04.09-06-48.webm

[PLAN8VR]

This is the dataset for testing (comes with apologies for poor focus and sub optimal alignment)

https://drive.google.com/file/d/1SW5wTXDJW-xkmlg8pkJyMW7jzqrL59t5/view?usp=sharing

[PLAN8VR]

Here is an (ugly) test-data set with 16bit PNG files. https://drive.google.com/file/d/1ZED7j8iipnvQYDoB4KJIaYjsx8JsDAEV/view?usp=sharing

If brush was actually training with the full 16bit, it would produce more details on the dark side:

Have you tried opening the same dataset in blender with KIRI engine? I think that it appears to be able to interpret the colour and dynamic range better. It would be interesting to see if you find the same with that dark export.

It's interesting that Arthur was saying with hdr images, the processing appears to favour the brighter areas.

So at the moment I'm finding that the data might well survive intact during export, but currently there doesn't appear to be a viewer that can correctly "tone map" the data. So if with a mostly dark data set, you increase "exposure" for example in supersplat. It increases between 0-1 meaning the image becomes flat. It doesn't use the full dynamic range. Same in blender kiri engine. But I will experiment more with this.

It would be cool if the "exposure" controls could leverage the full dynamic range.

[PLAN8VR]

I'm suggesting as per Arthur's comments that the viewer may well be screwed up, but that the data is actually in there, just currently the viewers aren't able to pull it out/ I'm just interested to see if you have valuable useable data in the dark areas, or if indeed, they are just dark with very little dynamic bit data,

[simonbethke]

As you can see at the absence of splats, the data is not there

[PLAN8VR]

I can't see, because it's black.. I don't know if there are black splats in there or nothing at all. If there are splats in there, then they may well have more colour data in them than you can see.

What colour space did you export the images as?

[simonbethke]

Sure you can. Every splat creates a blue dot. Don't say you didnt know...

[PLAN8VR]

Ahh, OK, I realised this after posting. So, what colour space did you export in?

[PLAN8VR]

After a few more tests bringing in high dynamic range exr datasets, using different colour spaces, it seems that Brush is perfectly capable of working with and outputting large hdr splats. This is really encouraging, but it seems to me that whilst it will train with the required colour spaces like linear rec 2020 and aces2065-1, it seems that brush can't read the image data properly. So I assume (not knowing enough about it) that there is no colour space transform capability to ensure correct interpolation of the larger colour space.

For example, a dataset imported as aces2065-1 will display as a dark, underexposed image, and will train accordingly. If brush could understand colour transforms, then I think it should be able to output high quality, truly HDR splats.

Again, I don't know enough about how this is dealt with, but I'm thinking it might not be all that hard to get this workflow working.

If a small set of colour space transforms could be added pre splat algorithm, I'd suggest linear rec 2020 as it is common and reasonably browser friendly, rec2100 hdr, as well as aces2065-1 as it is a massive space that seems to be future compatible, then I think HDR workflow could be possible.

I'm more than happy to supply some datasets in different colour spaces for testing.

In the meantime, I will post some examples of what I am seeing using different colour spaces in the exr hdr datasets.

[PLAN8VR]

So, the three images are just to demonstrate that brush is interpreting all images (no mater their colour space) as SRGB.

This is the same image in all 3 screen shots, which is in aces2065-1 colour space.

The two blender shots show it being displayed in the correct colour space and in SRGB . The SRGB version is identical to brush's interpretation, demonstrating that Brush incorrectly reads it as SRGB.

If Brush could understand (transform) the ACES2065-1 colour space, it would render splats in HDR correctly I believe.

[PLAN8VR]

This image shows a dataset with emitters set at a power of 1 (two white monkeys) and a red emitter set at a power of 1000 (red monkey) - The dataset is exr, but exported in SRGB colour space.

So, as Brush works in SRGB, the colours are correct, but the values over 1 get completely ignored as the colour gamut is out of scope for srgb.

This demonstrates that SRGB colour space is not suitable for exr HDR datasets. - IE Brush needs to be able to interpret larger colour spaces in order to work with true HDR images.

[simonbethke]

I think the assumption that brush works in srgb is not correct. Brush is color(-space) agnostic. It works with three floats for the three color-channels, thats it.
All algorithms that work with that are tuned for typical LDR spaces. I think a question that should be asked is how PSNR, SSIM and LPIPS work on HDR content. As soon, as the LDR-almost-black that was 0.01 is positioned in HDR space where it might be 0.4, it needs to cause splats in front of the 0.0 background. (Lesson learned: When moving from LDR to HDR, LDR-black needs to become transparent in terms of absence of data - or single colors from LDR need to define a color-range in HDR. That would be the most accurate way of conversion. Is there any format supporting that? - Two values per pixel defining upper and lower boundary)

[simonbethke]

I think this is the challenge that I am actually dealing with: https://claude.ai/share/58aacb9e-b2a9-41cc-961f-9062a53ac70c

Sure there is bracketing. But to me this feels like oversampling 100 times and throw away 90% of the collected data.

[PLAN8VR]

It's difficult for me to add to this as I'm really just guessing how things work, but for me, I have tested various image colour spaces and all will create splats that "look" like the images. So if I supply an hdr image that has been tone mapped, it will result in a splat that looks tone mapped, if I supply an image that is not in srgb, like rec2020, the image preview is incorrect (mostly dark and incorrect colours), and the resultant splat will look just like the incorrectly displayed images.

So, it seems quite likely to me that even if brush uses whatever colour spaces you throw at it, is only understands those colours in the context of the incorrect colour space adaptation. So the resultant splat from an incorrectly transformed colour space looks identical to the incorrectly transformed image.

Different colour space datasets (using exactly the same HDR exr images) will result in different splats. This means that brush does not interpret the colour spaces correctly. The only time I get correct splats is if I supply srgb images. But SRGB isn't wide enough to work with HDR.

[PLAN8VR]

Brush uses image crate according to Arthur. I asked ClaudeAI if this can utilise colour spaces. The response is the following. ..

Now let me check the image crate’s EXR support specifically:​​​​​​​​​​​​​​​​

Perfect! Now I have a clear answer. Let me provide you with a comprehensive response:

No, the image crate does NOT have built-in color space transformation capabilities.

Here’s what I found:

What the image crate provides:

1. Basic color type support - It can handle RGB, RGBA, Gray, etc., but treats them as simple pixel containers
2. EXR support - Via the exr feature flag (using the exr crate dependency), it can read/write EXR files
3. No color space awareness - The crate is explicitly “unopinionated about color management” and assumes all images are in the same color space

What you’ll need to add:

Since Brush needs proper color space support, you’ll need to integrate additional crates:

Recommended approach:

1. exr crate (already a dependency of image with the exr feature) - Can read chromaticities metadata from EXR files
2. palette crate - Best option for color space conversions, supports:

• Multiple RGB standards (sRGB, linear sRGB, etc.)
• Custom color spaces with different primaries
• XYZ intermediate color space for transformations
• Type-safe conversions

1. Alternative: colorspace crate - More focused on rendering applications, includes:

• ACES, DCI-P3, ProPhoto RGB
• XYZ, Lab* conversions
• Designed for pathtracing/rendering pipelines

Implementation strategy:

You’ll need to:

1. Use exr crate directly (not through image) to read chromaticities metadata
2. Use palette or colorspace for the actual color transformations
3. Create a wrapper that combines these to provide color-aware image loading

The guide I created earlier is still valid, but you’ll be using palette or colorspace for the ColorTransform implementation instead of writing matrices manually. Both crates already have ACES2065-1, ProPhoto RGB, and Rec.2020 support built in!

Would you like me to update the implementation guide to use palette or colorspace specifically?​​​​​​​​​​​​​​​​

[PLAN8VR]

Seems like it's just a case of using different crates instead of what seems like a fairly basic image crate. Not being a coder, I cannot comment on how complicated this is, but from a layman's point of view, it sounds relatively easy using off the shelf ready made components.

[PLAN8VR]

I asked ClaudeAI about this. And it has come up with a technical proposal to implement colour transforms into Brush. You can view the summary here https://claude.ai/public/artifacts/9a486b60-ec67-4675-8da8-ca0532205fc0

I'll have a go at cloning the repo tomorrow and see if I can get this working.

[PLAN8VR]

From ClaudeAI re Brushes current image crate handling ...

No, the image crate does NOT have built-in color space transformation capabilities.

Here’s what I found:

What the image crate provides:

1. Basic color type support - It can handle RGB, RGBA, Gray, etc., but treats them as simple pixel containers
2. EXR support - Via the exr feature flag (using the exr crate dependency), it can read/write EXR files
3. No color space awareness - The crate is explicitly “unopinionated about color management” and assumes all images are in the same color space

What you’ll need to add:

Since Brush needs proper color space support, you’ll need to integrate additional crates:

Recommended approach:

1. exr crate (already a dependency of image with the exr feature) - Can read chromaticities metadata from EXR files
2. palette crate - Best option for color space conversions, supports:

• Multiple RGB standards (sRGB, linear sRGB, etc.)
• Custom color spaces with different primaries
• XYZ intermediate color space for transformations
• Type-safe conversions

1. Alternative: colorspace crate - More focused on rendering applications, includes:

• ACES, DCI-P3, ProPhoto RGB
• XYZ, Lab* conversions
• Designed for pathtracing/rendering pipelines

Implementation strategy:

You’ll need to:

1. Use exr crate directly (not through image) to read chromaticities metadata
2. Use palette or colorspace for the actual color transformations
3. Create a wrapper that combines these to provide color-aware image loading

The guide I created earlier is still valid, but you’ll be using palette or colorspace for the ColorTransform implementation instead of writing matrices manually. Both crates already have ACES2065-1, ProPhoto RGB, and Rec.2020 support built in!

[simonbethke]

Why would you want to transform color spaces?

[PLAN8VR]

I think I have demonstrated the answer to that above, but if it's not clear, I'll repeat it here:

Currently, if I supply the same image dataset to brush, but in different colour spaces, the results differ hugely. If Brush doesn't care about colour spaces, then the results wouldn't differ with different colour spaces, but it does. If I supply images in rec2020, the image in preview is dark and underexposed with incorrect colour representation, and similarly the resultant splat is also underexposed with incorrect colour representation.

This simple test demonstrates that colour space does indeed matter for brush, and currently, it interprets all images as if they are plain srgb, which means that the resulting splats will not be correct if the images are in a larger colour space.

Therefore, Brush needs to know what the colour space is, and how to process it correctly in linear colour space. As mentioned before, srgb is not a good fit for hdr, where larger colour spaces like prophoto and aces2065-1 are required.

I think maybe what you are saying is that (A in = A out), regardless of the colour space, which maybe the case, but A colour space is not the same as B colour space and presently, brush cannot read colour space B correctly and outputs garbage as a result. We don't need Brush to export in the supplied colour space, we just need it to be able to read it correctly in the first place.

I may well be completely wrong, but I really don't think so. It's just a case of brush being able to correctly transform larger colour spaces into linear space. Which I don't think it does right now.

My point also, that currently Postshot seems to be the only software that can handle HDR images in EXR format correctly, and the specification for that to work is that the images are in ACES2065-1 colour space, if Postshot specifies that colour space for it to work, it means that it must be using a colour transform in order to process correctly, or otherwise you could supply any old colourspace. But that isn't the case, it needs that space because it understands how to transform it to corrected linear colour.

[mvaligursky]

What is suggested is that the brush does not care about color space. This should mean that if you give it HDR image in any HDR space, you get HDR splats out in the same space.
They might not render correctly as the viewer application does not understand this color space, as all renderers currently expect sRGB color space. The HDR viewer I added also does not understand color spaces, and expect RGB linear color space. But if ply is provided with something else, it would render it, but it would not be rendered correctly.

There are two solutions:

1. the HDR viewers would need to know what color space the ply uses, and convert it to RGB space that the output to the screen needs. This is not ideal, as each viewer / engine would need to handle all possible color spaces.
2. The training application should understand color space, and make sure the output is exported to a known space, perhaps the same Postshot uses, or just a linear HDR space.

But that should also mean, that if images are stored in a linear HDR (and not any extended spaces), that should just work.

[PLAN8VR]

Thank you for being able to explain with knowledge of someone who actually knows what they're talking about!!

[ArthurBrussee]

Reposting from the discord:

I still haven't had the time to properly think about all this, and really need to go deeper on this, but thanks everyone for all the helpful research! All of this requires some experiments my side as well, but here's how I understand things currently:

• Images are read from EXR, using exrs (this is what the image crate uses), in their original color space. That is, we just keep the coordinates as they are.
• These are then displayed in the dataset view. They get intepreted as sRGB naively. No color space conversion, no tonemapping, and are displayed in LDR.
• The splat output tries to match the original color space coordinates. In that sense, the color space "of the splat" is the same as the input. The high dynamic range trains quite different from just [0, 1] colors however, so results are bad.
• This splats gets rendered to a texture. This texture gets interpreted as sRGB naively. No color space conversion, no tonemapping, and are displayed in LDR.

The direction of things should be:

• images are read from EXR in their original color space.
• The dataset view convert their textures to a color space of your monitor. That might be in HDR eg. Rec.2020 on macbooks or in LDR with tonemapping, including exposure controls.
• The scene view does the same for rendered splats.
• The splat training does some automatic tonemapping so that training dynamics are roughly ok. The output is still in the original color space and maybe we should write this into the ply somewhere.

I think to get a handle on things displaying the images correctly tonemapped and maybe in HDR would be the first step. This might require dropping down from the image crate to the exrs crate directly, not entirely sure.

[simonbethke]

Would that change anything other than how the splat looks rasterized in brush?

[mvaligursky]

Looking further into this. ACES2065-1 is a very wide HDR format, that can store large intensities using float16 data., and so great for HDR. But it is not something that most engines use internally, as they use less wide format that is suitable for displays, typically Rec.709. If ply with ACES2065-1 colors is used, the tonemappers engines are using (ACES, Reinhard, filmic ...) do not produce correct colors.

If we do not want to do any conversion, the ply should use linear Rec.709.

We can also consume ACES2065-1 data, and there seems to be a simple 3x3 matrix to convert ACES2065-1 to linear Rec.709. We could apply this either during rendering, or perhaps during load time. Then colors from ACES2065-1 would work perfectly.

Details of the actual needed color conversion:

ACES2065-1 → linear Rec.709 matrix

This 3×3 matrix converts scene-linear ACES2065-1 (AP0, ACES white) into
scene-linear Rec.709/sRGB (D65), including a Bradford chromatic adaptation from the ACES white point (x=0.32168, y=0.33767) to D65 (x=0.3127, y=0.3290).

R709 = 2.5217121*R_ACES - 1.1341427*G_ACES - 0.3875592*B_ACES
G709 = -0.2764744*R_ACES + 1.3726916*G_ACES - 0.0962373*B_ACES
B709 = -0.0153806*R_ACES - 0.1530010*G_ACES + 1.1685496*B_ACES

This mapping preserves neutral greys:
ACES (0.18, 0.18, 0.18) maps to ≈ (0.18, 0.18, 0.18) in Rec.709.

Possible solution

Personally, I like the idea of standardizing on ACES2065-1, as it's better (wider, can store more colors) than Rec.709. So if all training software would output HDR in this color space, and somehow mark ply with this, so that the engine can detect this. The engines can then use the conversion matrix above to bring it to Rec.709, but also use different conversion if they internally support wider space. For example to P3 color space browsers support lately.

I could add support for this conversion to the HDR viewer I added, which would allow us to test dataset in ACES2065-1.

[simonbethke]

The big power, this project has is, that brush can set metadata in ply and superplat can make use of it.
If this stuff is getting momentum this can be the seed for standardization efforts in gaussian splatting in general. I already tried that with the up-vector... This is really useful stuff and required to get gaussian splatting in general into professional requirements.

[PLAN8VR]

I think that Brush should have the view transforms in it as well as just passing through whatever space it is given.

If you think about it in pure workflow terms, brush is an editor of sorts, supersplat is a viewer/editor. Both should be capable of understanding different colour transforms, but it is critical that the creator software should display colours correctly, so that the user knows what the media is looking like before delivery.

So like blender, photoshop, lightroom etc, there should be an intermediate view transform with the ability to export in whichever format chosen.

so, brush should display colours correctly, even if that is a view conversion only.

Blender for example has the ability to choose the correct colour for the input data, the colour space for the editing space and the colour space for the export space. As brush is an editor, it should be able to display a good representation of what will be output.

[PLAN8VR]

So, I did a test with the bathroom dataset in brush. First I worked out that the exr images are in rec.709. The first screen shot is having run the dataset in brush with exr rec709 the second screenshot is after I converted the exr's to srgb. As expected, the srgb version is perfect colour wise (same transform as brush and viewer).

The Rec.709 is underexposed and harsh colours, but should display correctly if the viewer or brush had a rec.709 - srgb transform. (Not suggesting rec.709 should be the preferred colour space)

[PLAN8VR]

Having said that, the srgb version (first in video) shows all over burn out when increasing the exposure, whereas the rec709 version retains a much more defined exposure differential. Both are highlighted with the bloom option which helps show the exposure areas.

So this kind of helps to demonstrate the importance of colour space in HDR. Theoretically, the aces2065-1 space should be amazing at HDR.

2025-12-18.22-47-56.webm

[PLAN8VR]

Could brush just add a display gamma slider? So if user supplies linear rgb (non gamma) a slider for gamma for display srgb could be used to transform for screen space?

[PLAN8VR]

The reason behind my thinking is that viewing linear colour HDR images in Blender, with no gamma correction, shows the image as dark, yet when gamma 2.2 is added, they display correctly, which is what I see in Brush - Images with HDR linear colour appear dark. They also solve dark too.

[praeburn]

Could brush just add a display gamma slider? So if user supplies linear rgb (non gamma) a slider for gamma for display srgb could be used to transform for screen space?

This is all really promising from a rendering point of view. Most rendering happens in a linear colour space and providing assets to a renderer in that space.

Blender already has the ability to use opencolorIO to do colour space transforms like this, just increasing the gamma slider is a little simplistic, when you have opensource libraries you can use that are industry standard and give you input transform and viewerLUT capabilities.