Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support RGBP encoding #4414

Merged
merged 16 commits into from
Jul 20, 2022
Merged

Support RGBP encoding #4414

merged 16 commits into from
Jul 20, 2022

Conversation

slimbuck
Copy link
Member

@slimbuck slimbuck commented Jul 8, 2022

This PR adds support for a new HDR texture encoding (similar to RGBM).

Instead of storing pixel intensity in alpha, RGBP stores roughly 1.0 - max(1.0, pixel_intensity).

This has some desirable properties, especially when stored as PNG:

  • should solve most banding issues on safari browsers which still premultiply PNG alpha (since only bright areas now lose bits)
  • PNG file size is reduced by up to 20% in typical environments due to simpler alpha data in LDR areas
  • resulting images are easier to manage, since much more colour data is left intact
  • slightly higher quality HDR areas compared to RGBM
  • pixel data is filterable (like RGBM, but unlike RGBE)

For example, given this HDR image:
hdr

RGBM colour and alpha (3.6MB):
rgbm

RGBP colour and alpha (3.1MB):
rgbp

@slimbuck slimbuck added enhancement area: graphics Graphics related issue labels Jul 8, 2022
@slimbuck slimbuck requested a review from a team July 8, 2022 18:04
@slimbuck slimbuck self-assigned this Jul 8, 2022
@slimbuck slimbuck changed the title Add support RGBP encoding Support RGBP encoding Jul 8, 2022
@@ -984,6 +984,13 @@ export const TEXTURETYPE_RGBM = 'rgbm';
*/
export const TEXTURETYPE_RGBE = 'rgbe';

/**
* Texture stores high dynamic range data in RGBP encoding.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if some detail on the format is helpful here. Same goes for RGBM and RGBE I guess.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, will add something to the engine in a future PR.

Copy link
Contributor

@willeastcott willeastcott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is super-cool. Any benchmark stats covering quality and size would be good though before merging, just to give us confidence this is genuinely superior.

@slimbuck
Copy link
Member Author

I created an RGBP version of a particularly problematic lightmap originally taken from https://playcanvas.com/project/345310/overview/orange-room.

The result running on iphone 7:

And comparison of the two lightmaps:

encoding size (gzipped) lightmap
RGBM 5.8MB
RGBP 4MB

@willeastcott
Copy link
Contributor

willeastcott commented Jul 17, 2022

Wow. I'd say that's a massive win! 😮

@@ -10,6 +10,7 @@ import {
import { DebugGraphics } from './debug-graphics.js';

const fixCubemapSeams = true;
const RGBA8_TYPE = TEXTURETYPE_RGBM;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • local variables should not be all capital in RGBA8_TYPE
  • should this default to RGBP now?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah this actually is a global constant .. ignore the first point :)

srgb: 'decodeGamma',
rgbm: 'decodeRGBM',
rgbe: 'decodeRGBE'
'linear': 'decodeLinear',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these tables should use TEXTURETYPE_RGBP and similar constants, instead of stating the strings again?
for example:

TEXTURETYPE_RGBP: 'decodeRGBP'

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this table is actually based on texture encodings not texture types, see https://github.com/playcanvas/engine/blob/main/src/graphics/texture.js#L652

The difference between texture encoding and type is subtle and I don't like it.

Texture encoding is based on texture type and pixel format:

  • texture with type 'default' and RGB8 pixels has encoding 'srgb'
  • texture with type 'default' and RGB16F pixels has encoding 'linear'
  • texture with type 'rgbm/rgbe/rgbp' is always encoding 'rgbm/rgbe/rgbp' no matter the pixel format.

Ideally we'd have just one enum, since it's confusing have two concepts.

(I actually added 'encoding' to texture as a helper, but it's getting used more and more).

@mvaligursky
Copy link
Contributor

This looks really awesome, simple solution that solves the problem, and likely even faster to encode to!

@slimbuck slimbuck merged commit 098e5cb into playcanvas:main Jul 20, 2022
@slimbuck slimbuck deleted the rgbp branch July 20, 2022 11:28
slimbuck added a commit that referenced this pull request Aug 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: graphics Graphics related issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants