Skip to content

Lookup table palettization#1823

Closed
petchema wants to merge 13 commits into
Interkarma:masterfrom
petchema:lut-palettization
Closed

Lookup table palettization#1823
petchema wants to merge 13 commits into
Interkarma:masterfrom
petchema:lut-palettization

Conversation

@petchema
Copy link
Copy Markdown
Collaborator

@petchema petchema commented May 15, 2020

Very fast palettization (up to 280~300 fps on my hardware) using a Texture3D as lookup table.
I get results pretty close to my previous shader by dropping 1 bit from each color component (128x128x128 lookup table). Texture takes 8MB and requires 2.4s 850ms (on my hardware, again) to initialize.
Other tradeoffs can be made by modifying lutShift in RetroRenderer.cs (that I haven't exposed as a setting... yet?)
I could also lower memory requirements to 1/4th by only storing palette indexes in the lookup table, at the price of an additional lookup in color palette...

Pierre Etchemaite added 8 commits May 15, 2020 00:59
Remove one bit from all color component in input, to reduce LUT size
(128x128x128). That's still a 8MB table.

Improvements:

- Reuse k-d tree work to accelerate initialization
- Only store palette indexes in the table to reduce its size by 4.
However, that means palettization will really be limited to 256 colors.
Also, harden against degenerated trees
@petchema petchema added the wip Work in progress label May 15, 2020
@petchema
Copy link
Copy Markdown
Collaborator Author

petchema commented May 15, 2020

Benefits:

  • no external tools required, the lookup table is generated on the fly from the provided palette;
  • possibility to have more than one palette at once,
  • could potentially be used with higher resolutions

@petchema
Copy link
Copy Markdown
Collaborator Author

initialization time down to 850ms using Color32

@petchema petchema removed the wip Work in progress label May 16, 2020
@petchema
Copy link
Copy Markdown
Collaborator Author

petchema commented May 16, 2020

no postprocessing:
0 10 23 no postprocessing
posterization:
0 10 23 posterization
0.10.23 palettization:
0 10 23 kdtree palettization
lookup table palettization lutShift=0
lut palettization lutshift=0
lookup table palettization lutShift=1
lut palettization lutshift=1
lookup table palettization lutShift=2
lut palettization lutshift=2
lookup table palettization lutShift=3
lut palettization lutshift=3
lookup table palettization lutShift=4
lut palettization lutshift=4
lookup table palettization lutShift=5
lut palettization lutshift=5
lookup table palettization lutShift=6
lut palettization lutshift=6

@petchema petchema force-pushed the lut-palettization branch from 32277bd to 57503b4 Compare May 16, 2020 11:29
I guess I should have done that before.
I hope it doesn't change the result, I wonder if it does any rounding
behind the scene...
@petchema
Copy link
Copy Markdown
Collaborator Author

I'm reverting FilterMode.Point, because it's rounding access and I want "rounding down" (floor) instead.
Why is it so hard to get that kind of information...

FilterMode = Point will do rounding in true color space, so remove
rounding in lookuo table.
@petchema petchema force-pushed the lut-palettization branch from 76cc753 to b897a14 Compare May 16, 2020 21:50
@petchema
Copy link
Copy Markdown
Collaborator Author

petchema commented May 16, 2020

Fix the issue differently; Since FilterMode = Point does rounding in truecolor space, remove the rounding inside the lookup table. The result is not strictly the same, but very similar and the implementation simpler.
lut palettization lutshift=1 pointfilter

@petchema petchema added the enhancement A new feature or an improvement to an existing one. label May 17, 2020
@Interkarma
Copy link
Copy Markdown
Owner

Superseded by #1828. Thank you @petchema! :)

@Interkarma Interkarma closed this May 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement A new feature or an improvement to an existing one.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants