-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Critically missing linearization #25
Comments
I'll add a small note that this is true for colored images like albedo maps and the image above, but it may not be true for normal maps or roughness maps that don't contain perceptual color images (e.g. they're not in gamma space to begin with). |
Indeed. It also only applies to downsampling. Upsampling gamma-corrected data should be just fine.
|
Effectively we need to ask the input "hey, are you linearized" and that is not something the example currently does, but since the On the other hand it is quite curious that only pink lines are selected, that's not so much a missing gamma linearization (which might also be missing, FWIW) but the way we sample the source image, "picking only 1 out of 4 pixels" just like the MS Paint example on that page. But as @KYovchevski pointed out to me that could be a quirk of the Lanczos filter we use, which weights adjacent pixels negatively, then the pixels next to it positively, and so on. Meaning that for a
Turning the The page also mentions that |
You may want to look at
A most likely correct implementation that could be looked at for reference is the one in OpenImageIO which matches the one found in ImageMagick in output (i.e. another OSS impl.). Speaking of which, note that the OIIO source has this comment which hints at you being in good company (Pixar, no less):
|
@virtualritz Thank you for these resources: I'm not well-versed in image conversion algorithms but will give them a good read. As expected, I think I have found the mistake that is happening in the code: we apply a 7x7 kernel to read source pixels and write them back to the target image, but this kernel is applied to the target texture dimensions instead of the source texture dimensions. Meaning that when we scale (normalized, in float...) coordinates from the target to the source dimensions (between which is a 2:1 ratio), we only sample odd or even rows/columns, hence resulting in this color pattern! ispc-downsampler/src/ispc/kernels/lanczos3.ispc Lines 71 to 72 in d674723
Furthermore there are also some strange clamps that will resample boundary pixels more often for edge pixels. |
https://github.com/Traverse-Research/ispc-downsampler/compare/lanczos3-kernel-range Quickly cobbled together the two changes in minimal form to test this out, still a bunch TBD:
|
For the linearization: I already have a local branch that introduces new formats for gamma corrected textures, as well as a new kernel for the linearization which we can invoke depending on the format. |
All texture resizing tools I ever used (and that's a ton) require a boundary mode to be specified during downsampling. Usually for mip-map generation. The modes are usually (specified for u & v separately, if needed).
The reason is that later, when you look up the texture, the texture sampler needs to know what to do and if you have an override that contradicts what was done during mip level generation, you may get artifacts. For downsampling normal images, e.g. photos you definitely want clamp though. |
Hi @virtualritz, thanks for the insights once again and apologies for forgetting about this issue. I've started solving all the other problems found in this crate and have now exposed a https://github.com/Traverse-Research/ispc-downsampler/compare/degamma Unfortunately there's still some color shifting, and I haven't investigated whether that might be caused by our implementation of the |
This already looks leaps and bounds better than before. The only suggestion I have is to look at/compare with the resp. filter code in OIIO. |
I briefly looked at the code and seems there is no linearization happening before downsampling.
u8
/channel color data will always have some gamma for obvious reasons.I.e. the results of the downsampling using this crate are just always wrong (as there are no
u16
/f16
orf32
images supported where there is enough bits to store data linearly in the pixel buffer).See http://www.ericbrasseur.org/gamma.html for a detailed explanation.
This is the test image from above website:
![gamma_dalai_lama_gray](https://private-user-images.githubusercontent.com/387036/243469164-47b6f022-5b7d-45ef-8967-bff7d4410ef3.jpg?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjA2OTU2MzYsIm5iZiI6MTcyMDY5NTMzNiwicGF0aCI6Ii8zODcwMzYvMjQzNDY5MTY0LTQ3YjZmMDIyLTViN2QtNDVlZi04OTY3LWJmZjdkNDQxMGVmMy5qcGc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzExJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcxMVQxMDU1MzZaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0wOTI2YmJlZjdlNTE3N2NmYzVjM2Y1MDY2OWU1NWJlNjM2ZDUzZmQ5ZmU5YzRiYjBlZTgyMjFkNjFlMjgyMDc3JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.FBLROkSYYvHWFJmbBBxw66PML36FZmgKbQZCwMA8wrI)
The expected result:
![gamma_dalai_lama_gray_good](https://private-user-images.githubusercontent.com/387036/243469746-ff5ae266-1057-4d1b-a41b-cff1057c9410.jpg?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjA2OTU2MzYsIm5iZiI6MTcyMDY5NTMzNiwicGF0aCI6Ii8zODcwMzYvMjQzNDY5NzQ2LWZmNWFlMjY2LTEwNTctNGQxYi1hNDFiLWNmZjEwNTdjOTQxMC5qcGc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzExJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcxMVQxMDU1MzZaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0yZDZmYjdhZWUyMGZlMmJhMjAyN2JhZmZjNDgyMWFmNGY1M2Q4YjIxZTc1ODZlODZjZmU3MDNmMWQ1NTA2NmIxJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.7Bx5QX5V2mtM0MjQSlgnnq-S77RFVACrHWlb8ksT9N8)
And what the code in this crate produces instead:
![gamma_dalai_lama_gray_result](https://private-user-images.githubusercontent.com/387036/243469949-60a6de7c-ee98-4522-bbf4-31d4121bae32.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjA2OTU2MzYsIm5iZiI6MTcyMDY5NTMzNiwicGF0aCI6Ii8zODcwMzYvMjQzNDY5OTQ5LTYwYTZkZTdjLWVlOTgtNDUyMi1iYmY0LTMxZDQxMjFiYWUzMi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzExJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcxMVQxMDU1MzZaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT01NDE4MGY5MzhjYmY3NDYyNWRmNjZjZDFlZTRmNTQxMDAxNWUzNjhhODg0MWRkM2ZmZGI1YThkZmY3MGIzMTU3JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.OS5Co32Z0Bft4kIVNXVHbYYcsZ7-YOw2nGgyEMfMvUY)
The text was updated successfully, but these errors were encountered: