correct gamma when scaling textures

[woelper]

Hi!
I've received a bug report about scaling images handling gamma correctly.
There was some interesting reading here: http://www.ericbrasseur.org/gamma.html?i=1
I thought I had correctly handled this with using

gfx.create_texture()
    .from_bytes(self, self.width() as i32, self.height() as i32)
    .with_mipmaps(true)
    .with_format(notan::prelude::TextureFormat::SRgba8)

which works perfectly on linux (meaning I get a non-grey scaled image with the sample image:

. However this looks differently in Mac and Windows.

• Am I doing something stupid and is there a better way?
• May there be a reason why this works on Linux and not on Windows and Mac? I have the feeling that Mipmaps are not generated on Windows.

Thanks so much!

[Nazariglez]

Hey @woelper! Sorry for the delay, I don't have much time on these dates. I'll try to take a look this week or next and come back. Thanks!

[woelper]

No worries. Whenever it fits you best. Thanks and enjoy the holidays if you have any!

[Nazariglez]

Tested in MacOS and Windows, if you use the resize filter it seems to work as expected. I wrote some info on your issue. Although I am keeping the issue open just in case we need to implement something as custom mipmaps.

[woelper]

Thank you so much for looking into this. Are you saying MacOS and Windows now work the same as Linux when scaling a texture?
Just for clarity: The resize filter in Oculante worked correctly as it is implemented in software - zooming in and out of the texture did not work in Mac & Windows (produces grey image), but it did work correctly in Linux (produces smaller image looking like original).

[Nazariglez]

No no, sorry, I explained myself wrong, is what you said, resize filter works fine, zooming in out doesn't. I am looking into this, leave me a few more days to test a few ideas. I tried to do the correction of the gamma on a custom pipeline for the draw2d API but it did not work, so I am going to do a step back, and try to get it working using the graphics API directly. After that, we can decide, if it is worth updating all the draw2d API or creating a custom pipeline for oculante.

[woelper]

Thank you so much! That sounds great!

I am also thinking that at some point in the future - even maybe at the expense of performance - I could load every texture as a floating point image and handle gamma at the end, maybe with a shader. That way, hdr and raw files could be adjusted and edited without loss of precision maybe - what do you think?

[Nazariglez]

I added support for Rgba32Float textures, but I am unsure how to fix the gamma from the shader, sorry :(.

This is what I discovered and what I tried, after test correcting the gamma on the shader with something like:

    #version 450
    precision mediump float;

    layout(location = 0) in vec2 v_texcoord;

    layout(location = 0) out vec4 outColor;

    layout(binding = 0) uniform sampler2D u_texture;
    layout(set = 0, binding = 0) uniform Locals {
        mat4 u_projection;
        float gamma;
    };

    void main() {
        vec4 color = texture(u_texture, v_texcoord);
        outColor = vec4(pow(color.rgb, vec3(1.0 / gamma)), color.a);
    }

I saw that this works well with values around 0.45-0.50, but the result is not the same as the article says, it's more like getting something similar to the real image.

Then I saw that using srgba is the way to go, but mipmaps don't work with gamma correction, this is a OpenGL thing, so the fix must be done by correcting the gamma on software and then uploading custom mipmaps. I am working on custom mipmaps on #208 , but I think this will not be a good solution for oculante, because we need to precalculate all the mipmap levels and upload them for each image that is open.

I was playing a little with how MacOS do this with the preview app, and it seems to me that they do not precalculate either, what I think they do, is load the images as SRGBA without mipmaps, then while the user is zooming in or out they do nothing, the image is displayed glitched but when the user "release" the zoom, they "fix" the image and then it's displayed right.

It makes sense to do something like that for oculante? I mean, load the image to the GPU as SRGBA, disable the mipmaps, let the user zoom in or out and then when they release the zooming input calculate by software the gamma correction and display it. This will increase the memory because it requires having the texture twice, one version to keep the original data, and then second to use as preview, each time the user "releases" the zoom you can use the original to calculate the corrected version and then use it as preview. No need to correct or use gamma on the fragment shader.

[woelper]

First of all, thanks so much for looking into this!

I think your idea could work well, I'd love to try it!
Initially I did not use mipmaps at all (I think Notan did not support it at first?) and this also was not that bad. I think having mipmaps looks especially good while zooming too, as they hide some of the noise when scaling an image, but in the end this may be personal taste and needs some testing. Maybe I could even go the simple route and only use "correct" behaviour once mipmaps are disabled and leave it to the user.

As for the shader, I might be completely off, but I was thinking when loading a source image to Rgba32Float with gamma applied, i'd "degamma" it ( 1.0/2.2 ) to linearize after loading, then convert to float, and apply gamma at the end again.

[Nazariglez]

You can load with the crate image rgba32 with

    let img = image::load_from_memory(include_bytes!("assets/dalai.png"))
        .unwrap()
        .to_rgba32f();

And then do the conversion you want and create anew texture on notan with:

    let texture = gfx
        .create_texture()
        .from_bytes(bytemuck::cast_slice(&bytes), ww as _, hh as _)
        .with_premultiplied_alpha()
        .with_mipmaps(true)
        .with_filter(
            notan::graphics::TextureFilter::Linear,
            notan::graphics::TextureFilter::Linear,
        )
        .with_format(TextureFormat::Rgba32Float)
        .build()
        .unwrap();

As for the shader, I might be completely off, but I was thinking when loading a source image to Rgba32Float with gamma applied, i'd "degamma" it ( 1.0/2.2 ) to linearize after loading, then convert to float, and apply gamma at the end again.

I tried to do what you said, but I think I am doing it wrong or I do not understand quite right what I need to do. Anyway, let me know how can I help you!

[github-actions]

This issue is stale because it has been open for 60 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale. Do not hesitate to open a new issue if you need it.