Problem with rendering to texture

[lenscas]

I have a weird problem when rendering to a texture. The generated image seems to always be black. However,if I copy the code to to another project it works as expected. The rest of the window gets rendered correctly. So, I'm not even sure if I have other drawing code that causes the problem :(

The code in question

gfx.flush(None)?;
let mut surface = Surface::new(
    gfx,
    Image::from_raw(gfx, None, 512, 512, PixelFormat::RGBA)?,
)?;
gfx.fit_to_surface(&surface)?;
gfx.clear(Color::BLUE);
gfx.fill_rect(&Rectangle::new((0, 0), (10, 10)), Color::GREEN);
gfx.flush(Some(&surface))?;
//gfx.clear(Color::RED);
gfx.fit_to_window(&window);
let image = surface
    .detach()
    .ok_or(quicksilver::QuicksilverError::SurfaceImageError)?;
gfx.draw_image(&image, Rectangle::new((10, 10), (512, 512)));

In case it helps, I commited how far I got, the code can be viewed at :
https://github.com/lenscas/mergui/blob/21ea53807a9097fdd1638ec01c265475a29692ca/src/widgets/input.rs#L140

I use the all example to test it

Sorry in advance of the state of that file. I was busy in redoing how it worked to make use of rendering to texture and I tried many things after that to try and discover why it didn't work.

• Environment: Debian 10.3 (Current stable)
• Rust compiler version: Stable
• Quicksilver verison: Latest alpha (0.4-alpha 3 if I'm not mistaken)

[IgneousRed]

Was working with @lenscas when we came to a concrete example img that does not load.
The image gets loaded but it displays just black.

[lenscas]

We discovered that this image failed to load due to how often we load it.

A bit of context: We decided to use the image crate to generate this image because of the bug in this issue. However, we do this multiple time in each frame as the exact image can change depending on where it was drawn.

This is when we first noticed that this image was being drawn totally black, despite our debugger showing that we have useful data before we threw it inside Image::from_raw. After we noticed this we got the generated image (the one linked above) and simply changed the code to load the file directly. This produced the same behavior with the image being drawn totally black.

We then made it so it only loads the image once and stores it for later use. THIS causes the image to be loaded properly.

In the original issue, I was also rendering to texture every frame.

[ryanisaacg]

I wonder if the problem is something to do with creating the textures, then. I'll try to look into that angle when I get a chance!

[ryanisaacg]

Hey, I know it's been a while since this issue. Can you see if the changes in #641 fix this?

[lenscas]

I updated the commit of mergui I mentioned in the first message and updated it to this branch. Sadly, I either did something wrong when I updated it, or the bug is still present :(

Updated code: https://github.com/lenscas/mergui/blob/check_if_bug_fixed/src/widgets/input.rs#L139

The picture that gets rendered:

Sadly, I am not sure if I have an instance of the project I did with @IgneousRed that contains the bug as that was for a gamejam and we didn't create the best commits....

Anyway, if it helps I can see if I can make a minimal example somewhere this week?

[ryanisaacg]

Yup, I didn't have terribly high hopes that #641 would fix it. A minimal example would help a lot, I think.

[lenscas]

So, I just tried a rather simple case that uses everything that my Mergui example uses and that one doesn't reproduce it.

current code

use quicksilver::{
    geom::{Rectangle, Vector},
    golem::blend::{BlendChannel, BlendFactor, BlendFunction, BlendInput},
    graphics::{blend::BlendMode, Color, Graphics, Image, Surface},
    graphics::{PixelFormat, VectorFont},
    Result, {run, Input, Settings, Window},
};

fn main() {
    run(
        Settings {
            size: Vector::new(800.0, 600.0),
            title: "Image Example",
            resizable: true,
            ..Settings::default()
        },
        app,
    );
}
async fn app(window: Window, mut gfx: Graphics, mut events: Input) -> Result<()> {
    let img = Image::load(&gfx, "./button.png").await?;
    let font = VectorFont::load("./font.ttf").await?;

    gfx.set_blend_mode(Some(BlendMode {
        equation: Default::default(),
        function: BlendFunction::Same {
            source: BlendFactor::Color {
                input: BlendInput::Source,
                channel: BlendChannel::Alpha,
                is_inverse: false,
            },
            destination: BlendFactor::Color {
                input: BlendInput::Source,
                channel: BlendChannel::Alpha,
                is_inverse: true,
            },
        },
        global_color: [0.0; 4],
    }));
    gfx.clear(Color::BLACK);

    gfx.present(&window)?;

    loop {
        while let Some(_) = events.next_event().await {}
        gfx.clear(Color::RED);
        for i in 0..10 {
            let mut font = font.to_renderer(&gfx, (5 * i) as f32)?;
            let location = Rectangle::new(
                Vector::new((10 * i) as f32, (10 * i) as f32),
                Vector::new((10 * i) as f32, (10 * i) as f32),
            );
            gfx.stroke_rect(&location, Color::CYAN);
            gfx.draw_image_tinted(&img, location, Color::CYAN);
            font.draw_wrapping(
                &mut gfx,
                "Just some text",
                Some(60.),
                Color::GREEN,
                location.pos,
            )?;
        }

        gfx.flush_window(&window).expect("could not flush");
        let mut surface = Surface::new(
            &gfx,
            Image::from_raw(&gfx, None, 512, 512, PixelFormat::RGBA)?,
        )?;
        gfx.clear(Color::BLUE);
        gfx.fill_rect(
            &Rectangle::new((0., 0.).into(), (10., 10.).into()),
            Color::GREEN,
        );
        gfx.flush_surface(&surface)?;
        let image = surface
            .detach()
            .ok_or(quicksilver::QuicksilverError::SurfaceImageError)?;
        gfx.draw_image(
            &image,
            Rectangle::new((10., 10.).into(), (512., 512.).into()),
        );

        gfx.present(&window)?;
    }
}

So, I guess that this means that it is triggered by a specific thing that both Mergui does and the gamejam game did that doesn't boil down to "use the gfx a good bit before rendering to texture" which would have been my guess before this. I'll see if I can figure out how to trigger it in a minimal example tomorrow

[lenscas]

Ok, I managed to reproduce it with a VERY small minimal example, and I fear I have bad news. The problem seems to happen when the creation and drawing of the surface happens in its own scape, separately from presenting it.

This one is broken

use quicksilver::{
    geom::{Rectangle, Vector},
    graphics::{Color, Graphics, Image, Surface},
    graphics::{PixelFormat},
    Result, {run, Input, Settings, Window},
};

fn main() {
    run(
        Settings {
            size: Vector::new(800.0, 600.0),
            title: "Image Example",
            resizable: true,
            ..Settings::default()
        },
        app,
    );
}

async fn app(window: Window, mut gfx: Graphics, mut events: Input) -> Result<()> {
    loop {
        while let Some(_) = events.next_event().await {}
        gfx.clear(Color::RED);
       //note how I create a new scope here        
      {
            gfx.flush_window(&window).expect("could not flush");
            let mut surface = Surface::new(
                &gfx,
                Image::from_raw(&gfx, None, 512, 512, PixelFormat::RGBA)?,
            )?;
            gfx.clear(Color::BLUE);
            gfx.fill_rect(
                &Rectangle::new((0., 0.).into(), (10., 10.).into()),
                Color::GREEN,
            );
            gfx.flush_surface(&surface)?;
            let image = surface
                .detach()
                .ok_or(quicksilver::QuicksilverError::SurfaceImageError)?;
            gfx.draw_image(
                &image,
                Rectangle::new((10., 10.).into(), (512., 512.).into()),
            );
        }
       //and close it before calling present
        gfx.present(&window)?;
    }
}

While this one works

use quicksilver::{
    geom::{Rectangle, Vector},
    graphics::{Color, Graphics, Image, Surface},
    graphics::{PixelFormat},
    Result, {run, Input, Settings, Window},
};

fn main() {
    run(
        Settings {
            size: Vector::new(800.0, 600.0),
            title: "Image Example",
            resizable: true,
            ..Settings::default()
        },
        app,
    );
}

async fn app(window: Window, mut gfx: Graphics, mut events: Input) -> Result<()> {
    loop {
        while let Some(_) = events.next_event().await {}
        gfx.clear(Color::RED);
       //note how I create a new scope here        
      {
            gfx.flush_window(&window).expect("could not flush");
            let mut surface = Surface::new(
                &gfx,
                Image::from_raw(&gfx, None, 512, 512, PixelFormat::RGBA)?,
            )?;
            gfx.clear(Color::BLUE);
            gfx.fill_rect(
                &Rectangle::new((0., 0.).into(), (10., 10.).into()),
                Color::GREEN,
            );
            gfx.flush_surface(&surface)?;
            let image = surface
                .detach()
                .ok_or(quicksilver::QuicksilverError::SurfaceImageError)?;
            gfx.draw_image(
                &image,
                Rectangle::new((10., 10.).into(), (512., 512.).into()),
            );
           //now, I call present before closing the scope
           gfx.present(&window)?;
        }
    }
}

I also tried to have the surface generation and drawing in a function with inline="always" together with a release build, but that also seems to cause the wrong behaviour.

I made this example on macos Mojave version 10.14.6
I targeted master when I made it. From Cargo.Lock:
name = "quicksilver"
version = "0.4.0-alpha0.5"
source = "git+https://github.com/ryanisaacg/quicksilver#a8caf76823797999f644879845984249b3c59aac"

Edit: Forgot to clarify, both release and debug builds have the problem

[ryanisaacg]

That is an extremely helpful issue reproduction and contrast. Unfortunately I’m being affected by a major power outage! When I have power back I can see if this is caused by the destructor of Surface (which I think is the culprit.)

[lenscas]

I actually have a different theory, which is that the data for an Image doesn't live long enough if you drop it before presenting the new frame.

The reason that I think that that is the case is because this bug also seemed to happen when I used the image crate and also dropped the generated image before presenting the frame, same with loading the image from disk. @IgneousRed and I only got it to work after implementing a cache.

Either way, good luck with the power outage and take your time with this issue. Looks like it is a nasty bug to solve.

[ryanisaacg]

You were entirely right, it was an image-destructor issue all along! That's a soundness bug in Golem that needs fixing, but the Quicksilver side of things is the same either way. Thank you for the report and the reproduction.