Async Image Loading

[arihant2math]

Hi, I'm writing a chat app client which loads a large amount of images in a dynamic fashion from the internet (I do not know the file locations during compile time). Currently I see no easy way to load these images without blocking the main thread for a while.

Currently I have a Message component that represents a message, and the messages are retrieved via the worker thread, and displayed via a closure in upgrade_in_event_loop.

I tried following this, but this gets really confusing as this hangs/stalls the worker thread until all the images are downloaded and accounted for. Ideally, I would prefer to display the messages first and then load the images dynamically.

Then I tried storing a Rc<VecModel<Message>> of the messages and using a init callback from each message to modify each. This has the downside 1) not working and 2) if it did work it would be slow as it searched through the messages to find the one with the right id. (It fails due to a BorrowMut error when I try to set row data.

Now what I could do is spawn many more threads, each downloading a single image, then looping through the message model to find the right one, and then modifying the appropriate component, but this would involve some messy code and is not a preferable solution., while also blocking the main thread while searching for the right message.

Essentially, what I'd like to do is modify a component in a for loop after it has been created while not blocking the main thread as much as possible, is there any clean/idiomatic way to do this? Alternatively if there is a way to load images from the internet asynchronously, that would work too.

Thanks.

[ogoffart]

I think you have the choice between having a thread that can use upgrade_in_event_loop for each Message and you can user set_row_data on the paricular row to set the image when loaded.

Alternative is a similar pattern, but using async rust (slint::spawn_locale)

[vonnieda]

I recently spent a lot of time getting lazy image loading working well in my app. Here is roughly how I do it.

Here is an example of my "artist list" page, which is a grid view with thousands of images. I added a few comments to note the image handling:

pub fn artist_list(app: &App) {
    let librarian = app.librarian.clone();
    let ui = app.ui.clone();
    let images = app.images.clone();
    // Do the initial database query and sort in a thread to avoid blocking UI.
    std::thread::spawn(move || {
        let mut artists: Vec<Artist> = librarian
            .list(&Artist::default().into(), None)
            .unwrap()
            .map(Into::into)
            .collect();
        artists.sort_by_key(|a| a.name.clone().unwrap_or_default().to_lowercase());

        // Pass the database results to the UI for conversion to models.
        ui.upgrade_in_event_loop(move |ui| {
            // Convert the database results to Cards, which each contain an image
            let cards: Vec<CardAdapter> = artists.iter().cloned().enumerate()
                .map(|(index, artist)| {
                    let mut card: CardAdapter = artist.clone().into();
                    // Get the image, if it is cached, or schedule it for lazy loading.
                    card.image.image = images.lazy_get(artist.model(), 200, 200, move |ui, image| {
                        // This closure is the "magic" part. Once the image is lazy
                        // loaded in the background this closure will be called
                        // WITHIN the UI thread. We use the ui reference to
                        // get the existing card from model by index, set
                        // image and then set the data back on the model.
                        let mut card = ui.get_artist_list().cards.row_data(index).unwrap();
                        card.image.image = image;
                        ui.get_artist_list().cards.set_row_data(index, card);
                    });
                    card
                })
                .collect();
            let adapter = CardGridAdapter {
                cards: ModelRc::from(cards.as_slice()),
            };
            ui.set_artist_list(adapter);
            ui.set_page(Page::ArtistList);
        }).unwrap();
    });
}

And here is the lazy_get function that is called above:

    pub fn lazy_get<F>(&self, model: dimple_core::model::Model, width: u32, height: u32, set_image: F) -> slint::Image
            where F: Fn(AppWindow, Image) + Send + Copy + 'static {
        let entity = model.entity();
        let cache_key = format!("{}:{}:{}:{}", 
            entity.type_name(), entity.key().unwrap(), width, height);
        if let Some(buffer) = self.cache.lock().unwrap().get(&cache_key) {
            return Image::from_rgba8_premultiplied(buffer.clone())
        }
        {
            let images = self.clone();
            let model = model.clone();
            let ui = self.ui.clone();
            self.threadpool.execute(move || {
                if let Some(dyn_image) = images.load_model_image(&model) {
                    let dyn_image = dyn_image.resize(width, height, 
                        image::imageops::FilterType::Nearest);
                    let buffer = dynamic_to_buffer(&dyn_image);
                    images.cache.lock().unwrap().insert(cache_key, buffer.clone());
                    ui.upgrade_in_event_loop(move |ui| {
                        let image = Image::from_rgba8_premultiplied(buffer);
                        set_image(ui, image);
                    }).unwrap();                    
                }
            });
        }
        Image::from_rgba8_premultiplied(self.default_model_image(&model))
    }

So, the way this all works is that when we go to the artist list page, we nearly instantly get the view of thousands of cards, and they will either have placeholder or cached images. Each request to create a card adds a request to the image loader, which goes into a thread pool. When the request is executed on the thread pool it loads the image (the slow part), resizes it (also slow), and then calls the set closure that was specified in the initial call to lazy_get.

Since these set clousures update the model in place, rather than resetting the model, we get a stable view where the images pop in as they are loaded.

The result is that the UI stays snappy and the images load in as quick as I can.

One thing to note: In Debug mode this will still be a little clunky. There is a slowdown that I think is within Slint when setting the image on the model that I have not been able to track down. But, in Release mode this is completely negligible and the result is extremely fast. Even in Debug mode it's usable - the scroll view just skips a little while the images are still loading.

Hope this helps. It was a real struggle to get it performant and I'm very happy with the final result.

Jason