How to use image cache in version 9.1.0?

[zjalen]

I'm using the ESP32S3, but when loading multiple images, the animation becomes very choppy. In version v8.3, I could use image cache img.cache_set_size(10), but this method was removed in version v9. I can't find a way to use it in version v9.1. I tried img.cache_resize(2000), but it's uselsess.

Can anyone guide me? Thanks.

[kdschlosser]

I think you have to edit lib/lv_conf.h and in that file is going to be a macro that you can change.

Are you using the filesystem driver that is in LVGL? or are you loading the image from the file system yourself and keeping the data resident in memory?

[zjalen]

@kdschlosser Could you give me information. I'm trying to load image from sd card.

# v8.3
class Demo:
...
    def draw_img(self):
        try:
            with open('1.png', 'rb') as f:
                png_data = f.read()
        except:
            print("Could not find .png")
        wink_argb = lv.img_dsc_t({
            'data_size': len(png_data),
            'data': png_data
        })
        self.img1 = lv.img(lv.scr_act())
        # self.img1.cache_set_size(10)   # cache line
        self.img1.set_src(wink_argb)

d=Demo()
d.draw_img()
for i in range(5000):
    x,y=random.randint(0,350),random.randint(0,200)
    d.img1.move_to(x,y)
    time.sleep(0.2)

If I use this cache line, the animation will become smooth; otherwise, it will lag.Now in this project,lvgl version is v9.1. I want my animation to become smooth. Which part should I edit in lib/lv_conf.h? I can't find any examples or explanations on the official website.

[kdschlosser]

ok so first thing is this. deep six the use of classes and instance variables. The time it takes to search for the variable and return it to the user is fairly large and that amount of time is proportional to how many variables you have set.

What you have is not really an animation. You are just moving an image around the display into random locations. The PNG is loaded a single time...

an animation is where you have say 10 images and the images change. Just like the frames in a video. That is where "caching" the images would be helpful. They wouldn't be loaded and unloaded from memory which would require decoding the images.

The code you are using above would not work properly if you were changing images because the data for the image is going to get garbage collected.

Here is an example of image caching...

import lvgl as lv
import gc
import time


def load_png(filename):
    with open(filename, 'rb') as f:
        data = f.read()

    image_header = lv.image_header_t()

    lv.image_decoder_get_info(data, image_header)
    draw_image_dsc = lv.draw_image_dsc_t()
    draw_image_dsc.init()
    draw_image_dsc.src = data
    draw_image_dsc.header = image_header
    draw_image_dsc.image_area.x1 = 0
    draw_image_dsc.image_area.y1 = 0
    draw_image_dsc.image_area.x2 = image_header.w
    draw_image_dsc.image_area.y2 = image_header.h
    
    canvas = lv.canvas(None)
    
    size = image_header.w * image_header.h * 4
    
    buf = bytearray(size)
    canvas.set_buffer(buf, image_header.w, image_header.h, lv.COLOR_FORMAT_ARGB8888)
    
    layer = lv.layer_t()
    canvas.init_layer(layer)
    
    lv.draw_image(layer, draw_image_dsc, draw_image_dsc.image_area)
    
    canvas.finish_layer(layer)
    canvas_img = canvas.get_image()
    
    image_dsc = lv.image_dsc_t()
    image_dsc.header = canvas_img.header
    image_dsc.data_size = size
    
    canvas.delete()
    
    return image_dsc, buf
    
    
frame1_img_dsc, frame1_data = load_png('frame1.png')
frame1_mv = memoryview(frame1_data)
frame1_img_dsc.data = frame1_mv
gc.collect()

frame2_img_dsc, frame2_data = load_png('frame2.png')
frame2_mv = memoryview(frame2_data)
frame2_img_dsc.data = frame2_mv
gc.collect()

frame3_img_dsc, frame3_data = load_png('frame3.png')
frame3_mv = memoryview(frame3_data)
frame3_img_dsc.data = frame3_mv
gc.collect()

frame4_img_dsc, frame4_data = load_png('frame4.png')
frame4_mv = memoryview(frame4_data)
frame4_img_dsc.data = frame4_mv
gc.collect()


scrn = lv.screen_active()

img = lv.image(scrn)
img.set_src(frame1_img_dsc)
img.center()

current_frame = frame1_img_dsc
display = scrn.get_display()
while True:
    time.sleep(10)
    if current_frame == frame1_img_dsc:
        current_frame = frame2_img_dsc
    elif current_frame == frame2_img_dsc:
        current_frame = frame3_img_dsc
    elif current_frame == frame3_img_dsc:
        current_frame = frame4_img_dsc
    else:
        current_frame = frame1_img_dsc
        
    img.set_src(current_frame)
    lv.tick_inc(10)
    lv.refr_now(display)

in this example we open the image file and then using the canvas we decode the image and render it to a buffer... Now that portion of the work is not going to have to be done again. The buffer and also an image description get returned from the function. We create a memoryview of the data and that gets set into the image description. This portion of the work is done at the module level because it cannot go out of scope. Once a function closes any references to anything that was inside that function gets marked to be garbage collected. We do not want the data going out of scope so it needs to be returned from the function. We don't want the memoryview object going out of scope either since that is what is the link between the image description and the actual data.

The image data returned form the canvas should be raw RGBA data since the rendering to the canvas was done using a layer.

The work to load, decode and render the image to a buffer has been done ahead of time so the work is not going to have to be done when loading each "frame" into the image widget. all it has to do is copy the buffer data to the frame buffer when the images change. That's it.

That is what image caching is.

[kdschlosser]

That being said, by your example I am going to say your issue is not stemming from caching but instead it is stemming from something else. Without seeing how you are starting up the display I would be throwing darts at a dartboard while being blindfolded and spun around 10 times. The chances of me "guessing" what the problem is and that guess being correct are 1 in a billion.

I need to see all of the code to be able to assist further.

[zjalen]

@kdschlosser Thank you for your help. Here is my demo.I want to display a background image and a foreground object that moves back and forth.

class Demo(object):

    def __init__(self):
        # Display init
        self.rgb_bus = lcd_bus.RGBBus(de=40, vsync=41, hsync=39, pclk=42,
                                      data11=45, data12=48, data13=47, data14=21, data15=14,
                                      data5=5, data6=6, data7=7, data8=15, data9=16, data10=4,
                                      data0=8, data1=3, data2=46, data3=9, data4=1,
                                      freq=13000000, hsync_front_porch=8, disp=-1,
                                      hsync_pulse_width=4, hsync_back_porch=8, hsync_idle_low=True,
                                      vsync_front_porch=8, vsync_pulse_width=4, vsync_back_porch=8,
                                      vsync_idle_low=True, de_idle_high=False, pclk_idle_high=False,
                                      disp_active_low=False, refresh_on_demand=False, pclk_active_low=1)

        _BUF1 = self.rgb_bus.allocate_framebuffer(_WIDTH * _HEIGHT * 2, lcd_bus.MEMORY_SPIRAM)
        _BUF2 = self.rgb_bus.allocate_framebuffer(_WIDTH * _HEIGHT * 2, lcd_bus.MEMORY_SPIRAM)
        self.display = rgb_display.RGBDisplay(data_bus=self.rgb_bus, display_width=_WIDTH, display_height=_HEIGHT,
                                              frame_buffer1=_BUF1, frame_buffer2=_BUF2, backlight_pin=2,
                                              color_space=lv.COLOR_FORMAT.RGB565, rgb565_byte_swap=False)
        self.display.set_power(True)
        self.display.init()
        # self.display.set_rotation(lv.DISPLAY_ROTATION._180)
        self.display.set_backlight(100)
        self.task_handler = task_handler.TaskHandler()
        self.screen = lv.screen_active()

        # GT911 touch init
        self.init_touch()

        # SD Card init
        self.SD_CS = 10
        self.SD_FREQ = 1000000
        self.spi_bus = SPI.Bus(host=2, sck=12, mosi=11, miso=13)
        self._sd_ready = False
        self._sd_mounted = False
        self.mount_sd()

        # file driver. "S:/sd" is the path of sd card
        fd = lv.fs_drv_t()
        fs_driver.fs_register(fd, 'S')

...

    def load_background(self):
        self.bg_img = lv.image(self.screen)
        self.bg_img.set_src('S:m1.jpg')
        self.bg_img.align(lv.ALIGN.LEFT_MID, 0, 0)

    def load_foreground(self):
        try:
            with open('obj.png', 'rb') as f:
                png_data = f.read()
        except:
            print("Could not find obj.png")
            sys.exit()

        wink_argb = lv.image_dsc_t({
            'data_size': len(png_data),
            'data': png_data
        })

        self.obj_img = lv.image(self.screen)
        self.obj_img.set_src(wink_argb)
        self.obj_img.align(lv.ALIGN.RIGHT_MID, -350, 0)

        # animation
        a1 = lv.anim_t()
        a1.init()
        a1.set_var(self.obj_img)
        a1.set_values(-500, 0)
        a1.set_time(1000)
        a1.set_playback_delay(100)
        a1.set_playback_time(300)
        a1.set_repeat_delay(500)
        a1.set_repeat_count(lv.ANIM_REPEAT_INFINITE)
        a1.set_path_cb(lv.anim_t.path_linear)
        a1.set_custom_exec_cb(lambda a, val: self.set_x(self.obj_img, val))
        lv.anim_t.start(a1)

d=Demo()
d.load_background()
d.load_foreground()

I just want this animation to become smooth.

[kdschlosser]

OK so you are using an RGB display... what is the resolution of the display? You didn't provide that information.

[zjalen]

OK so you are using an RGB display... what is the resolution of the display? You didn't provide that information.

800*480

[kdschlosser]

Oh Geesh. You are using the wrong kind of display with an ESP32 if you are trying to get smooth animations. You need to be using a display that connects using an I8080 bus instead of the RGB display.

Look at it this way..

800 * 480 * 2 = 768,000

That's how many bytes have to be sent each time an update occurs.

Now lets break that down.

768,000 * 8 = 6,144,000

that's the number of bits that has to be sent. There is a limitation of 65,536 bytes (524,288 bits) for a DMA transfer. so the transfer has to get broken up into different transactions. That adds latency. If we don't even factory that latency in and only use the frequency as the number which is 13mhz that means that 13,000,000 bits per second is able to be transferred. knock 15% of that off for overhead due to GPIO switching and things of that nature. that brings the bits per second down to 11,050,000. You are using 16 lanes so the theoretical is 176,800,000 bits per second. Looks like a big number yes?... well if you are sending 6,144,000 bits how many times is that able to be sent in one second... 28.77 times. Now here is the real kick in the ass. The frame buffers are located in SPI RAM which is slow. IF you are using octal SPIRAM at 80MHZ you will only get 40mhz from it. Remember that speed has to be shared while the transfer is taking place. so 40mhz across 4 lanes. We are going to say you are using quad SPI for the ram... 40,000,000 * 4 = 160,000,000 * 0.85 (15% losses) so now we are down to 136,000,000 bits per second. Now we are down to 22.13 frame per second.. Remember this is not factoring rendering times. The more data on the display that is changing the longer it is going to take to render. If the background image is the entire display size expect it to take at least 20 milliseconds to render. Each time a DMA transaction completes a callback gets called via an ISR and that stops everything from running until the next DMA transaction starts. That's the latency that gets added in.

Those are theoretical numbers and you will not see those kinds of speeds. Not even close to those speeds. when you get down into the teens for frames per second it is not going to be smooth. You are going to end up with choppy animations.

I have been working on a new RGB driver. This driver is going to use a little more memory but it should be able to perform updates a lot faster.

Things you can do to help improve the speeds if you want to stick with the RGB display. Get a board that has an ESP32-S3 that has octal flash and octal ram. I say this because we can overclock the ram to get an effective clock speed of 240mhz instead of the 80mhz. This will help a massive amount in terms of updating the display.

If you can get an I8080 display that is the size and resolution you want that would make for faster updating of the display.

stop using the LVGL build in file system driver and get the images decoded before creating the widgets. Pass the decoded images to the widgets instead. This will eliminate having to decode the image each time it is moved...

[kdschlosser]

Do this as a test...

IDK now much memory you have and if you have enough do the following.

Edit lib/lv_conf.h and change the MICROPY_CACHE_SIZE to 768000
also change LV_IMAGE_HEADER_CACHE_DEF_CNT to 10

[zjalen]

Do this as a test...

IDK now much memory you have and if you have enough do the following.

Edit lib/lv_conf.h and change the MICROPY_CACHE_SIZE to 768000 also change LV_IMAGE_HEADER_CACHE_DEF_CNT to 10

@kdschlosser I am very grateful for your help. I am a beginner and not very familiar with this module. My board has an ESP32-S3-WROOM and an integrated screen.According to the information, the display driver chip is ILI9485. I have tried modifying the lv_conf.h file, and there was some improvement, but not significant. Could you please help me with how to make the most improvements, such as modifying the configuration driver or how to adjust settings for overclocking ram?

[kdschlosser]

You cannot overclock the memory unless you have Octal SPI RAM and Octal SPI Flash.

If you can give me a link to the datasheet for the board you have I can determine what is the best route to go.

[kdschlosser]

are you using the Sunton ESP32-8048S070C board? If you are this board is using quad SPI flash not octal.

I have in development a new RGB Bus driver that will hopefully improve the performance. I have written the code and it compiles but I have not tested it yet to see if it works or how it does performance wise.

My suggestion to you is to use an I8080 display. The problem with that is there are none that have an ESP32-S3 onboard in a 7: display with a resolution of 800x480. All the ones I am able to locate are RGB displays which is what you really need to stay away from.

If you are not opposed to wiring the ESP32 to the display yourself you can do this..

You can buy this display...
https://www.buydisplay.com/7-tft-screen-touch-lcd-display-module-w-ssd1963-controller-board-mcu

and this ESP32 devkit...
https://www.mouser.com/ProductDetail/Espressif-Systems/ESP32-S3-DevKitC-1-N32R8V?qs=Li%252BoUPsLEnvTvWIWLPCZ4g%3D%3D

You will need to get some 2.54mm female jumper wires to make the connections between the display and the ESP32-S3. It's not hard to do. you will need to design a board using a program like KiCad if you want something that you don't have to wire yourself. I can design the board for you but you would need to have the board made. The cost to make the board isn't bad if you do the assembly yourself. the price goes up if you have the board assembled with all of the needed components.

[zjalen]

You cannot overclock the memory unless you have Octal SPI RAM and Octal SPI Flash.

If you can give me a link to the datasheet for the board you have I can determine what is the best route to go.

It's ESP32-8048S043C. Here is a link: https://wiki.makerfabs.com/Sunton_ESP32_S3_4.3_inch_800x400_IPS_with_Touch.html

[zjalen]

I have another board with the display IC HX8369-A. According to the seller, it supports 8080. I tried to use I80Bus and RGBDisplay for display initialization, but it crashed immediately. I'm not sure if it’s a configuration error on my part.The esp module is ESP32-S3-WROOM-1-N16R8.

class Demo:
...
        i80bus = lcd_bus.I80Bus(
            dc=6,
            wr=11,
            cs=9,
            freq=13000000,
            data0=17,
            data1=18,
            data2=19,
            data3=20,
            data4=21,
            data5=22,
            data6=23,
            data7=24,
            data8=25
        )
        _BUF1 = self.rgb_bus.allocate_framebuffer(_WIDTH * _HEIGHT * 2, lcd_bus.MEMORY_SPIRAM)
        _BUF2 = self.rgb_bus.allocate_framebuffer(_WIDTH * _HEIGHT * 2, lcd_bus.MEMORY_SPIRAM)
        self.display = rgb_display.RGBDisplay(data_bus=i80bus, display_width=800, display_height=480,
                                              frame_buffer1=_BUF1, frame_buffer2=_BUF2, backlight_pin=2,
                                              color_space=lv.COLOR_FORMAT.RGB565, rgb565_byte_swap=False)

HX8369-A_DS_preliminary_v01_100713.pdf

[kdschlosser]

I just added the driver for the HX8369