Dotclockframebuffer

[jepler]

This requires a change in our fork of esp-idf:

• Allow allocating in PSRAM when program adds a portion of PSRAM, not all esp-idf#12

My test platform is https://www.tindie.com/products/makerfabs/matouch-esp32-s3-parallel-tft-with-touch-7/ [800x480 model]

The display is unstable (horizontal tearing) at higher dot clocks, and on my test screen the low refresh rate of ~12Hz is a bit flickery. We think that this is due to sub-optimal configuration of PSRAM which may be fixed/improved by #6913
(see Lzw655/ESP32_Display_Panel#4 (comment)) -- this kind of problem is apparently prevalent in esp-idf software and some other suggestions are here: https://docs.espressif.com/projects/esp-faq/en/latest/software-framework/peripherals/lcd.html#why-do-i-get-drift-overall-drift-of-the-display-when-driving-an-rgb-lcd-screen

The display will glitch during CIRCUITPY writes or other things that hold the internal flash/spiram bus in use.

May not all be a part of this PR but here are some TODOs:

• pixel start may be nonzero & framebuffer width may not match visible width (e.g., one display is 300 wide but must be initialized with a visible width of 400), so add a rowstride property & make it work
• initialization of some displays uses SPI; sometimes SPI bus is on an I2C I/O expander; can bitbang this in python code to begin with.
• check if psram is really being set up in octal mode [sdkconfig has CONFIG_SPIRAM_MODE_OCT=y]
• see if psram performance can be boosted without IDF change [sdkconfig has CONFIG_SPIRAM_SPEED_80M=y and _120M for octal RAM doesn't seem to be in idf4 and may require core speed to be 240MHz]

@kmatch98 you may be interested
@ladyada it's on

[ladyada]

its on like donkey kong!

[tannewt]

This requires a change in our fork of esp-idf:

Instead of changing the IDF could we enable CONFIG_SPIRAM_USE_CAPS_ALLOC? I'm not sure how many other modules will start trying to use PSRAM then but maybe that's ok. We don't really need it all for CP.

[tannewt]

Just a few minor things. Thanks for getting this going!

[tannewt]

Another thing to check after IDF 5 merge is SPI flash settings since they are on a shared set of pins. The longer the flash reads take, the longer the PSRAM transactions will have to wait. I'm reworking how these settings are done in IDF 5.

[jepler]

Instead of changing the IDF could we enable CONFIG_SPIRAM_USE_CAPS_ALLOC? I'm not sure how many other modules will start trying to use PSRAM then but maybe that's ok. We don't really need it all for CP.

I was reluctant to do this because it looked like a big change (we like to just manage the PSRAM ourselves so we always have a nice big contiguous heap). As far as I understand it, we have to change that if we change this setting. If I'm wrong about that then I'd be happy to change it.

[jepler]

Code for TL021WVC02 480px round display & Adafruit TFT Experiment Rev A https://gist.github.com/jepler/71e9ae8b1e45616fa75988d1139140c1

[RetiredWizard]

I've got @kmatch98's Hack Tablet working with this PR. The bouncing ball block demo seems to work fine but the only other thing I've tested so far is the animated Homer Simpson gif which works but is very small on this display 😁. I'll keep playing and see what I can do, but let me know if there's anything in particular you want tested.

[kmatch98]

I haven't build CircuitPython in a while. Is there a way to collect the .bin artifacts for the Makerfabs 7" build or the HACKtablet?

I actually had to go digging to find mine.

[RetiredWizard]

Is there a way to collect the .bin artifacts for the Makerfabs 7" build or the HACKtablet?

The HACKtablet artifact isn't being built by Jeff's PR yet but I've attached the one I've built. I don't see the Makerfabs artifact, I think you normally find the artifacts by clicking on the "checks" tab on this page and then select "Build CI", the artifacts would be at the very bottom but I only see adafruit_esp32s3_rgb_tft_experiment for some reason.

HACK Tablet firmware.zip

[RetiredWizard]

Keep in mind that the latest build from main isn't compatible with .mpy files so you need to grab .py versions of any libraries you want to use.

Here's the code I'm using to initialize the display.

import displayio
import framebufferio
import dotclockframebuffer
import board

displayio.release_displays()

fb=dotclockframebuffer.DotClockFramebuffer(**board.TFT,**board.TIMINGS800)
display=framebufferio.FramebufferDisplay(fb)

I initially had the CIRCUITPY_RESERVED_PSRAM variable in settings.toml set to 1024000 but the board would crash when trying to re-initalize the display. I increased it to 4098000 which solved the issue but I'm pretty sure 4 Meg is overkill. But with 8 Meg of RAM I figured I could worry about that later 😁

[RetiredWizard]

Oops, sorry, you said .bin artifact...
HACK Tablet firmware.bin.zip

[jepler]

There's no board definition for the hack tablet in this branch and because of the way the CI works now (tries to build as few boards as possible) you have to go through a few hoops to find the makerfabs one.

I think it'll be: makerfabs_tft7

but basically you go back through the green check marks until you find a build that has your board in it, then go to that link, then click summary, and finally you can scroll down and find the artifact. It's .. pretty bad, but that's github's UI in a nutshell.

[kmatch98]

Thanks jepler and RetiredWizard. I got the makerfabs display working with the binary and the awesome ESP webserial uploader.

Agreed on the difficulty to find GitHub artifacts, I clicked and clicked but didn't know the secret pathway to find what I was looking for.

Special thanks RetiredWizard for the setup instructions for the display, these worked perfectly on the makerfabs binary too.

[jepler]

@RetiredWizard I'd be happy to add the hack tablet as a board as part of this PR!

@tannewt I think I made SPIRAM_USE_CAPS_ALLOC work, please review again! However, I didn't change all sdkconfig files to use it, I figured we could do that when we update idf. Let me know if you concur.

[RetiredWizard]

I'd like to make a few more changes to the pins file (for consideration anyway) and then I'll attach the board files to this PR 😁

Thanks!

[RetiredWizard]

Here are the board files I'm using for the Hacktablet...
espressif_esp32s3_devkitc_1_n8r8_hacktablet.zip

[tannewt]

Looks like there are compilation issues still.

[kmatch98]

Jepler, just a few things I remember were issues specific to the S3. Perhaps you have resolved them.

• I got tearing because the ESP-IDF is set to automatically trigger a refresh from some internal timer. I had to turn off the IDF’s auto refresh and created a function where I could allow CP to call for refreshes. This may have been include in v5.0 but I have not verified. That added another issue where typically CP (expecting the display to have its own memory) doesn’t call for a refresh unless something changes. I know foamyguy covered a resolution of this in a stream.

• there are PSRAM access collisions that degrade the refresh rate or downright cause glitches when CP and the LCD DMA both want to access PSRAM bandwidth and the LCD gets starved. The ESP-IDF has a “Bounce buffer” to essentially tie up the processor to avoid this. Maybe there is a better way in CP to avoid and give the LCD preference but I assume there are trade offs.

I don’t think either of these are killers but if not resolved will likely limit the refresh rates. In the end performance will likely be limited to some combination of PSRAM writes by CP and the LCD PSRAM bandwidth required to meet a certain refresh rate.

My apologies if you already addressed these but wanted to leave a paper trail in case someone bumps into a limit while trying to push this driver to its limit. Also I haven’t followed the latest IdF updates so maybe the Espressif folks have found some new tricks.

I’m excited to see what folks will do with all these cool displays!

[jepler]

Yes, almost anytime there's contention for the access to the SPI bus the display will shift and tear, but, it self heals. we're hoping IDF 5 will improve things, and will fine tune then.

I didn't encounter the problem you're mentioning with the lack of automatic refresh. the "relax_on_idle" flag does need to be set to false to get automatic refresh, but that seemed to be all that was necessary.

[jepler]

@tannewt CI is green now. I'd like to leave the refresh argument, because it matches rgbmatrix, will be helpful if/when we add double buffering, and doesn't affect displayio use (displayio calls the equivalent internally).

[tannewt]

Thanks for getting the CI green. Just a couple other minor things.

[RetiredWizard]

When you take a look at the HackTablet board files......

I've reduced and tested the DEFAULT_RESERVED_PSRAM setting as follows:

// a 800x480 16BPP framebuffer + some breathing room
#define DEFAULT_RESERVED_PSRAM      (800 * 800 * 2)

[dhalbert]

This looks OK to me, though of course I don't know some details. But it's consistent and I understand the related changes. Thanks!

done, thanks!

[ladyada]

🥳

[bwhitman]

I saw the note about this work in the circuitpython newsletter. Just wanted to offer any help on performance of this, we've spent a LOT of time / code getting the same thing going in Micropython for TulipCC.

We're using ESP5.1-rc2 and running the SPIRAM at 120MHz. We're getting pretty stable 30FPS on full screen refreshes on a 1024x600 RGB dot clock display. We use the bounce buffers reading from a SPIRAM framebuffer that's bigger than the screen size (for scrolling effects.) Most of the ESP32S3 code is here https://github.com/bwhitman/tulipcc/blob/main/tulip/esp32s3/esp32s3_display.c and the platform-independent display code (writing text, sprites, scrolling etc) is here https://github.com/bwhitman/tulipcc/blob/main/tulip/shared/display.c

I'd be delighted to merge whatever we've done into Circuitpython or whatever makes sense for everyone, I'm not fully up to speed on the differences between Micropython and Circuitpython.

[ladyada]

@bwhitman hihi thanks - we've got the proof-of-concept going, and will probably focus more on automation over the next bit. the text/sprites stuff is already managed by 'displayio' and the REPL-on-display is built in as well as the mass storage filesystem. it may be a fairly easy 'port' if you want to try adapting it to circuitpy

[tannewt]

@bwhitman Thanks! Do you have any mapping between espressif modules and flash and ram settings? We haven't optimized it in CP and (I suspect) have a lot to gain from it. We have 131 boards so it's tricky to know the best settings for each.

[bwhitman]

@tannewt Two places to look: (1) for DDR 120MHz PSRAM you need "octal flash", which are the modules that say OT in this chart
https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/hw-reference/esp32s3/user-guide-devkitc-1.html#ordering-information

(2) You want to see this list of "groups" of settings: https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-guides/flash_psram_config.html#f8r8-hardware

Note that the N8R8 in (1) is OT PSRAM but "only" quad flash -- while this other doc indicates it should support 120MHz DDR RAM, I have not been able to get it going as you need to also select octal flash to indicate 120MHz ram. I think that there is a forthcoming or otherwise new rev of the N8R8 that supports OT PSRAM and flash. The N8R8s I have were purchased from Adafruit earlier this year and do no support OT flash.

So as far as I know, the only modules that will support the fastest RAM settings are ESP32-S3-WROOM-2-N32R8V and ESP32-S3-WROOM-2-N16R8V, the forthcoming or otherwise rev2 of the N8R8, and then all other R8s "only" support 80.

For actual sdkconfig settings, see https://github.com/bwhitman/tulipcc/blob/main/tulip/esp32s3/boards/TULIP4_N32R8/sdkconfig.board for the fast PSRAM speed and https://github.com/bwhitman/tulipcc/blob/main/tulip/esp32s3/boards/TULIP4_N8R8/sdkconfig.board for the "slow" one. We see a real improvement of 20% or so of "GPU" type access times using the faster speed. But it requires ESP-IDF 5.1.