enk-pi e-paper device

[snowpie]

Hi, I've got one of these: https://shop.sb-components.co.uk/products/enkpi ; it's driven by a raspberry pi pico W, so it's super limited for memory etc . Before I got Home Assistant, I had it rendering weather reports, energy reports from my solar panels etc but the low resources meant everything had to be offloaded to a server and it was all a bit messy. Natively it runs micropython, though I believe you can flash it with other firmware.

I'd love to get it working with Home Assistant and make it a bit more flexible. Would that platform be able to run tesserae? Happy to help with testing it out and a bit of dev if needed, if you can give me a few pointers.

[dmellok]

Short version: yes, but the EnkPi would be a Tesserae client, not the thing running Tesserae. The server runs wherever Home Assistant lives (there's an HA App) or on any Docker host, renders the dashboard, and the Pico W just fetches a ready-made frame and pushes it to the panel. That's the offload-to-a-server model you already had, just with the server side done for you and the HA data coming from widgets rather than your own scripts.

The board itself sounds like a good fit for @bablokb's generic CircuitPython client, https://github.com/bablokb/tesserae-device-circuitpython. It's built for exactly this case: one client, many boards, with the polling, ETag handling, sleep and battery workflows already done. Getting a new board on it is a matter of a configuration (and possibly a small HAL) rather than new firmware, and Bernhard has run it on a Pico W before.

From the sbcshop repos, the 7.5" is 800x480 with separate black and red planes, 48000 bytes each, plus a micro SD slot, RTC, 6 buttons and a battery connector. Which size do you have? The 2.9/4.2/5.83 variants would be smaller still.

Bernhard, two questions I'd rather you answer than me guess at:

• Memory. You hit a MemoryError on the Inky Frame 5.7 (600x448, 7 colours, Pico W). Does a 2- or 3-colour 800x480 panel fit on a Pico W, or does it need the SD caching on your list? This board has the SD slot, so it might be a useful test case for that.
• Driver. Do you know of a displayio driver for the Waveshare 7.5" black/red panel these boards use? sbcshop ships a MicroPython driver, so the init sequence is at least available to port.

Nothing is needed on the server side. The client registers as the generic CircuitPython kind and declares the panel size and gamut itself (bwr_3 for the black/red variants), and the server already renders an indexed BMP in that palette. Protocol docs for reference: https://docs.tesserae.ink/dev/client-protocol/ and https://docs.tesserae.ink/install/rest-transport/.

If you're happy to be the tester, that would be a real help in testing this firmware on microcontroller-class hardware.

[bablokb]

Bernhard, two questions I'd rather you answer than me guess at:

* Memory. You hit a MemoryError on the Inky Frame 5.7 (600x448, 7 colours, Pico W). Does a 2- or 3-colour 800x480 panel fit on a Pico W, or does it need the SD caching on your list? This board has the SD slot, so it might be a useful test case for that.

* Driver. Do you know of a displayio driver for the Waveshare 7.5" black/red panel these boards use? sbcshop ships a MicroPython driver, so the init sequence is at least available to port.

Memory won't be enough for three colors. 2x48000 is 96000, and the Pico-W has about 95K free before loading the image. Two color operation might work. SD-caching is on the list. With at least CP 10.3.0, caching should also be possible directly on the system. Work on this will start soon.

Waveshare is not specific about their driver. There is an UC8179 driver available from Adafruit. They also have an 800x480 tri-color display in their shop, so chances are high that the driver could work. But porting it from MicroPython should not be a big deal.

[snowpie]

Mine is the 7.5" version, 800x480, black/white/red. I just installed Tessarae to my Homeassistant - looks very cool so far, way better than what I had built. Thanks for the tip, I'll take a look at the circuitpython link. My board has micropython - not sure what the differences are in detail and how much work would be needed to port it, but I guess that's a project for the weekend!

[bablokb]

Adafruit has excellent guides on getting started with CircuitPython. From a programming perspective, there is not so much difference, but the workflow is different. So the getting started guide is a good read. For your use-case, the next step would be to give the U8179 driver I mentioned above a try. The driver repo has a ready to run example, this will show if the display is supported out of the box or if there are additional tweaks necessary.

Once this is clear, you have to wait until my firmware supports caching. The idea is to seperate download and presentation: after download the system would restart into a mini-program that just loads the data and updates the display. This won't use anything of the precious RAM for the network stack and therefore there should be enough RAM for the display-update.

[snowpie]

I got Circuit python installed on my enk-pi board and copied over the libraries and example code for UC8179 - it basically just worked - I had to change the pin definitions, that's about all:

spi = busio.SPI(board.GP10, board.GP11) # Uses SCK and MOSI
epd_cs = board.GP9
epd_dc = board.GP8
epd_reset = board.GP12
epd_busy = board.GP13

[bablokb]

That's great! The next step is to head over to my repository at https://github.com/bablokb/tesserae-device-circuitpython. The Readme tells you how to clone the firmware. At the end of the "Installation" section (the section is actually about the first step: cloning the repo including submodules) you will find a link to a second document that describes the installation on the board itself.

If something is unclear, please give feedback. A good documentation is important and any help to improve it is highly appreciated.

[snowpie]

Well, it's almost working - I set the resolution to 800x240 and it is rendering something that looks like what HA is sending, though there's a lot of noise which suggests the pixel depth isn't set right. I tried mono as well as bwr_3 but there wasn't enough ram for the full resolution for that. The example code did render clearly at smaller resolutions so there's something funky happening there

I think the only part of the docs that jumped out was where the settings file needs to be written to (not /src) and a bit more clarity around how the hw_config stuff works might be useful. I'll have another look tomorrow, need to do life stuff now :-) Once it's working I'll do a diff and let you know what I changed.

[bablokb]

Hm. Adding highlight_color puts the driver into BWR mode, i.e. it expects two bits per pixel, four pixels per byte. So gamut="bwr_3" should be correct for this. Without highlight_color, gamut="mono" is the correct choice. In theory at least.

busy_pin is important, but colstart defaults to zero anyhow. There are a few possible reasons why this does not work as expected. One could be a mismatch of the image-format sent by Tesserae and what CircuitPython expects. Maybe @dmellok could comment on this. I do have a huge 1.5" 152x152 BWR display, I will wire it up and see how that does.

Another problem can be the display init-sequences. The Adafruit UC8179-driver uses a very simple setup that is probably not optimized for what your display needs. This also explains why the demo app has a different type of flickering. The only way to sort this out is to reverse-engineer the demo-app.

[dmellok]

Bernhard, snowpie: had a proper look at the server side.

What the circuitpython_bmp renderer sends is a plain uncompressed indexed BMP, 1 bpp for mono and 4 bpp for bwr_3, bottom-up, with a normal BGRA colour table where red is (255, 0, 0). The 2-bits-per-pixel packing only exists in the .bin renderers for the ESP32 boards and never touches this path. On the CircuitPython side the bit packing shouldn't matter to the panel anyway: adafruit_imageload turns the file into a Bitmap plus Palette, and EPaperDisplay with highlight_color sorts pixels into the black and red planes by colour when it refreshes. So I'd read the noise in the first photo as the missing busy_pin, which fits snowpie getting a clean mono frame once it was added.

There was a real mismatch hiding behind it, though. The packer was compacting the colour table to the colours a page actually used. A bwr_3 page with no red on it went out as a 2-colour 1 bpp file, and the next page with red in it went out as 4 bpp with a 16-entry table. Bernhard, your client sizes the displayio.Bitmap from the first frame's header and reuses it, so a depth change between frames would corrupt the second one. Fixed in 0.395.0: the table is now the gamut's full palette in a fixed order on every frame (bwr_3 is always black, white, red, 4 bpp, biClrUsed = 3). Because biClrUsed is now 3 rather than 16, imageload should also allocate the bitmap at 3 values, i.e. 2 bits per pixel, so the 600x240 frame drops from 72 KB to 36 KB of bitmap. Worth a retest of bwr_3 on 0.395.0 with the busy_pin in place.

On "more red than expected": with Floyd-Steinberg on, anything coloured in the dashboard dithers into red and black speckle. If the page is mostly text, switching the device's dither setting to "None" in Settings → Devices gives flat nearest-colour output.

[bablokb]

Sounds worth a try. I will do that late afternoon (CEST) with my BWR display. Although there are no suitable widgets for that size, the calibration image should be sufficient to do some tests.

@snowpie: any objections that I put your hw_config into the contrib directory? Of course I will add the busy-pin...

[snowpie]

Late last night I had it working in 3 colour mode; I'm not at home right now but I'll check tonight when I get back. No objection to using my he config but I'd rather check it first. I'd like to check your memory fix as well once it's ready

[bablokb]

I managed to make my display work in 3 color mode, but I had to tweak some settings. I added the hw_config to contrib/hw_config_il0373.py.

[snowpie]

I'd consider the driver side fixed now for the enk-pi 7.5" display, albeit at reduced resolution because of the limited memory available. In Tesserae in Home Assistant I made a new theme based on the 1 bit paper theme, to set all the colours to either full red, black or white. There are a few grey shades that it generates but they are rendering quite nicely now. The reduced resolution means the unused parts of the screen don't get refreshed properly - it's probably fine for testing but might be best not to leave the display running 24/7 like it, just in case anyone else is tempted :-)

My hw config file, if you want to share it:
hw_config.py

I also had to set app_config.url to the actual address of my HA server, ie http: //192.168.21.128:8765 , and I also enabled app_config.debug which has been really useful.

Thanks for your help everyone, look forward to seeing this project progressing. If there's anything you want me to test with this display, let me know, happy to help.

[dmellok]

Great news, and thanks for sharing the hw_config. I'll mark the EnkPi as tested on the compatibility page. Full 800x480 should follow once Bernhard's download-then-restart split lands. Appreciate the offer to test more, I'll take you up on it.

[bablokb]

If there's anything you want me to test with this display, let me know, happy to help.

Yep: in debug mode, the program spits out memory information. If you run at full resolution, which figures do you have?

[snowpie]

Is this what you need?
ui_application: creating Bitmap for display with600x240@4
ui_application: free memory before Bitmap allocation: 90048
...
DataProvider: fetching dashboard for HTTP-code 200
DataProvider: free memory before imageload: 50256
DataProvider: free memory after imageload: 47840

[snowpie]

For your SD card code, I did a quick test to make sure I can write to the card; this code works with an existing formatted card and shows which pins my board uses:

import board
import busio
import digitalio
import storage
import adafruit_sdcard

spi = busio.SPI(board.GP18, MOSI=board.GP19, MISO=board.GP16)
cs = digitalio.DigitalInOut(board.GP17)

sdcard = adafruit_sdcard.SDCard(spi, cs)

vfs = storage.VfsFat(sdcard)
storage.mount(vfs, "/sd")

print("Writing to SD card...")
with open("/sd/test.txt", "w") as f:
    f.write("Hello from Raspberry Pi Pico and CircuitPython!\n")

# Read from the file and print content to the serial console
print("Reading from SD card:")
with open("/sd/test.txt", "r") as f:
    print(f.read())

Would your code work for the on-mcu storage as well? There's about 200KB left on mine.

[bablokb]

Is this what you need?
ui_application: creating Bitmap for display with600x240@4
ui_application: free memory before Bitmap allocation: 90048
...
DataProvider: fetching dashboard for HTTP-code 200
DataProvider: free memory before imageload: 50256
DataProvider: free memory after imageload: 47840

Perfect. 600x240/4 is 36K, so this fits well into the figures.

For your SD card code, I did a quick test to make sure I can write to the card; this code works with an existing formatted card and shows which pins my board uses:

The code is fine, but it should use the builtin module sdcardio instead of adafruit_sdcard (saves a library installation and is slightly more performant). The mount will go into an _init() that is added to the hw_config:

def _init(hal):
   ...
...
hw_config.init = _init

This is just a preview, I will add all the details to the docs and the contrib/ examples. Your device for example does not share the SPI-bus for the SD-card and the display, others might do that so it is important to do it right.

Would your code work for the on-mcu storage as well? There's about 200KB left on mine.

Short answer: yes, this is on my list. Long answer: there are some challenges. The internal storage is exposed to the host as the CIRCUITPY-drive and therefore is not writable from within CircuitPython. Starting with CP 10.3.0, it can be re-mounted read-write, but some precautions are necessary to prevent corruption caused by concurrent writes. For systems running on battery or usb-power only, there are no issues. For systems connected to a PC or laptop, there are some things to be aware of. So this needs sufficient documentation to prevent frustration.

[bablokb]

@snowpie: This is now implemented. Pull the newest version (git pull --recurse-submodules) and follow the instructions in docs/configuration.md (I added a section for low-memory systems). Caching to on-mcu storage should also work: your dashboards have about 800x480/4 = 96KB, so this will fit into the remaining space on flash.

I have updated the file contrib/hw_config_uc8719_tricolor.py: untested, because I don't have the device, but it should fit your needs.

If you run into problems, just reach out.

[snowpie]

Hi, thanks for the update. I've made the suggested changes, and it appears to be mounting the SD card. However, it still runs out of ram at the native resolution - quite early on in the boot process.

Also, I think there's a problem with the etag - it's now always telling me there is nothing to update, and the screen never refreshes. I've removed and re-added it to the tesserae plugin in Home assistant:

Console log with 480x240:
con1.txt

Console log with 800x480:
con2.txt

[bablokb]

con1.txt is with the old version?! It is definitly not the newest. And con2.txt does not use app_config.dl_mode="FSCACHE". It tries to allocate the bitmap directly after program start and fails because of memory. With app_config.dl_mode="FSCACHE", no bitmap will be allocated in the first pass, the dashboard image will go directly to where you set app_config.dl_dir=.

Regarding the etag: this is normal: if there was a successful update, e-ink displays don't upate as long as the server sends a HTTP304. To fix this, make sure that the server updates the dashboard. There is one caveat currently: if the e-ink update fails, the etag is not deleted. But judging from your logs, you did not reach this stage. con1 is a "normal" 304 behavior, con2 fails so early it did not contact the server at all.

[snowpie]

It's confusing - I've actually deleted the old dashboard and linked a new one to the display device, but it's still telling me there's no display update required; previously it would always refresh on first power-up even if later on the update was skipped, and I could always send a new image manually to it via Send / Upload a file. The link in the log file here:

DataProvider: api.frame(): HTTP-code: 304
DataProvider: {
DataProvider:   url: http://192.168.21.128:8765/renders/68d74c6e9d41dde4.bmp?sig=Ii9yZW5kZXJzLzY4ZDc0YzZlOWQ0MWRkZTQuYm1wIg.aqgwNg.tTdFOiGqhtv1Bizv1m2AA7KqcNk
DataProvider: }

is valid and is the one I would expect to see, but it's not being rendered onto the screen. I can confirm it is using the new mode:

ui_application: using dl_mode: FSCACHE

I have been up since 5am so it's possible I'm doing something wrong; I might wipe the device and start from scratch tomorrow and see if that solves it.

[bablokb]

Yes, the client explicitly skips rendering when it receives a 304 from the server. So this part works as designed. The question is why does the server send a 304 if you think you pushed a new image?!

After printing "ui_application: using dl_mode: FSCACHE" the next line should print the value of dl_dir. But that is not in the logs. In addition, both of your logs have

ui_application: creating Bitmap for display with480x240@4  (from con1.txt)
ui_application: creating Bitmap for display with800x480@4 (from con2.txt)

Note the typo: there is a missing space after "width". This was fixed four days ago. So your device still has old code and starting from scratch after getting some sleep is certainly a good choice.

[dmellok]

snowpie, I had a look at the server side with your con1.txt. The 304s are the server doing what it was told: the device is presenting the ETag of the render it last downloaded, and a 304 only ever goes out when that ETag equals the current render.

The interesting line is the second poll in con1. It got a 200, downloaded the new frame to /sd/dashboard.bmp, then printed "no display update" and saved the new ETag to NVRAM. So the frame was never painted, but the device now claims to hold it, and every poll after that (and every manual Send that renders the same pixels) is a correct 304. That matches Bernhard's read that main.py on the board predates 10 Sep: the version from 8 Sep downloads to the cache but doesn't yet hand off to image_viewer.py. Re-copying src/ to the device should sort it.

If you'd rather not wipe: after the client is current, History → Resend on the device row forces one download even when the ETag matches, so you don't need a new dashboard to get a first paint.

Bernhard, one thing you may already have on the list: run_end() persists the ETag before the viewer runs, so if image_viewer.py fails (e.g. OnDiskBitmap running out of memory) the device ends up in the same 304 lock. Persisting only after a successful paint would cover both cases.