Espruino on P8 smartwatch - ST7789 display driver in Inline C

[espruino-discuss3]

Posted at 2020-05-10 by @fanoush

The P8 smartwatch is good one to hack and put Espruino onto it. It can be updated without taking apart and is quite cheap ans has full touchscreen and 240x240 ST7789 display.

Here is demo of Espruino build with display driver implemented as Inline C code.

https://www.youtube.com/watch?v=PgB1PQA5_OQ

While there is already support for this display for banglejs, that driver is not SPI based. There is another one for previous F5 watch here but that one has framebuffer as static variables. I tried to update it to banglejs driver style but the whole "1. build espruino, 2. flash to watch, 3. see it crashing , 4. repeat" cycle was too slow so I first tried to prototype it in javascript and inline C and see ho far I could go with this.

This uses DMA in similar way as the spilcd driver but was written from scratch without using nordic SDK to learn about ST7789 and nrf52 EasyDMA and SPI hardware (first version just bitbanged SPI RXD and TXD registers without using DMA).

BTW it is interesting how much JS interpreter is slower, try to use g.jsflip and see the difference, the code is otherwise the same. While Espruino performance is otherwise OK here it really struggles.

SDK14 and SDK11 based builds are here with some instructions.

Oh and BTW thanks for cube animation, it was stolen from pixl.js conference badge code

[espruino-discuss3]

Posted at 2020-05-11 by jeffmer

Very impressive indeed. How does the speed of the SPI display compare with the 8-bit parallel Bangle version?

[espruino-discuss3]

Posted at 2020-05-11 by @fanoush

I don't know as I did not see both next to each other, but previously I thought 8Mbps SPI with 240x240 must be pathetic (see this forum post and the video there) but this is not so bad.

There is no DMA for 8-bit parallel mode so the code must do both at once - palette conversion and/or scaling and sending to display. Wth DMA you can compute first block of pixels in advance and then both can run in parallel. With larger blocks than just few bytes the DMA could amost reach maximum theoretical 8Mbps speed. So e.g filling 240x240 in 12 bits (1.5bytes per pixel) takes 96ms when using dma in 24byte blocks.

Even if there is mode when the DMA does not stop and can send several buffers (or same buffer) repeatedly on its own automatically, for some reason letting it loop over 6 byte buffers or 24byte buffers in such mode is a difference. With just the 6 byte buffer in code above it it was still like 125ms ~= same as sending it without DMA, and the 3byte buffer was even slower.

[espruino-discuss3]

Posted at 2020-05-11 by @fanoush

Also this was first meant as quick prototype to try various methods of sending data quickly, I am a bit surprised it can be probably used as standalone driver as is. But of course proper way is to merge it with driver code https://github.com/espruino/Espruino/blob/master/libs/graphics/lcd_st7789_8bit.c and possibly just change macros/methods here to put same data into some buffer and trigger DMA when buffer is filled istead of using 8bit mode.

[espruino-discuss3]

Posted at 2020-05-11 by jeffmer

The rotating cube looks quite fast - fast enough to use as you note. With hardware supported SPI, I have previously managed to get quite good performance for a relatively simple Arduino based ILI9341 driver - see.

[espruino-discuss3]

Posted at 2020-05-11 by @fanoush

ILI9341 driver - see.

Oh, nice, thank you for linking that, the canvas idea with different smaller areas of different bit depth is interesting and could save memory. I think that is what @gfwilliams was adding to Espruino in recent days? Then you don't need to have whole framebuffer (28K in my case for 16 color 4bit one) and could blit such areas directly. E.g. for fonts there is otherwise no other way than to draw each pixel separately which is very slow as seen in the simple driver example here - that one was very painful to see on P8.

Not sure what it will do with memory fragmentation to create such (relatively large) memory blocks dynamically. With one framebuffer you create it once on the beginning. And with around 300 variables free I am experiencing issues to get small flat strings created, this gets triggered sometimes unless I put the code to flash.

The rotating cube looks quite fast

Oh, that one is even fully drawn in javascript running via setInterval() so that the device is still responsive.

[espruino-discuss3]

Posted at 2020-05-11 by jeffmer

Yes, Gordon’s createArrayBuffer and drawImage with palettes are exactly the same mechanism. Actually you can usually get away with one or two buffers and move them around to write at different times. My gpsnav app uses only one buffer which is used every 200ms to draw the compass display and once a second for numerical data display.

Combining javascript and compiled C in your driver is really neat, I was going to ask to see it!

[gfwilliams]

Posted at 2020-05-12 by @gfwilliams

That looks really cool! Actually I believe there is already code to handle ST7789 via SPI (with DMA!) in Espruino, since it was used on the ID205 that I'd been considering for Bangle.js: https://github.com/espruino/Espruino/blob/master/libs/graphics/lcd_spilcd.c

However, that uses an offscreen buffer - which worked on the nRF52840, but is probably not so great on the nRF52832 :)

It'd be really cool to pull something like this into Espruino itself - actually sending individual pixels but via DMA looks really interesting performance-wise.

There are some hacks you can steal from the existing 8 bit driver too - like detecting if you're sending to X,Y then X+1,Y and not sending new XY coordinates each time.

[espruino-discuss3]

Posted at 2020-05-15 by @fanoush

Just a followup. I also got the 16bit mode working and it is visibly slower. 30ms more is really visible difference. Also rewrote the framebuffer/palette lookup reading code so that I can have 3-bit 8 color mode (pixels not aligned to byte boundary). Unfortunately Espruino does not support that, learned after it was already done. After adding bpp=3 to isValidBPP here https://github.com/espruino/Espruino/blob/master/libs/graphics/jswrap_graphics.c#L181 it almost works, just have some wrong thin blank vertical stripes where the byte boundary is crossed. will check how easily it can be fixed.

Also as for colors I tried some dynamic palette for bpp=1 mode so I could change colors between flips - to have multiple colors with only 1 bit framebuffer. And it can work like that for many things without a need for full 2 or 4 bpp. With 1bpp the first cube rotating screen and then second shape fill screen look almost same as before. Only random lines show some nice ZX Spectrum style color artifacts :-)

Current code is still here also briefly described how one could self host the compiler https://github.com/fanoush/ds-d6/wiki/Espruino-Inline-C if you want to modify it.

It is actually pretty nice environment with inline c, one could upload code quickly without even touching the flash memory and try anything freely. With watchdog enabled bugs are harmless. Thinking about exporting unused interrupt vectors from Espruino as this is currently clear limitation of inline C - I cannot hook into SPI interrupt in the driver now. Something like E.set/getInterruptVector(intNo) would do. Any vector unused by current espruino build could be exported like that.

[espruino-discuss3]

Posted at 2020-10-12 by jeffmer

With reference to your recent post - congratulations on getting execution from SPI working.

Does the new build include lcd_spi_unbuf? I would be interested in how it performs in comparison with the driver you describe here and I would like to try it now my P8 has arrived.

[espruino-discuss3]

Posted at 2020-10-12 by @fanoush

Yes, it is enabled, there are also board files used to build it in https://github.com/fanoush/ds-d6/tree/master/espruino/DFU/P8 and there is USE_LCD_SPI_UNBUF=1 inside, for the InlineC driver try upload example from this gist (also linked in readme). It should now work also with build with storage in SPI flash.

I actually didn't try lcd_spi_unbuf with the storage it SPI flash, quite likely it may not work out of box due to shared SPI, so for comparison better take version with storage in internal flash (the one without _SPIFLASH suffix) which is still good enough unless your code is over 120KB.

If the guide in readme is confusing also check https://github.com/enaon/ninebot-one-nRF52/tree/master/p8-nb

[espruino-discuss3]

Posted at 2020-10-14 by jeffmer

Thanks, I managed to get my P8 flashed with Espruino with no problems. I did the following measurements for the Lcd_spi_unbuf driver.

I compared the time taken to fill a 240 x 160 pixel rectangle with the time to draw a 240 x 160 1 bit image. I include the results for Bangle and ESP32 (T-watch for comparison.

RESULTS

Bangle: fillRect 11ms, drawImage 66ms.

ESP32: fillRect 44ms, drawImage 82ms.

P8: fillRect 256ms, drawImage 331ms.

The speed is not great, however, its is worth noting that the issue is not rendering palleted images but it is simply getting pixels sent to the driver. There are at least two improvements I can think of:

1. The buffer at 128 (256 bytes) is nearly exactly the wrong size as the implementation of spiSendMany uses EasyDMA with a maximum transfer size of 255 bytes i.e. the current buffer size causes two transfers, the second of 1 byte. I would like to try a buffer size of 240 (480 bytes).

2. spiSendMany is currently used synchronously in spiSendMany so it would be interesting to try double buffering to speed things up.

I would really like to be able to build the firmware to test this and also to add a Bluetooth hack to support my ANCS widget. I see that you have made the board description public but I would guess that you also need the other files (bootloader etc) to build it?

Thanks agian for making the package available - the P8 has a bright display and its good to experiment with the touchscreen. I found it really easy to transfer the apps I have been running on the T-watch to the P8. Will share when I clean things up.

function time_fill(){
    g.setColor(0x07E0);
    var time= Date.now();
    g.fillRect(0,40,239,199);
    g.flip();
    time = Math.floor(Date.now()-time);
    console.log("Time to Draw Rectangle: "+time+"ms");
}

var pal1color = new Uint16Array([0x0000,0xF100]);
var buf = Graphics.createArrayBuffer(240,160,1,{msb:true});
buf.setColor(1);
buf.fillRect(0,0,239,159);

function time_image(){
    var time= Date.now();
    g.drawImage({width:240,height:160,bpp:1,buffer:buf.buffer, palette:pal1color},0,40);
    g.flip();
    time = Math.floor(Date.now()-time);
    console.log("Time to Draw Image: "+time+"ms");
}

[espruino-discuss3]

Posted at 2020-10-14 by @fanoush

P8: fillRect 256ms, drawImage 331ms.

Did you test also g.flip of my driver? should be below 100ms for full 240x240 screen

I see that you have made the board description public but I would guess that you also need the other files (bootloader etc) to build it?

It is built with nordic SDK11 so you need just that, but I have it slightly patched, you can get
targetlibs_nrf5x_11.tgz from https://github.com/fanoush/ds-d6/tree/master/espruino and extract this in Espruino folder and build with make -j BOARD=P8-SDK11 RELEASE=1 DFU_UPDATE_BUILD=1. No bootloader needed, you already have it.
Only recently there is new i2c slave code that modifies sdk12 so for now easiest for building with SDK11 is to edit makefile and remove this line with nrf_drv_twis.c otherwise it breaks with error that no slave devices are enabled.

SDK11 because there is a bit more flash and variables available and also because arduino environment is SDK11 too so it is easier to switch between them then. However with storage in spi flash there is also good reason to move to SDK12 as other espruino devices to simplify builds so I'll make softdevice+bootlader upgrade package to move it to SDK12 too.

[espruino-discuss3]

Posted at 2020-10-14 by jeffmer

Did you test also g.flip of my driver? should be below 100ms for full 240x240 screen.

I did not get around to testing it as I realised it was faster which makes me believe that it should be possible to speed up lcd_spi_unbuf which has the advantage of flexibility in terms of screen buffering.

Many thanks for the information on building - I will have a go and get back to you. Look forward to SDK12 as I think the Apple ANCS widget requires secure connections.

[espruino-discuss3]

Posted at 2020-10-14 by @fanoush

it was faster which makes me believe that it should be possible to speed up lcd_spi_unbuf

one trick I use is the 12bit color mode which cannot be used for single pixels (2 pixels are stored in 3 bytes). it makes 30ms difference in fullscreen update. The number doesn't look high but if you try both you'll clearly see the difference.

which has the advantage of flexibility in terms of screen buffering

Well, the flexibility could be there too with separate smaller Graphics arraybuffers. If you pre-render stuff into images for speed you are mostly doing same stuff. The native inline code can be simply called with colormap, bitmap rectangle and bpp/stride so can possibly draw also partial areas (that's what it is is already doing - it redraws only modified parts) without any fullscreen buffer and possibly of different bit depths each.

[espruino-discuss3]

Posted at 2020-10-14 by @fanoush

as for lcd_spi_unbuf , it is good when the spi bus is fast enough so that drawing separate pixels is bearable (lines, circles, fonts). that may be true for ESP but IMO not for nrf52. Also the driver is small an yet it has lot of stuff hardcoded - bpp16, ST7789 (the real name should perhaps be lcd_spi_unbuf_st7789_bpp16). It breaks with 12bit mode and for DK08 I even need 6bit (RGB222) mode and 2a/2b/2c commands are different on ST7301 too. If you would remove that too then not sure what would remain - just async generic spi dma writing perhaps.

[espruino-discuss3]

Posted at 2024-02-13 by @enaon

super, thanks a lot, I am still on 6.0.0,because 6.1 was messing up the flash :)

also, do you remember the version the compiler change happened? I just tried 2.17 and your driver works as it was.

edit: I tried the inline driver with 2.21 and the changes you mentioned, but it outputs a "Compacting.." message on the ide when I am uploading, and after a while it disconnects, I think it softbricks, I do a reset using the watchdog to connect again.

Maybe I did something wrong? I just replaced all E.toString with E.toFlatString, and used var bin=E.toFlatString(atob("..­...")) ( I also used E.toString, same result)

[espruino-discuss3]

Posted at 2024-02-14 by @enaon

update: it works :)

I cannot send the file using the ide, it results in a soft brick, but the loader did it, thank you.

[espruino-discuss3]

Posted at 2024-02-14 by @fanoush

but the loader did it

IDE and official loader now does pretokenisation of strings and also expands atob into binary string before upload for 2.21 firmware. which loader worked? maybe your older forked one which does not have this feature yet? or maybe you turned off pretokenisation in loader and have it enabled in ide?
there also was a bug in ide/loader regarding this, so older cached version could break it too.

if you preview the file in storage in IDE it can be seen whether atob was replaced or not.

[espruino-discuss3]

Posted at 2024-02-14 by @enaon

yes I used the older forked loader, I can see the atob was not replaced. I see I have some catching up to do, for now magic3 with 2.21 and 6.1.1 seems to work fine, thanks again.

[espruino-discuss3]

Posted at 2024-02-15 by @fanoush

and 6.11 seems to work fine

You mean S140 6.1.1? and you reduced writing block size to 2048?
If you find any difference/bug with 6.1.1 please report it to issue espruino/Espruino#2465, it looks like it might be good to upgrade to 6.1.x for proper long range/coded phy support.

[espruino-discuss3]

Posted at 2024-02-15 by @enaon

You mean S140 6.1.1? and you reduced writing block size to 2048?

I think so, it is the one you have on your github, and I think I managed to reduce the block size to 2048, I am not very sure I did it right though.

It works, but I am facing some issues when connecting to my unicycle, I think I have seen the same problem in the past, I remeber that I was able to overcome it, but I need to remember how. I have the same problem with the bangleJS2 on 2.21, so it seems that it is not magic3 related, the bangle has very simmilar behaviour ( I am using 7.5ms connection interval) .

The connection takes some more time, and it feels like there are two of them at the same time, one goes through and one fails, causing my script to restart comms, but it maybe be something else, I will play a bit more.

also, I did loose the advertising of the magic(2.21 /6.1.1 -and I tested 6.1.0 too, the same as 6.1.1 as far as I can tell) two or three times, I am not sure what caused it, reset()/E.reboot() did not help, I had to use the watchdog to restart, and then it was visible again.

I have a disable/enable BT switch on the gui, this does not work too, if I disable the BT it will not be visible once enabled, reset()/E.reboot() does not help, only the watchdog works for restoring incomming connections. (this also aplies to the banglejs2, so maybe it is espruino 2.21 related -Edit: disable/enable BT using the bangle os bluetooth connectable menu works, so maybe something has changed and the way I do it is not correct anymore.)

[espruino-discuss3]

Posted at 2024-02-15 by @enaon

sory to spam on this thread, last one :)

This is what I am getting (most of the times, some times it conects) on the bangle2 2.21 when trying to connect to a unicycle using my scripts.

Uncaught Error: Unhandled promise rejection: Disconnected
Uncaught InternalError: BLE task completed that wasn't scheduled (SERVICE/NONE)
Uncaught Error: ERR 0x8 (INVALID_STATE) (:2082)

is bangle using 6.1.x too?

[espruino-discuss3]

Posted at 2024-02-15 by @fanoush

is bangle using 6.1.x too?

I don't think so. there is 6.0.0. when building full hex here https://github.com/espruino/Espruino/blob/master/make/family/NRF52.make#L42
unless you updated it there should be 6.0.0

maybe some delay between connecting and reading data could help, I think there is MTU negotiation running right after device is connected, but it is there too also even for SDK12

[espruino-discuss3]

Posted at 2024-02-15 by @enaon

maybe some delay between connecting and reading data could help, I think there is MTU negotiation running right after device is connected, but it is there too also even for SDK12

thanks, I think I remember talking about this in the past, adding delays did help, it now allways connects, but only if the interval is 7.5-100, 7.5-7.5ms will not work on the bangle js2 for me, nothing like the p8 or the magic with 6.0.0. Strange, I will play more., thank you for your time.

Edit: I just used 6.0.0 in the magic with 2.21, it behaves like the bangle, connects if delays are in place but not in 7.5 ms, comms are slow and it disconnects every 5 secs or so, strange, I think I remember that it was related with the feature of multiple connections that was introduced some time back, but I may have it mixed up in my mind.

[espruino-discuss3]

Posted at 2024-02-16 by @fanoush

OK, I'll dig up my Magic3 and try. BTW they are quite cheap on aliexpress now. Here https://www.aliexpress.com/item/1005005373233740.html it is for US$11.50 for me including VAT and free shipping. I couldn't resist (again) and got some more.

[espruino-discuss3]

Posted at 2024-02-16 by @enaon

I couldn't resist (again) and got some more.

nice, I got two more too, for this price it is super.

this one is ~8 if I clear cookies do not login, yours went to 12euro when I loged in to aliexpress, still very nice. :)

https://www.aliexpress.com/item/1005005935723945.html

[espruino-discuss3]

Posted at 2024-02-17 by @fanoush

this one is ~8 if I clear cookies do not login

you can buy without login? I see the discount only for one piece and I need to login or register to buy

[espruino-discuss3]

Posted at 2024-02-18 by @fanoush

I tried a build of espruino_2v21.9 on p8 and magic, and it seems the inline display driver gets broken.

I have updated the code.js example at https://github.com/fanoush/ds-d6/tree/master/espruino/DFU/Magic3 and made fresh build of 2v21 for MAGIC3. Then I uploaded the example to internal flash and it works :-) I can see the .bootcde has atob() expanded and pretokenized fonts and the Inline C driver binary and it did not break it. Also same source still works with 2v10.

[espruino-discuss3]

Posted at 2024-02-19 by @enaon

thank you fanoush :)

[enaon]

hello @fanoush :)

I have a small issue, I upgraded the firmware on the magic3, and I get this when on 2.25 or 2.27,

compared to this on 2.21

do you have any hints? :)

[fanoush]

Is that always behaving like that and is it always the same or is it random? Two things I guess that may break something - flat strings can now move when garbage collection runs so caching flat string addresses in variables may stop working(?) and there is the createArrayBuffer change in msb/lsb default espruino/EspruinoDocs#750 (comment) but not sure those changes would break stuff like this

[gfwilliams]

there is the createArrayBuffer change in msb/lsb default

This looks possible? That screenshot looks like a byte order issue to me.

It might be the app itself that is using an offscreen buffer? Check all your createArrayBuffer calls (and in the display driver if it's a JS display driver?) and see if they have an msb:... line?

[enaon]

and there is the createArrayBuffer change in msb/lsb default espruino/EspruinoDocs#750 (comment) but not sure those changes would break stuff like this

thank you both, this solved the problem

var g = Graphics.createArrayBuffer(240, 280, bpp,{msb:false});