full-screen background image

[espruino-discuss2]

Posted at 2020-01-14 by Andreas_Rozek

Good morning!

This morning I had some spare time with my Bangle.js and I managed to upload and display a bitmap with 240x240 pixels and 16-bit colours.

Upload takes a while, but I wrote a web form, that does all the hard stuff for you. Displaying the whole bitmap takes approx. 2.2 secs on my Bangle.js.

Now I have to find out, how to write only parts of such an image in order to make it fast enough to be used as a clock background...

Stay tuned...

P.S.: yes, sorry, those web pages are still in German - trouble with files simply took far more time than I expected. Sorry - but I haven't forgotten all those enthusiasts without thorough knowledge of the German language...

[gfwilliams]

Posted at 2020-01-14 by @gfwilliams

Nice! if you're after a bit more performance you might find it's better to write separate binary files for each 'slice' of the image. You can then load them directly without having to do atob.

Not sure if you spotted it but there is already a way to get slices of image as well: https://github.com/espruino/EspruinoWebTools/blob/master/examples/imageconverter.html#L80

[espruino-discuss2]

Posted at 2020-01-14 by Andreas_Rozek

Ah,

thanks for the hints! Let me see, what I can do with this info!

[espruino-discuss2]

Posted at 2020-01-14 by JumJum

Wow, great job !!
BTW, in an application I added translation and read text out in some languages.
It would take some time to seperate everything from my application.
If you are interested, please let me know (auch in deutsch ;-)

[espruino-discuss2]

Posted at 2020-01-14 by Andreas_Rozek

@jumjum

thanks for the offer, but my problem is not the translation itself (DeepL already does a pretty good 80% translation), but to isolate text from markup before translation and adding markup properly afterwards).

I'm working on it, but I only have little time...

[espruino-discuss2]

Posted at 2020-01-14 by JumJum

@Andreas_Rozek
I couldn't resist, see your page with translation by Google.
https://www.jumware.com/Testing/BangleTranslate.html
As far as I can see, all formatting is ok, at least with chrome browser.
Lines 516, 522 and 943 to 953, are added to your html.
Translation itself is google like (Schachtelsätze sind furchtbar)
Far away from being perfect, but much better than nothing.

[espruino-discuss2]

Posted at 2020-01-14 by Andreas_Rozek

Very interesting...

although the translation itself is horrible, it may indeed be "better than nothing".

It is impressive to see how far Google gets with keeping the formatting - although I have not looked into the details with all my annotations, embedded code etc. It won't help me anyway as my pages are created automatically using my own kind of CMS and I will have to translate all the building block individually.

Nevertheless, until then people may help themselves using this translator.

Thanks for this hint!

[espruino-discuss2]

Posted at 2020-01-19 by Abhigkar

Honestly, I had this idea before to use a full image as clock face. Since lack of actual Bangle device and a quick learning about low RAM took me down to drop the idea.

After some time this idea again activated in my mind with a bit difference. So this time I split the 240x240 image into 16 slices with dimensions of 60x60 each and converted image to binary using image converter tool.

Now the idea is to save each binary image file to storage, and draw all image once when face is initialized. 16 images will help to identify the segment of display to be updated on each minute and update only those image which is effected by the clock hands to be re-print.

---

Attachments:

• bg - Copy.jpg

[espruino-discuss2]

Posted at 2020-01-19 by Abhigkar

16 images are just a max slices required for an asymmetric watch face, but the actual numbers of images can be reduces by re-using the same image on different screen position.

Like corner images are same but just filliped and rotated. So one saved image can be used to get 4 different ways.

It will save some space still having 16 segments of display..

[espruino-discuss2]

Posted at 2020-01-19 by Abhigkar

Here is an ugly but working example.

Edit-
https://www.espruino.com/ide/emulator.html?codeurl=https://raw.githubusercontent.com/abhigkar/banglejs-pacman-clock/master/image-clock-face.js&upload

[espruino-discuss2]

Posted at 2020-01-19 by Robin

Sun 2020.01.19

Nicely done @abhinav and a code block for others to emulate too! I have a feeling your technique will get used a lot!!

[espruino-discuss2]

Posted at 2020-01-19 by Abhigkar

Thanks @robin, I am still learning the BangleJs ecosystem. :)

[gfwilliams]

Posted at 2020-01-20 by @gfwilliams

Wow, nice! Just looking at that and your suggestion of rotating by 90 degrees each time, I guess there's even the possibility of having a small radial slice and rotating by 30 degrees each time?

In terms of the big image, I added some code last week which should allow one single big image to be written to flash and rendered from there - but I've still got to add some tools to let you actually upload that image (and right now it won't work in the emulator because there is no concept of flash storage).

[espruino-discuss2]

Posted at 2020-01-20 by Abhigkar

Thanks @gfwilliams!!

Yes possibilities are endless, however

rotating by 90 degrees each time, I guess there's even the possibility of having a small radial slice and rotating by 30 degrees each time

I struggled to render the image with rotation if rotation angle is less. It does not render and sometimes result is a distorted image. See attachment.

I added some code last week which should allow one single big image to be written to flash and rendered from there

This is great!! Where I can find this in source code?

Another thing that I really like to have fake storage in emulator. so that read and write can be done via chrome storage that emulate the device storage. Is it possible??

Attachments:

• rotation_issue.png

[espruino-discuss2]

Posted at 2020-01-21 by Robin

Mon 2020.01.20

'I added some code last week . . . .'
'Where I can find this in source code?'

Until Gordon is able to reply with a specific link, unless that reference was specific for the emulator which is in#2 post;

the running list of updates

https://github.com/espruino/Espruino/commits/master

location of all source code files

https://github.com/espruino/Espruino/tree/master/src

It is much easier to navigate the source using VSCode and I had been given a tremendous amount of assistance getting started. Took me a bit to get going as can be seen from this thread, which may be used as an overview, should that choice be desired.

http://forum.espruino.com/conversations/339160/

[gfwilliams]

Posted at 2020-01-21 by @gfwilliams

When you specify a rotation, the image starts getting centered in the middle - so all you need to do is use some of the code used in other watchfaces with Math.sin/cos to change the x and y coordinates. Rotation is in radians not degrees too: https://banglejs.com/reference#l_Graphics_drawImage

Faking flash storage in the emulator is possible, but it's low priority for me right now since there's more work to do to actually make the emulator 'remember'. Right now I'm interested in actually getting the real watch working well ready for next month :)

big image ... Where I can find this in source code?

It should 'just work'. If you write an image into flash memory as an 'image string' (I'm working on some tools to do this nicely in the IDE) then you can just do g.drawImage(require("Storage").read("img")) and it'll do it

[espruino-discuss2]

Posted at 2020-05-25 by FransM

Got redirected here by Abhigkar

I've been pondering at this as well.
With respect of the 2.2 seconds to draw a full hires bitmap:
Where does the time go? Reading the file from flash? Processing it? Writing the data to the LED controller?
I suspect the last.

The solution I was thinking of is to restore only the display used by the hand (or by the graphic representing a digit). The latter is easier as the position of the hand is moving.
Idea is that when the hand moves the pixels of the old hand are restored from the background image, after which the new hand is written.
And to make it even faster only those pixel that are not overwritten by the old hand need to be restored.
I think something like rasterop/bitblt could help here, (or some lowlevel helper functions)
Key to this is easy and efficient access to the bits of the image on the flash.

Of course it is a bit less trivial as I sketched since we have two hands and they can be overlapping, but I hope you get the idea.
And it is probably easier with a digital watch where the digits are actual bitmaps.
Basically it would require drawing a section from a background image.

How does this sound?

[espruino-discuss2]

Posted at 2020-05-25 by @allObjects

Had outlined @FransM 's approach a while ago in a different conversation... even before background image support was made available... worst case - with a long hand it is still redrawing a quarter of the full image with axis-parallel, rectangular support only. Going a step further would require rectangular support of rectangles of any orientation... (Or triangles... that then allow any polygon to be handled).

Partial redrawing (of background image) from the storage is though not sufficient to satisfy all requirements... because there are multiple hands and may be other dials 'in the face, over one another', like day of the month, moon-phase, etc. At one point it's getting to the point that each of the 'moving' / changing areas have to be updated and the updates have to be buffered. Just with three hands this makes easy more than 60 percent... of the full image...

Therefore, it all depends, and a real solution is very dedicated to all what is going on in a watch's face.

[gfwilliams]

Posted at 2020-05-26 by @gfwilliams

In recent builds I have added a function called drawImages which allows you to draw multiple 'layers' at once, at different scales and rotations: https://github.com/espruino/Espruino/blob/feb45fe7378010be74e8952e087d23950dac050f/libs/graphics/jswrap_graphics.c#L2234

It should go a long way towards making this much easier.

Right now, the 'Image Clock' app exists which does a fullscreen background: https://github.com/espruino/BangleApps/blob/master/apps/imgclock/app.js

And that works with a digital clock by just copying the area of the image where the clock is to an image stored in flash (The new drawImages would be a better way to handle that though).

In terms of update speed, it's a mix of things -the default path for rendering has to deal with multiple BPP, screen rotation, etc so has a lot of overhead. New firmwares are a lot faster though as there's a 'fast path' for common operations.

In terms of an analog clock - I'd suggest using drawImages. For example:

var cg = Graphics.createArrayBuffer(16,200,1,{msb:true});
var cgimg = {width:cg.getWidth(),height:cg.getHeight(),bpp:1,transparent:0,buffer:cg.buffer};
cg.fillPoly([8,0, 16,100, 8,110, 0,100]);
g.clear(1);
g.drawImages([
  {image:img,x:0,y:24,scale:4},
  {image:cgimg,x:120,y:120,center:true,rotate:0.1},
  {image:cgimg,x:120,y:120,center:true,scale:0.7,rotate:1.1},
],{})

Potentially you could even use a bounding box to redraw only what had changed... For hour/minute drawImages will be fine, but for second hand you may notice some lag from the redraw if you don't attempt to do something to restrict how much gets drawn.

[espruino-discuss2]

Posted at 2020-05-27 by FransM

@allObjects updating 60% is of course not desirable.

@gfwilliams
The idea I was thinking of is a bit different.
What I would like to do is to use something like drawImage but instead of drawing the actual image I would like to select the pixels from another image, while using the transparency and position information of the actual image.

To phrase it differently.
I would like to erase the clock hand by redrawing those pixels from the background image that were written when the clock was drawn.

Or, if I simplify the drawing of the clock hand to:
for all clock hand pixels that are not transparent write the pixel to LCD
Then the restore could be
for all clock hand pixels that are not transparent write the pixel from the background image to LCD.

Only pixels that are overwritten by the hand need to be restored.
So basically instead of writing a clock hand pixel we write a background image pixel to restore the background.
And after restoring the background of course a new clockhand can be drawn.

Does this sound feasible?

edit: a possible call could be something like
g.restoreImage({image:fgimg,x:120,y:120,center:true,scale:0.7,rotate:1.1, pixelimg: bgimg})

[espruino-discuss2]

Posted at 2020-05-28 by @allObjects

To phrase it differently. I would like to erase the clock hand by redrawing those pixels from the background image that were written when the clock was drawn.

@FransM, great idea... almost like some IO (SPI?) protocol works: send bytes for no reason just to get the clock going and getting something back.

Instead of drawing the pixels you say to draw, you pick the pixls from an image / buffer and put it to display.

Redrawing the same with background or previous color works quite well. I used in http://www.espruino.com/ui | Modular and extensible UI framework and ui elements. - have to update that - and also when I tried to get a pac man game going - watch .mp4 clip...

[gfwilliams]

Posted at 2020-05-28 by @gfwilliams

Only pixels that are overwritten by the hand need to be restored.

Right now, I don't think that's possible to do - but the extra computation is likely to outweigh the savings of not writing to the screen. It's not that slow drawing to the screen - it's the work that has to be done around that (eg loading the image from flash) that's the issue.

If you want to do that your best bet is to just work out the area that was changed previously and use drawImages with an area specified to update just that area. Even in the wort case (45 degrees) you're still updating basically 1/4 of the screen area that you would do normally.

[espruino-discuss2]

Posted at 2020-05-31 by FransM

I fear the amount of available memory will become the limiting factor.
drawImages might be useful, but I am worried about the complexity.
as @allObjects pointed out you may want to rewrite multiple hands (at least the minute and hour one).

Also with a minute hand I might need 60 different images (unless there is rotational symmetry in the background in which case multiple hand backgrounds could use the same image to restore.

[espruino-discuss2]

Posted at 2020-05-31 by @allObjects

@FransM,

it all depends... how you go for it. But yes, memory is a factor as well as the processing.

Unfortunately, you can go only once every minute after the plain image in the store and for performance it is best to know the bounding boxes of your hands.

Let's have a walk thru the sencarios:

1. ---event: watch turns on
2. draw image from store
3. draw hour hand
4. remember hour hand bounding box
5. draw minute hand
6. remember minute hand bounding box
7. calculate bounding box of second hand
8. capture image in second hand bounding box from display buffer and save it for restore when second hand has to change
9. draw second hand
10. --- event: second hand changes to non-full-minute value (any from 1 thru 59)
11. restore saved data to restore current image of passed second
12. resume at 7.
13. --- event: second hand changes to full-minute value (0)
14. restore saved data to restore current image of passed second
15. restore minute hand bounding box from storage
16. restore hour hand bounding box from storage
    17 resume at 3.

You can optimize 14, 15 and 16 to draw overlaps of all three bounding boxes only once.... give most relieve for 2:07 and :08, 4:22 and :23, 7:37 and:38 and 10:52 and :53.

[gfwilliams]

Posted at 2020-06-01 by @gfwilliams

Also with a minute hand I might need 60 different images

I'm not sure I understand... The code I'd posted above draws two clock hands over a solid background image. drawImages can handle transparency and rotation so it's pretty easy to just rotate a hand wherever you want it

[espruino-discuss2]

Posted at 2020-06-02 by FransM

Rotate is quite ok for drawing the hand.
What I wanted to say is that you might need 60 images each containing the background for each minute.

I still feel it might be doable to restore background images based on the clock hand, if you have reasonably fast flash access and know where the picture data is (it does help if the data is stored continuously, so not having it scattered over the flash in different sectors; haven't studied the file system to see how the FS works).

How much time do you think is needed to read a byte from flash at a given position?
How much time to write a byte to LCD?
And of course this does require low level access to make it performant.

I'll see if I can come up with some pseudocode or C code to illustrate what I want, but it may be Friday or Saturday before I get to that.

[gfwilliams]

Posted at 2020-06-03 by @gfwilliams

I think you need to post up some code - I'm really not understanding what you're after.

drawImages can happily rotate the clock hands and overlay them on an unrotated clock background while updating only the area you specify, so I'm not entirely sure why you'd need 60 different background images.

Honestly, the answer to all those questions is 'it depends'. The best thing to do is to benchmark.

Flash memory is laid out intentionally such that everything is contiguous, so yes, you can just randomly access pixels if needed.

[espruino-discuss2]

Posted at 2020-06-03 by FransM

Let me try to clarify.
What I want to achieve is a 16 bit full-screen clockface.
My understanding is that if the hand moves I need restore (at minimum) the area that was covered by the hand at the old position that will not be overwritten by drawing the new hand.
So this is not about drawing the hand, it is about restoring the background.

Of course one can rewrite the old background in full: expensive.
Or you can rewrite only sections (as suggested by Abhigkar earlier in this thread) (bounding box based, but you would need images to draw)
Or you can store the background of every hand position an redraw that one (that is the 60 background images I was suggesting before, note that the background might be different when the hand is at a different position (I just want to have an image as watchface
Or you can restore the background based upon the "old hand": that is instead of rendering the hand, restore/re-render the background pixels that were covered when the hand was drawn at the previous position.

Or, rephrasing the problem:
Suppose as watchface I want to have a high-res image of my kid. When the minute hand moves from say 0 to 1 am looking for an efficient way (both CPU and RAM efficient) to restore the area that has been overwritten by the hand while at minute 0 after which I can write the hand for minute 1. To do this as efficient as possible, I would like to redraw only the pixels that are actually overwritten.

Did I now express my problem better?

[gfwilliams]

Posted at 2020-06-04 by @gfwilliams

What I want to achieve is a 16 bit full-screen clockface.

When the minute hand moves from say 0 to 1 am looking for an efficient way (both CPU and RAM efficient) to restore the area that has been overwritten

The code I'd posted a few posts back would do that, and the ability to restrict the bounds you draw to just the area including the old and new hands would get the redrawn area down to something sensible. Obviously it's not as efficient as something purpose built, but I feel it's 'good enough' and it's also really flexible.

But the best way to explain is with an example, so here's an analog clock with an image background, no flicker, and that's fast enough to include a second hand:

espruino/BangleApps@50a2df6

If someone feels like a fun project it'd be nice to use the 'customiser' from the background image clock (https://github.com/espruino/BangleApps/tree/master/apps/imgclock) and add it here, so we could have a single clock with multiple different backgrounds.

I'm definitely up for ways to boost the performance of drawImage and drawImages (without increasing memory usage too much) though.