How to write/draw to LCD screen?

[ndward]

Me again, the webmaster of ev3python.com where I try to make it easier for beginners to make a start with ev3dev and ev3 Python.

I haven't been able to write text or graphics to the EV3 LCD screen using EV3 Python except by running the ridiculously complex code on http://www.ev3dev.org/docs/tutorials/using-ev3-lcd/ Thirty lines of code just to draw two lines and a circle?! There has to be a better way. And how do I display test, or an image file. Please help!

[ddemidov]

The Screen class has draw property, which is just an instance of pillow's (python imaging library) ImageDraw.Draw class. See pillow's documentation here: https://pillow.readthedocs.io/en/3.3.x/

All operations are applied to internally help image, which is dumped to the actual LCD when update() method is called. Here is an example that draws an alternating smiley-frawny face on EV3 screen:

from time import sleep
import ev3dev.auto as ev3

screen = ev3.Screen()

smile = True

while True:
    screen.clear()

    # Screen.draw returns a PIL.ImageDraw handle
    screen.draw.ellipse(( 20, 20,  60, 60))
    screen.draw.ellipse((118, 20, 158, 60))

    if smile:
        screen.draw.arc((20, 80, 158, 100), 0, 180)
    else:
        screen.draw.arc((20, 80, 158, 100), 180, 360)

    smile = not smile

    # Update lcd display
    screen.update()

    sleep(1)

[ndward]

Your code gives me the following error:
File "smiley.py", line 4, in
screen = ev3.Screen()
File "/usr/local/lib/python3.4/dist-packages/python_ev3dev-0.6.0-py3.4.egg/ev3dev/core.py", line 2310, in init
File "/usr/lib/python3/dist-packages/PIL/Image.py", line 2030, in new
return Image()._new(core.fill(mode, size, color))
TypeError: integer argument expected, got float

Once I've managed to write or draw to the screen, how do I get Brickman back?

[ddemidov]

I am still using python2 on ev3 (its the default I think). It looks like division on line 2310 should be changed to integer division (//) for this to work in python3. I am afraid I won't be able to make a pull request in a couple of weeks (on vacation, away from computer). For now, you could try to run the script with python2.

[ndward]

No worries, for I'm just leaving also for a two week vacation.

[ddemidov]

Once I've managed to write or draw to the screen, how do I get Brickman back?

When you run your scripts from brickman menu, you should get your own screen. It will switch back to brickman when your script finishes. If you run your script from ssh session, you would get weird results: brickman and your script will both write to the same screen.

[WasabiFan]

@ddemidov I can test that out and open a PR if it fixes the problem.

Also, about the screen thing: I think there's a way to do it via SSH as well but can't remember the command. @dlech will have do advise on that.

[ndward]

I've never yet run a script from Brickman. I have to modify the file permissions before I can do that, don't I? Will Brickman know that it should use Python3?

[WasabiFan]

@dlech will have do advise on that.

@dlech has spoken! To run an application that writes to the display via SSH:

1. Run sudo chvt 6. This just switches the display to show you an empty terminal instead of Brickman, so that Brickman doesn't mess with you.
2. Run sudo conspy. This will make your SSH session show the same terminal that is on the EV3's display. Log in to the prompt it shows you.
3. Run your program. You should see it show up on the EV3 display, as you'd expect.
   • You can work from this terminal like normal, run your program multiple times, etc.
4. Press ESC 3 times to exit the display terminal
5. Run sudo chvt 1 to get Brickman back.

It seems a bit complicated, but it's reasonably easy. If you don't mind a little blinking cursor showing up on top of your program, I believe you can skip steps 2 and 4. This also means that the console isn't small, because instead of using the one on the tiny EV3 display, you are using a normal one and just displaying your code's shapes on the display.

---

As for running it with Brickman, I believe you just need to do two things (once per Python program).

1. Run chmod +x my-file.py, using the real name of your file
2. Add the line #!/usr/bin/env python3 to the top of your file. Make sure it is the very fist line!

[dlech]

You have to press ESC 3 times to exit conspy

[WasabiFan]

Fixed.

[WasabiFan]

I am still using python2 on ev3 (its the default I think). It looks like division on line 2310 should be changed to integer division (//) for this to work in python3.

This change has already been made, however @ndward is using version 0.6.0, which doesn't support Python 3 in full. See #195 (comment).

[ndward]

It's common to run, via SSH, Python scripts that can only be terminated with Ctrl+C. What is the equivalent trick to stop the same scripts if they are launched from within Brickman?

[ddemidov]

You can long press back button to end any program that was started with brickman. Another useful shortcut is back+enter to reset the brick.

[ndward]

I followed the instructions above to make a script executable from Brickman:
Run chmod +x my-file.py, using the real name of your file
Add the line #!/usr/bin/env python3 to the top of your file. Make sure it is the very fist line!

And it worked for this script

#!/usr/bin/env python3
# so that script can be run from Brickman
from ev3dev.auto import *  # needed for Sound
Sound.beep()

but not for this one (I used chmod as suggested)

#!/usr/bin/env python3
# so that script can be run from Brickman
from ev3dev.auto import *
from time import sleep
m = LargeMotor('outB')

m.run_to_rel_pos(position_sp=360, duty_cycle_sp=60, stop_command="hold")
sleep(5)

The same script works fine via SSH.

How come the line
#!/usr/bin/env python3
is needed? Aren't comments ignored?
And is it OK to add anything at the end of that line?

[ddemidov]

When the first line starts with a #! followed by a command, its called a shebang. Bash knows to execute the command specified and to pass the rest of the script as its argument.

but not for this one (I used chmod as suggested)

Make sure the file has unix line-endings. When you forget to convert from Windows format, bash thinks that an extra \r in the end of the first line is part of the command.

[dlech]

If you run the program vis SSH using ./my-file.py instead of python3 my-file.py, it will verify that the shebang magic is working or not.

[ndward]

WasabiFan says to use #!/usr/bin/env python3 but I've also seen #!/usr/bin/python. The difference is due to Python 2 vs Python 3, I suppose?

[ndward]

'Make sure the file has unix line-endings.' This is of course easier said than done! Especially since I'm targeting Python beginners who use Windows. Can I assume that this unix line ending issue is only an issue for shebangs and shouldn't be an issue elsewhere?

Your shebangs are my shenanigans!

[dlech]

It is safe to assume that Windows line endings will cause all sorts of problems on Linux. IMHO, it would be best to make beginners aware of this from the very start and point out how to save files with unix line endings in whatever text editor you are using so that they just get in the habit of always using unix line endings.

You will never be able to launch a program from the brick without a working shebang, so it's best to just get it right from the start.

[WasabiFan]

How come the line ... is needed? Aren't comments ignored?

Have you ever tried to open a file on Windows and gotten the "Open with..." dialog asking you to choose a program? This is the equivalent, but for Linux. Instead of teaching the operating system to assume that .py files are Python like you would in Windows, in Linux you add that "shebang" which tells the operating system how to run your file.

So, if you run your code as python3 file.py, you are telling it in the command that Python 3 should be run. But if you run it as ./file.py, as Brickman does, it needs to look at the shebang to know that your code is Python.

WasabiFan says to use #!/usr/bin/env python3 but I've also seen #!/usr/bin/python . The difference is due to Python 2 vs Python 3, I suppose?

Yes, as I understand it the first will invoke Python 3 while the second will run Python 2.

As for the line endings issue, yes, you will need to use the UNIX-style line endings. Your editor probably has a button for that, so it shouldn't be an issue.

[ndward]

We just had a power outage so my computer restarted and now the shebang seems to work fine even where it did not before. I've been using MobaXterm's internal text editor recently and I notice it is set to Unix format by default so I should never have had any problems with code typed there. Maybe a problem was caused by me pasting some code there that was copied from somewhere else? I'm not expecting this to be an issue, then, for Windows people who use MobaXterm or a Python IDE.

When someone tries to open within Brickman a Python script that is not executable maybe it would be helpful for Brickman to offer to make the file executable instead of just giving an error message?

[dwalton76]

"When someone tries to open within Brickman a Python script that is not executable maybe it would be helpful for Brickman to offer to make the file executable instead of just giving an error message?"

If it ended in ".py" it could also auto-add the shebang if needed.

@ndward I am going to mark this one as closed since this isn't really an issue with the python library and 737 is open to track the Brickman side of things.

[ndward]

This issue is closed but I'd like to reopen it but I don't see how to do that. Or should I start a new issue? For a long time I was unable to print or draw to the EV3 screen – I suppose I was using a too-old (cycle 9) version of the Ev3dev kernel or it had become corrupted or it had not upgraded properly. Since I flashed a recent nightly image I can draw and write to the screen and I've put some working sample code on https://sites.google.com/site/ev3python/learn_ev3_python/screen . I can draw simple shapes and print tiny text using 'text', but I still have some questions:
How to change font, font size?
How to use the align parameter and does it work for 'text' as well as 'multiline_text'?
Where are the fonts kept?
How to display an image file on the screen?
Can the image files of EV3-G be made available to EV3 Python (easily the case with EV3 Basic)?
Sample code demonstrating the above would be great!

[ddemidov]

I don't have easy answers for these, because I don't have much experience with pillow library either. But after reading their reference documentation, I've managed to do this:

http://nbviewer.jupyter.org/gist/anonymous/a268bf8000d7be7f0f49c67c859f1070

Although I am not sure if the fonts will be available on the EV3.

[ndward]

Thanks. I ran this

#!/usr/bin/env python3

**from ev3dev.ev3 import *  # Do I need this?**
from PIL import Image, ImageDraw, ImageFont, ImageOps

# Create new image, same size as EV3 LCD:
im = Image.new("1", (178, 124), "white")
draw = ImageDraw.Draw(im)

# Draw some eyes:
def draw_eye(x, y):
    eye_xrad, eye_yrad = 12, 18
    pup_xrad, pup_yrad =  5,  5

    draw.ellipse((x-eye_xrad, y-eye_yrad, x+eye_xrad, y+eye_yrad), )
    draw.ellipse((x-pup_xrad, y-pup_yrad, x+pup_xrad, y+pup_yrad), fill='black')

draw_eye( 60, 90)
draw_eye(120, 90)

# Write some text:
font = ImageFont.truetype("arial.ttf", 12)
draw.multiline_text((10,10), "Freeze?\nI'm a robot.\nI'm not a refrigerator.", font=font)

# Paste an image:
logo = Image.open('ev3dev_logo.png')
im.paste(ImageOps.invert(logo.convert("L")), (130,10))

# Show the image:
figure(figsize=(8,5))
imshow(np.asarray(im.convert("RGB")));

and got the following error:
File "demidov-graphics.py", line 22, in
font = ImageFont.truetype("arial.ttf", 12)
File "/usr/lib/python3/dist-packages/PIL/ImageFont.py", line 240, in truetype
return FreeTypeFont(font, size, index, encoding)
File "/usr/lib/python3/dist-packages/PIL/ImageFont.py", line 137, in init
self.font = core.getfont(font, size, index, encoding)
OSError: cannot open resource

If I could just get multiline_text going that would be a step forwards..
Is the vertical resolution of the LCD 124 or 128?
By the way, it's my understanding that it's a bad idea to try to use IPython / Jupyter to program the EV3 because of the EV3's very limited RAM. Do you agree?

[ddemidov]

from ev3dev.ev3 import *  # Do I need this?**
from PIL import Image, ImageDraw, ImageFont, ImageOps

You need both of these lines, but you can omit Image from PIL imports. That is already imported by ev3dev.ev3.

im = Image.new("1", (178, 124), "white")

This is not necessary. In the example above I just created an image inside ipython notebook to use it as a test of pillow functionality. When you do this on the EV3 you need to replace the line with

screen = Screen()

and then replace im.draw in the lines below with screen.draw. Also in the 'paste image' example you need to make sure the image (ev3_logo.png or some other) is actually exists in the current directory, and also replace im.paste with screen.image.paste.

font = ImageFont.truetype("arial.ttf", 12)
File "/usr/lib/python3/dist-packages/PIL/ImageFont.py", line 240, in truetype
return FreeTypeFont(font, size, index, encoding)
File "/usr/lib/python3/dist-packages/PIL/ImageFont.py", line 137, in init
self.font = core.getfont(font, size, index, encoding)
OSError: cannot open resource

This is what I was afraid of when I said that font file may be unavailable. You can try to search for any ttf files on ev3 (from bash commandline) with

find /usr/share -iname '*.ttf'

May be @dlech knows if there a way to install some font files onto ev3?

[ddemidov]

Note the negative y coord for the first string and note that if the x coord for the second string is zero the 'w' gets slightly cut off - that shouldn't happen should it?

I guess pillow does some simplistic math to compute where to put the text, and w breaks it, being the widest letter in the alphabet. You can try with a monospace font like courier.

[ddemidov]

I haven't yet succeeded in displaying a bitmap file on the EV3 LCD.

Bitmap the pillow documentation refers to is not a bmp image, its a bitmask (where each pixel is a single bit). To load a bmp, simply use the example I provided, and replace ev3_logo.png with file name of the image you want to load. It should support bmp format.

[ddemidov]

Do you think that means that we are the only two people on the planet who can display those fonts on the EV3?

Nah, I am afraid we've spilled the beans already; its out in the open now. Even if we delete our comments, github has already spammed the 18 people watching the repo with the know-how.

[ddemidov]

It seems draw.bitmap allows to do the same thing image.paste does more easily. The image that is being pasted seems to be converted to required format automatically. This works for me (not tested on ev3 though):

# Paste an image:
logo = Image.open('ev3dev_logo.png')
screen.draw.bitmap((130, 10), logo)

EDIT: here is a more complete example: http://nbviewer.jupyter.org/gist/anonymous/bb174089699ae66443f5a212c5e37b66

[ndward]

Is there an easy way to do screen captures on the EV3?

PS Denis, your English is really good. I see 4 possible explanations:
A) You are using your supercomputer to translate stuff into English
B) You spent time in the US
C) You spent time in the UK
D) You are a genius
Which is correct (multiple answers allowed)?

PPS While working on EV3basic.com I exchanged emails with Андрей Степанов who could be an interesting contact for you (mailsea@yandex.ru). See http://karandashsamodelkin.blogspot.fr/

[ddemidov]

Is there an easy way to do screen captures on the EV3?

Here is an excerpt from irc log I found:

Robotsquare
Can anyone point me to the right guide or search term for taking screenshots of brickman?
dlech
Robotsquare, you can use fbcat or fbdump. One of them saves to a png file, I forget which.
dlech
you can use the --help option to see

I see 4 possible explanations:

Its E) I've spent time in NL :).

[WasabiFan]

@ndward For taking screenshots, I actually have a Python script that I use for taking screenshots for documentation. It runs fbcat as @ddemidov suggested and then recolors it to look like the EV3 screen; if you want to capture a black-and-white screenshot, you can remove the color conversion and loops (everything from line 12 onward).

That script is here.

[ndward]

Denis, dus spreek je ook mischien een beetje Nederlands? Ik ook, omdat ik was in Brussel van 2005 tot 2014.

Brilliant. After an hour of trials I was finally able to display an image. First I put into a folder 'pics' within my 'robot' folder all 107 official Lego EV3 BMPs. These BMPs are monochrome, correctly dimensioned for the EV3 screen. The following script worked fine to display a BMP file 'Bomb.bmp':

#!/usr/bin/env python3
from ev3dev.ev3 import *
from time import sleep
from PIL import Image, ImageDraw, ImageFont, ImageOps

lcd = Screen()

logo = Image.open('pics/Bomb.bmp')
lcd.image.paste((0,0), logo.convert("L")) # works
#lcd.draw.bitmap((0, 0), logo.convert("L"))  # works but inverted
#lcd.draw.bitmap((0, 0), ImageOps.invert(logo.convert("L"))) # works

lcd.update()
sleep(6)

What is the function of convert("L")?

The three lines with comments all display the image but the middle one displays it in inverse colors. Therefore it seems to me the best option is the first. Do you agree?
The BMPs are listed on https://sites.google.com/site/ev3python/learn_ev3_python/screen/bmp-image-collection
and I put a zip file containing all the BMP files at the bottom of the same page (will I go to jail for this?)

(Similarly, I moved the folder containing the fonts into my robot folder so that I can easily find them and back them up next time I need to reflash my card, and to make the path much simpler. There is no reason NOT to move the font folder in this way, is there?)

[ddemidov]

What is the function of convert("L")?

That converts image to a different 'mode'. Modes are described here: http://pillow.readthedocs.io/en/3.3.x/handbook/concepts.html#modes. The image that is stored internally in Screen class uses "1" mode, so I would try that instead of "L".

Therefore it seems to me the best option is the first. Do you agree?

Will lcd.image.paste((0,0), logo) work (no call to convert)? If not, then the first option looks simple enough.

will I go to jail for this?

Not sure. But I guess those images are copyrighted by Lego, so I would take care. Also, I imagine that drawing an image file in paint and then putting it into robot screen would be even more fun for a kid :).

There is no reason NOT to move the font folder in this way, is there?

Move or copy? Are you talking about /usr/share/fonts folder? I would keep the system files (anything not in /home) intact, so copy would be better than move.

[ndward]

Using "1" instead of "L" gives me this error:
File "display-bitmap.py", line 11, in
lcd.draw.bitmap((0, 0), ImageOps.invert(logo.convert("1"))) # works
File "/usr/lib/python3/dist-packages/PIL/ImageOps.py", line 368, in invert
return _lut(image, lut)
File "/usr/lib/python3/dist-packages/PIL/ImageOps.py", line 56, in _lut
raise IOError("not supported for this image mode")
OSError: not supported for this image mode

lcd.image.paste((0,0), logo) work (no call to convert) does not work, gives error:
File "display-bitmap.py", line 12, in
lcd.image.paste((0,0), logo)
File "/usr/lib/python3/dist-packages/PIL/Image.py", line 1332, in paste
self.im.paste(im, box, mask.im)
TypeError: an integer is required (got type tuple)

WasabiFan's screen capture script works well for me. It saves a file 'screenshot.png' to the directory the script is located in. Here is an example:

[WasabiFan]

Yeah, those errors make sense; glad to see it works overall though!

[ndward]

I thought I had the LCD nailed but I have a small problem. This script runs forever and if I start the script from Brickman I can't exit it properly with a long-press on the Back button ( the brick just starts toggling back and forth between Brickman and the graphic display). I have to a long press on Back and Enter to restart the brick.

#!/usr/bin/env python3
from time import sleep
from ev3dev.ev3 import *

lcd = Screen()

smile = True

while True:
    lcd.clear()
    lcd.draw.ellipse(( 20, 20,  60, 60))
    lcd.draw.ellipse((118, 20, 158, 60))

    if smile:
        lcd.draw.arc((20, 80, 158, 100), 0, 180)
    else:
        lcd.draw.arc((20, 80, 158, 100), 180, 360)

    smile = not smile
    lcd.update()
    sleep(1)

[ndward]

Do you agree that the simplest way to do a screen capture is to run this at the command line?
fbgrab "screenshot.png"
It seems to work for me. It saves a black and white png image called 'screenshot.png' into the current folder.

[WasabiFan]

Do you agree that the simplest way to do a screen capture is to run this at the command line?

Yes. That's actually what my Python script does; it just goes a step further in recoloring and scaling it.

This script runs forever and if I start the script from Brickman I can't exit it properly with a long-press on the Back button ( the brick just starts toggling back and forth between Brickman and the graphic display).

I'm not sure what's going on there, but I was able to repro it. I know that Brickman sends SIGTERM with pkill, but for a reason which I am unaware of it doesn't work for Brickman but does for me. Simplest workaround is to run pkill python3 after attempting to end it with Brickman.

[ddemidov]

lcd.image.paste((0,0), logo) work (no call to convert) does not work, gives error:

My bad, should be lcd.image.paste(logo, (0,0)) (reordered arguments). Also, not sure how your above examples of paste worked like this.

[ndward]

Yes, when I changed lcd.image.paste((0,0), logo) into lcd.image.paste(logo, (0,0)) it worked, so I deduce that the simplest code to display an image would be

#!/usr/bin/env python3
from ev3dev.ev3 import *
from time import sleep
from PIL import Image, ImageDraw, ImageFont, ImageOps

lcd = Screen()

logo = Image.open('pics/Bomb.bmp')
lcd.image.paste(logo, (0,0))
lcd.update()
sleep(6)  # when running from Brickman, need time to admire image

In fact the linefrom PIL import Image, ImageDraw, ImageFont, ImageOps could be shortened to from PIL import ImageDraw but perhaps it is a good habit to use the fuller line?

[ddemidov]

In fact the linefrom PIL import Image, ImageDraw, ImageFont, ImageOps could be shortened to from PIL import ImageDraw but perhaps it is a good habit to use the fuller line?

I guess a good habit is to just import what is necessary. Here you only use Image to load the logo, so you could omit other imports. You may also need ImageFont to load fonts. ImageOps is no longer necessary since you found a better way to load and display pictures. And you don't need to explicitly import ImageDraw because Screen class creates the draw handle for you.

[ndward]

... and you said in an earlier message that "you can omit Image from PIL imports. That is already imported by ev3dev.ev3." so the whole PIL import line can go, leaving

#!/usr/bin/env python3
from ev3dev.ev3 import *
from time import sleep

lcd = Screen()

logo = Image.open('pics/Bomb.bmp')
lcd.image.paste(logo, (0,0))
lcd.update()
sleep(6)  # when running from Brickman, need time to admire image

[ddemidov]

That accidentally works when you do from ev3dev.ev3 import *, as this brings into global namespace everything the ev3 module defines and imports. But in general a user has no way of knowing what is being imported by the specific module, so its better to explicitly import modules one uses.

Here Image module is used explicitly to load the bomb logo, while lcd.draw and lcd.image are instances constructed by the Screen class constructor. Strictly speaking, a user should not even know/care what classes are these instances of (except for that they have same interfaces as ImageDraw.Draw and Image).

Baseline: I would still import Image explicitly.

[ndward]

Is it possible to use print() to print to the EV3 LCD screen when a script is run from Brickman? I was expecting this script to work in Brickman but it does not, though it works fine from SSH.

#!/usr/bin/env python3

# this prints the cosine of 45 degrees
# works from SSH but not from Brickman
from time import sleep
import math

print (math.cos(math.radians(45)))
sleep(6) # give enough time for result to be read in Brickman

[ddemidov]

The following example works for me (from brickman):

#!/usr/bin/env python3
import time

for i in range(30):
    print('#' * 30)

time.sleep(5)

Keep in mind that the default font is tiny and its possible to overlook small amount of text.

[ndward]

Yes, that works fine, thank you, and so does the script that was failing for me previously - I don't know what was wrong.

[ddemidov]

Ok, it looks like this may be closed now. Feel free to reopen if necessary.

[anzy0621]

How do I input some coordinate values into the lego ev3 brick so that the robot and return to the x, y position that was provided as an input to it