Line editing & history problem

[drawkula]

Trying to scroll through history lines, the 1st line is not displayed until hitting the up key again or CTRL-L.

While edititing a line, the cursor often is displaced by 1 position, again CTRL-L helps.

This happens while being connected serially or over websocket running mpfshell in an xterm with screen.

It does not happen when connecting using webrepl.html or using screen /dev/ttyUSB0 115200 inside the same xterm like above.

[wendlers]

Need to check that later on. If you don't mind, you could try using 'miniterm.py' (which comes with pyserial), and see if it behaves also odd for you. Because "miniterm.py" is used for the REPL. It should work (for pyserial >= 3) like so:

miniterm.py /dev/ttyXXX 115200

[drawkula]

miniterm.py (default python is python2 here) and python3 /usr/bin/miniterm.py running in xterm with and without screen are showing the same effects.

[wendlers]

Ahhh, that's bad news. Looks like I need to write a own terminal emulation sooner or later :-/ Thanks for testing!

[wendlers]

This is really crazy. I get the errors as described with Python 3 + PySerial 3. It is not there with Python 2 + PySerial 3.

[wendlers]

For whatever reason, today (with latest micropython from repo) I am not able to reproduce this any more. Not with Python 3, not with Python 2. Thus, I will close this issue for now.

[drawkula]

Testing commit 099f776735d0aa04111011dbec9ac59063f37f4c with MicroPython v1.8.1-162-g92a9078 on 2016-07-02; ESP module with ESP8266 on ttyUSB0 and Python2 and Python3 showed that problem still being alive here...

Typing

>>> 1
1
>>> 2
2
>>> 3
3

gives a single >>> prompt. Then UP-ARROW seems to do nothing but a following RETURN makes the missing line appear and executes it.

Or type something and hit LEFT-ARROW... the 1st press does not show any reaction, the second one does but the cursor and the real point where keypresses get inserted are off by one...

This happens in xterm+screen and in xterm alone.

[GHPS]

I can confirm that the bug is still present with MicroPython v1.8.4-10-gbc28ac8 on 2016-09-09; ESP module with ESP8266 and Python 3.5.2 using PySerial 3.2.1.

The problem is indeed related to miniterm as
/usr/local/bin/miniterm.py /dev/ttyUSB0 115200
has the same problem.

It was suggested [1] that the problem was caused by a missing "-f direct" parameter when calling miniterm. However this didn't fix the problem for me.

So I tried taking a closed look at what is send/received by miniterm - the "-f debug" switch.
When typing "print() RETURN UP" I get this:
`/usr/local/bin/miniterm.py -f debug /dev/ttyUSB0 115200

--- Miniterm on /dev/ttyUSB0 115200,8,N,1 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
[TX:'p'] [RX:'p'] p [TX:'r'] [RX:'r'] r [TX:'i'] [RX:'i'] i [TX:'n'] [RX:'n'] n [TX:'t'] [RX:'t'] t [TX:'('] [RX:'\n'] [TX:')'] [RX:')'] ) [TX:'\r\n'] [RX:'\r']
[RX:'\n']
[RX:'>>> '] >>> [TX:'\x1b']

Perhaps I'm wrong but if I don't seen any answer here is it simply because MicroPython doesn't send one?

1: https://forum.micropython.org/viewtopic.php?t=2502

[drawkula]

PYB: soft reboot
#5 ets_task(40100164, 3, 3fff8310, 4)
could not open file 'main.py' for reading
MicroPython v1.8.5-110-g3272afe on 2016-11-03; ESP module with ESP8266
Type "help()" for more information.
>>> print()

>>> print()█

On a naked, freshly flashed MicroPython typing print(), ENTER and UP-ARROW via screen /dev/ttyUSB0 115200 does behave correctly, so I assume MicroPython sends the correct stuff but PySerial somewhere looses it.

MicroPython does not know who is listening on the serial so how should it behave differently depending on the terminal being connected?

[GHPS]

so I assume MicroPython sends the correct stuff but PySerial somewhere looses it.

Yes, PySerial could be the culpit, too.

So I took a look at the USB link with wireshark. After the outgoing traffic for the UP-ARROW key
there is no more incoming traffic.

MicroPython does not know who is listening on the serial so how should it behave
differently depending on the terminal being connected?

You are correct - it somehow depends on the terminal program or precisely the way they connect.

Could you please check the difference between screen and miniterm in the initial sequence when connecting to MicroPython? (Wireshark should do the job...)

---

Update 1: Don't look any further: The problem is caused by miniterm. The cursor keys are handled incorrectly here (in getkey() to be precise).

Update 2: Reported the bug in pyserial, see link below.

[GHPS]

Since no-one on the pyserial repository came up with an idea I tried tweaking the code a bit to see where the problem originates. Things start getting wrong in the getkey function in miniterm.py (line 161). Here key presses get delayed and sent out later as a block.

After extensive testing I finally changed the time out parameter from 'None' to 0 and the REPL works fine now:
[Line 162] ready, _, _ = select.select([self.enc_stdin, self.pipe_r], [], [], 0)

If you are affected by this bug please check if this patch solves your problem.

[GHPS]

There is a fix for this problem in pyserial/miniterm. It's not yet part of an official release but you can get the patched version of miniterm.py here

https://github.com/pyserial/pyserial/blob/cab3dabc81ecbf010d81ff38433db99c1f6d667c/serial/tools/miniterm.py

I consider this bug fixed.

[GHPS]

Final remark: On March 8th the patch has been released as part of the official version 3.3 of pyserial. There is no longer a need to download miniterm.py - just updated the package with pip (or your favorite package manager).