'mpremote mip install' fails if lib folder does not exist

[mattytrentini]

Port, board and/or hardware

Adafruit Qt Py SAMD

MicroPython version

I believe any board with an empty filesystem will exhibit the issue.

Reproduction

$mpremote mip install neopixel

Expected behaviour

$mpremote mip install neopixel
Install neopixel
Installing neopixel (latest) from https://micropython.org/pi/v2 to /lib
Installing: /lib/neopixel.mpy
Done

Observed behaviour

$mpremote mip install neopixel
Install neopixel
Installing neopixel (latest) from https://micropython.org/pi/v2 to /lib
Installing: /lib/neopixel.mpy
Traceback (most recent call last):
  File "/usr/local/bin/mpremote", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/mpremote/main.py", line 538, in main
    handler_func(state, args)
  File "/usr/local/lib/python3.11/dist-packages/mpremote/mip.py", line 166, in do_mip
    _install_package(
  File "/usr/local/lib/python3.11/dist-packages/mpremote/mip.py", line 133, in _install_package
    _install_json(transport, package, index, target, version, mpy)
  File "/usr/local/lib/python3.11/dist-packages/mpremote/mip.py", line 94, in _install_json
    _download_file(transport, file_url, fs_target_path)
  File "/usr/local/lib/python3.11/dist-packages/mpremote/mip.py", line 69, in _download_file
    _ensure_path_exists(transport, dest)
  File "/usr/local/lib/python3.11/dist-packages/mpremote/mip.py", line 29, in _ensure_path_exists
    if not transport.fs_exists(prefix):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/mpremote/transport.py", line 100, in fs_exists
    self.fs_stat(src)
  File "/usr/local/lib/python3.11/dist-packages/mpremote/transport.py", line 96, in fs_stat
    raise _convert_filesystem_error(e, src) from None
  File "/usr/local/lib/python3.11/dist-packages/mpremote/transport.py", line 94, in fs_stat
    return os.stat_result(self.eval("os.stat(%s)" % ("'%s'" % src)))
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/mpremote/transport_serial.py", line 259, in eval
    ret = self.exec("print(repr({}))".format(expression))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/mpremote/transport_serial.py", line 270, in exec
    raise TransportExecError(ret, ret_err.decode())
mpremote.transport.TransportExecError: Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: 2

Additional Information

$mpremote --version
mpremote 1.24.0

A workaround is to create the lib directory and reinstall:

$mpremote mkdir :lib
$mpremote mip install neopixel
Install neopixel
Installing neopixel (latest) from https://micropython.org/pi/v2 to /lib
Installing: /lib/neopixel.mpy
Done

Code of Conduct

Yes, I agree

[mattytrentini]

I'm reporting this on behalf of a Discord user.

However, I have tried reproducing it (on a mimxrt board) and cannot reproduce the issue:

$mpremote rm :lib
rm :lib
$mpremote mip install neopixel
Install neopixel
Installing neopixel (latest) from https://micropython.org/pi/v2 to /flash/lib
Installing: /flash/lib/neopixel.mpy
Done

I'll leave this issue open until I hear back from the user about whether there may have been something unusual about their board...

[felixdoerre]

A quick guess what happens: mpremote scans the repl reply for symbolic OSError fragments (https://github.com/micropython/micropython/blob/master/tools/mpremote/mpremote/transport.py#L57), however the reported stack trace shows OSError: 2 (maybe a board with less verbose error strings to save space?).

This causes the OSError to not be translated but stay a generic TransportExecError which is then not caught by https://github.com/micropython/micropython/blob/master/tools/mpremote/mpremote/transport.py#L102. And that leads to the directory not being detected as non-existent, but an error to be raised.

[felixdoerre]

So, I guess, to reproduce this, you'll need a build without MICROPY_PY_ERRNO (

micropython/py/objexcept.c

Line 183 in 0e490b7

#if MICROPY_PY_ERRNO

)

[robert-hh]

I just tried mpremote mip install neopixel with a SAMD QT PY board, and it worked fine. The /lib directory did not exist when I tried it.

[mattytrentini]

Our best guess is that another tool (rshell in this case) was already connected to the device preventing mpremote from installing. I think the lack of a lib folder was a red herring!

It would be good to catch the various error conditions and provide human-readable error messages...but that's a different issue entirely!

I'll close this issue.