Upload & Monitor and Test find wrong serial port for Arduino Leonardo, others

[sphereinabox]

Configuration

Windows 10, also OS X 10.14:

PlatformIO Version (platformio --version): PlatformIO, version 4.1.0b3

Description of problem

"Upload & Monitor" and "Test" will open the wrong serial port on devices such as the Arduino Leonardo or Adafruit M0 & M4 devices that use a software USB interface that re-enumerates after uploading a new program.

Steps to Reproduce

1. Set up project with code below
2. Try "Upload & Monitor" and get either wrong serial port, or error.
3. Try "Test" and after code uploads get wrong serial port, or error.

Actual Results

Depending on configuration upload & monitor or test fail in one of the following ways:

• You're prompted for which serial port to use, with a stale list of devices by the time you choose one.
• The wrong serial port is chosen, if for example you have another serial device besides the device. On OS X this is often the bluetooth serial port, so the test output will never appear.
• Windows cannot find the file specified if you specify the corresponding monitor_port or test_port

Expected Results

Platformio should wait until the device enumerates again and choose the correct serial port.

If problems with PlatformIO Build System:

The content of platformio.ini:

[env:leonardo]
platform = atmelavr
board = leonardo
framework = arduino
;monitor_port = COM10
;test_port = COM10

Source file to reproduce issue:
src/main.cpp

#include <Arduino.h>

void setup() {
  Serial.begin(9600);
  pinMode(LED_BUILTIN, OUTPUT);
}

void loop() {
  Serial.println("On");
  digitalWrite(LED_BUILTIN, HIGH);
  delay(1000);
  Serial.println("Off");
  digitalWrite(LED_BUILTIN, LOW);
  delay(1000);
}

test/test_main.cpp

#include <Arduino.h>
#include <unity.h>

// void setUp(void) {
// // set stuff up here
// }

// void tearDown(void) {
// // clean stuff up here
// }

void test_led_builtin_pin_number(void) {
    TEST_ASSERT_EQUAL(13, LED_BUILTIN);
}

void test_led_state_high(void) {
    digitalWrite(LED_BUILTIN, HIGH);
    TEST_ASSERT_EQUAL(HIGH, digitalRead(LED_BUILTIN));
}

void test_led_state_low(void) {
    digitalWrite(LED_BUILTIN, LOW);
    TEST_ASSERT_EQUAL(LOW, digitalRead(LED_BUILTIN));
}

void setup() {
    // NOTE!!! Wait for >2 secs
    // if board doesn't support software reset via Serial.DTR/RTS
    delay(2000);

    UNITY_BEGIN();    // IMPORTANT LINE!
    RUN_TEST(test_led_builtin_pin_number);

    pinMode(LED_BUILTIN, OUTPUT);
}

uint8_t i = 0;
uint8_t max_blinks = 5;

void loop() {
    if (i < max_blinks)
    {
        RUN_TEST(test_led_state_high);
        delay(500);
        RUN_TEST(test_led_state_low);
        delay(500);
        i++;
    }
    else if (i == max_blinks) {
      UNITY_END(); // stop unit testing
    }
}

Additional info

If I configure monitor_port or test_port then I get the "cannot find file specified" error in windows.

If I don't, then I will get a serial error on upload & monitor because it chooses the leonardo programming serial port which disappears when the leonardo reboots.

Here's some other people probably hitting the same issue:

• failing to run remote tests on Adafruit Feather M0 platform-atmelsam#45
• https://community.platformio.org/t/teensy-unit-testing-doesnt-execute/8101

[sphereinabox]

get_test_port() in this file:
https://github.com/platformio/platformio-core/blob/a785c238b1778d5907b3b6d106a2462069fc2cb0/platformio/test/embedded.py#L103

This function seems to be doing a lot:

• Return the configured port if there is one (regardless of if the file exists yet)
• Check & Retry for system serial ports and return the first one that appears (even if it doesn't match hardware IDs). Oh, and only in this case make sure it's possible to connect to the serial port.

I'm not entirely sure what the fix for this should be because at least on classic arduinos with a dedicated serial chip, you can't depend on the USB hardware IDs.

At the very least, if test_port or monitor_port are configured I think platformio should wait & retry for it to show up.

[sphereinabox]

Error messages on Windows 10

upload & monitor with no port configured.

notice how it's trying COM5. COM10 is the port it should use

============================================================================= [SUCCESS] Took 4.62 seconds =============================================================================
--- Miniterm on COM5  9600,8,N,1 ---
--- Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
Exception in thread rx:
Traceback (most recent call last):
  File "C:\Python27\Lib\threading.py", line 801, in __bootstrap_inner
    self.run()
  File "C:\Python27\Lib\threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "c:\users\nwint\documents\spacemacs\.platformio\penv\lib\site-packages\serial\tools\miniterm.py", line 445, in reader
    data = self.serial.read(self.serial.in_waiting or 1)
  File "c:\users\nwint\documents\spacemacs\.platformio\penv\lib\site-packages\serial\serialwin32.py", line 257, in in_waiting
    raise SerialException("ClearCommError failed ({!r})".format(ctypes.WinError()))
SerialException: ClearCommError failed (WindowsError(22, 'The device does not recognize the command.'))

upload & monitor with monitor_port configured

Correctly tries COM10, but doesn't wait for the device to show up.

============================================================================= [SUCCESS] Took 4.67 seconds =============================================================================
could not open port 'COM10': could not open port 'COM10': WindowsError(2, 'The system cannot find the file specified.')

Test with no test_port configured

[avr upload omitted]
avrdude: verifying ...
avrdude: 6224 bytes of flash verified

avrdude: safemode: Fuses OK (E:CB, H:D8, L:FF)

avrdude done.  Thank you.

Testing...
If you don't see any output for the first 10 secs, please reset board (press reset button)

Error: Traceback (most recent call last):
  File "c:\users\nwint\documents\spacemacs\.platformio\penv\lib\site-packages\platformio\__main__.py", line 102, in main
    cli()  # pylint: disable=no-value-for-parameter
  File "c:\users\nwint\documents\spacemacs\.platformio\penv\lib\site-packages\click\core.py", line 700, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\nwint\documents\spacemacs\.platformio\penv\lib\site-packages\click\core.py", line 680, in main
    rv = self.invoke(ctx)
  File "c:\users\nwint\documents\spacemacs\.platformio\penv\lib\site-packages\platformio\commands\__init__.py", line 41, in invoke
    return super(PlatformioCLI, self).invoke(ctx)
  File "c:\users\nwint\documents\spacemacs\.platformio\penv\lib\site-packages\click\core.py", line 1027, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\nwint\documents\spacemacs\.platformio\penv\lib\site-packages\click\core.py", line 873, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\nwint\documents\spacemacs\.platformio\penv\lib\site-packages\click\core.py", line 508, in invoke
    return callback(*args, **kwargs)
  File "c:\users\nwint\documents\spacemacs\.platformio\penv\lib\site-packages\click\decorators.py", line 16, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "c:\users\nwint\documents\spacemacs\.platformio\penv\lib\site-packages\platformio\commands\test.py", line 168, in cli
    "succeeded": tp.process(),
  File "c:\users\nwint\documents\spacemacs\.platformio\penv\lib\site-packages\platformio\test\embedded.py", line 52, in process
    return self.run()
  File "c:\users\nwint\documents\spacemacs\.platformio\penv\lib\site-packages\platformio\test\embedded.py", line 83, in run
    line = ser.readline().strip()
  File "c:\users\nwint\documents\spacemacs\.platformio\penv\lib\site-packages\serial\serialwin32.py", line 273, in read
    raise SerialException("ClearCommError failed ({!r})".format(ctypes.WinError()))
SerialException: ClearCommError failed (WindowsError(22, 'The device does not recognize the command.'))

============================================================

An unexpected error occurred. Further steps:

* Verify that you have the latest version of PlatformIO using
  `pip install -U platformio` command

* Try to find answer in FAQ Troubleshooting section
  https://docs.platformio.org/page/faq.html

* Report this problem to the developers
  https://github.com/platformio/platformio-core/issues

============================================================

The terminal process terminated with exit code: 1

Test with test_port configured

It doesn't wait for COM10 to become available.

Testing...
If you don't see any output for the first 10 secs, please reset board (press reset button)

could not open port 'COM10': WindowsError(2, 'The system cannot find the file specified.')
============================================================================= [FAILED] Took 5.57 seconds =============================================================================  

Test    Environment         Status    Duration
------  ------------------  --------  ------------
*       uno                 IGNORED
*       adafruit_hallowing  IGNORED
*       leonardo            FAILED    00:00:05.574
======================================================================== 1 failed, 0 succeeded in 00:00:05.574 ======================================================================== 
The terminal process terminated with exit code: 1

[sphereinabox]

Error messages on OS X 10.14.6

PlatformIO, version 4.1.0b3

leonardo serial should be on /dev/cu.usbmodem141101

Upload & Monitor with no monitor_port configured

============================================================================= [SUCCESS] Took 7.50 seconds =============================================================================
--- Miniterm on /dev/cu.usbmodem141401  9600,8,N,1 ---
--- Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---

--- exit ---
Exception in thread rx:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Users/nick/.platformio/penv/lib/python2.7/site-packages/serial/tools/miniterm.py", line 445, in reader
    data = self.serial.read(self.serial.in_waiting or 1)
  File "/Users/nick/.platformio/penv/lib/python2.7/site-packages/serial/serialposix.py", line 509, in read
    raise SerialException('read failed: {}'.format(e))
SerialException: read failed: [Errno 6] Device not configured

Upload & Monitor with monitor_portconfigured

avrdude done.  Thank you.

============================================================================= [SUCCESS] Took 5.98 seconds =============================================================================
could not open port '/dev/cu.usbmodem141101': [Errno 2] could not open port /dev/cu.usbmodem141101: [Errno 2] No such file or directory: '/dev/cu.usbmodem141101'

Test with no test_port configured

platformio tries /dev/cu.usbmodem141401 which disappears as the leonardo reboots

avrdude done.  Thank you.

Testing...
If you don't see any output for the first 10 secs, please reset board (press reset button)

Error: Traceback (most recent call last):
  File "/Users/nick/.platformio/penv/lib/python2.7/site-packages/platformio/__main__.py", line 102, in main
    cli()  # pylint: disable=no-value-for-parameter
  File "/Users/nick/.platformio/penv/lib/python2.7/site-packages/click/core.py", line 700, in __call__
    return self.main(*args, **kwargs)
  File "/Users/nick/.platformio/penv/lib/python2.7/site-packages/click/core.py", line 680, in main
    rv = self.invoke(ctx)
  File "/Users/nick/.platformio/penv/lib/python2.7/site-packages/platformio/commands/__init__.py", line 41, in invoke
    return super(PlatformioCLI, self).invoke(ctx)
  File "/Users/nick/.platformio/penv/lib/python2.7/site-packages/click/core.py", line 1027, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/nick/.platformio/penv/lib/python2.7/site-packages/click/core.py", line 873, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/nick/.platformio/penv/lib/python2.7/site-packages/click/core.py", line 508, in invoke
    return callback(*args, **kwargs)
  File "/Users/nick/.platformio/penv/lib/python2.7/site-packages/click/decorators.py", line 16, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/nick/.platformio/penv/lib/python2.7/site-packages/platformio/commands/test.py", line 168, in cli
    "succeeded": tp.process(),
  File "/Users/nick/.platformio/penv/lib/python2.7/site-packages/platformio/test/embedded.py", line 52, in process
    return self.run()
  File "/Users/nick/.platformio/penv/lib/python2.7/site-packages/platformio/test/embedded.py", line 86, in run
    line = ser.readline().strip()
  File "/Users/nick/.platformio/penv/lib/python2.7/site-packages/serial/serialposix.py", line 509, in read
    raise SerialException('read failed: {}'.format(e))
SerialException: read failed: [Errno 6] Device not configured

============================================================

An unexpected error occurred. Further steps:

* Verify that you have the latest version of PlatformIO using
  `pip install -U platformio` command

* Try to find answer in FAQ Troubleshooting section
  https://docs.platformio.org/page/faq.html

* Report this problem to the developers
  https://github.com/platformio/platformio-core/issues

============================================================

The terminal process terminated with exit code: 1

Test with test_port configured

avrdude done.  Thank you.

Testing...
If you don't see any output for the first 10 secs, please reset board (press reset button)

[Errno 2] could not open port /dev/cu.usbmodem141101: [Errno 2] No such file or directory: '/dev/cu.usbmodem141101'

[ivankravets]

Does classic uploading work as expected?

[sphereinabox]

Yeah, Upload alone works fine, and Monitor alone works fine, there's just the race condition when trying to do one then quickly do the other on these devices that do software USB serial.

I think get_test_port() for example should:

• If there is a test_port or monitor_port configured, then use the retry & timeout logic instead of immediately trying and failing to open that port after uploading code.
• If the hardware has USB device IDs for serial, then I don't think platformio should to be trying to connect with serial ports that don't match these device IDs. In particular platformio should wait to find the leonardo or whatever instead of connecting to the always-there /dev/cu.Bluetooth-Incoming-Port or maybe I should be able to blacklist /dev/cu.Bluetooth-Incoming-Port

If I were to make a pull request with the changes I propose, which branch should I start from?

[elguiri]

If anybody else has this issue prior to a fix, there is a workaround described here...
https://community.platformio.org/t/any-way-to-configure-timeout-for-upload-monitor/3812