Serial monitor character encoding option

[Ivan-Perez]

Describe the request

It would be great if in the serial monitor had an option to change the encoding used.

In my case, my sketches are using UTF-8, so print messages use that encoding. By default, serial monitor is using ISO-8859 (probably the default one of Windows 7), so those print messages are not shown properly:

Additional context

I've found other issues talking about this problem. Instead of detecting the encoding used, it might be easier if the user could select the charset he wants to use. The option (a selectable list with the most used character sets) may be put in the bottom right corner, left of the two existing selects (baud rate and line feed).

---

Additional requests:

• serial monitor (interfase) add mode ASCII or ANSI or UTF8 Arduino#7856

[lmihalkovic]

Problem identified in 2014 with editor arduino/Arduino#2430, but May 2015 comment contains reference to limitation with serial monitor encoding support

[aknrdureegaesr]

I did some research on UTF-8 behavior today. I found this code in arduino-core/src/processing/app/Serial.java which does the conversion from the incoming bytes to the strings displayed by the UI:

  public synchronized void serialEvent(SerialPortEvent serialEvent) {
    if (serialEvent.isRXCHAR()) {
      try {
        byte[] buf = port.readBytes(serialEvent.getEventValue());
        if (buf.length > 0) {
          String msg = new String(buf);
          char[] chars = msg.toCharArray();
          message(chars, chars.length);
        }
      } catch (SerialPortException e) {
        errorMessage("serialEvent", e);
      }
    }
  }

The String-constructor used decodes the incoming bytes according to the particular platform's default charset, its documentation says.

I think the standard should be to always use UTF-8. So a sketch developed and programmed on a Mac works with a Linux or Windows box and vice versa. If we later want to add HEX display, as proposed by #1727, here would be one place to do it.

There is a catch here: In our context, there is no guarantee whatsoever that the bytes that come in do us the favor to split neatly at character boundaries. It is quite feasible they include only the first byte of a character that is encoded into several bytes, and the other bytes come later, with the next call.

In this case, the String-constructor is documented to exhibit undocumented behavior :-) .

In actual tests, I indeed see erratic output whenever I use non-ASCII characters in my sketches. Sometimes it works, sometimes it doesn't. (This is Linux, and I'm almost certain the platform encoding is UTF-8.)

The String-constructor documentation advises to use a CharsetDecoder instead.

I think that's good advice. This would give control over the encoding used. Clean UTF-8 decoding even in the split character case is a feature included in CharsetDecoder.

[AtosNicoS]

Hi,
I also recently stumbled across that limitation. The Serial Monitor now supports reading in UTF-8, so writing should also default to UTF-8.