encode/decode strings with UTF-8

[dirk-thomas]

This patch rolls back the changes from #26 and encodes / decodes (non W-)strings as UTF-8.

• Linux
• Linux-aarch64
• macOS
• Windows

[hidmic]

This patch rolls back the changes from #26

Does it? Also, I wonder, will this play along with other non-Python nodes? Encoded UTF-8 isn't ASCII nor any stream of bytes is UTF-8 decodable. What's the premise here?

[dirk-thomas]

Does it?

It doesn't revert the exact change but rolls back the semantic change of enforcing ASCII.

will this play along with other non-Python nodes?

Yes, the referenced test cases cover C++ as well as Python and the tests in test_communication ensure that the information is correctly passed between languages.

Encoded UTF-8 isn't ASCII nor any stream of bytes is UTF-8 decodable.

We follow the X-Types spec to encode (non W-)strings with UTF-8. If you only use ASCII characters nothing changes here since all ASCII characters have the same representation when encoded with UTF-8. If you do use non-ASCII characters they are being encoded, the wire format contains UTF-8 encoded bytes, and the receiver side decodes the string.

If you have an external entity which publishes strings with data which can't be UTF-8 decoded (an violate the X-Types spec) our receiving nodes will bail.

[hidmic]

We follow the X-Types spec

Oh, that's it. Perfect. LGTM too!