You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
C++11 introduced standardized strings with encoding via char16_t, char32_t (and later char8_t). Those are "at least" 2 or 4 bytes respectively wide. So using utf_traits<char16_t> might fail if that happens to be 4 bytes wide on some platform.
Proposed solution: Templatize utf_traits over an encoding instead. This can be a plain-old-enum where values UTF8, UTF16, UTF32 are set to 1, 2, 4 to allow this to be backwards compatible. Then dispatch (via an indirection if required) the charN_t types to their known encoding. Not sure if with the current impl one can directly make utf_traits<char16_t, N>==utf_traits<char16_t, 2> so indirection might be required.
The text was updated successfully, but these errors were encountered:
Flamefire
changed the title
Encoding with utf_traits can be wrong on C++11
Encoding with utf_traits can be wrong with C++11 string types
Dec 13, 2019
The
utf_traits
class uses the size of the char to switch between UTF-8, UTF-16, UTF32, e.g. forsizeof(CHAR)==2
it uses UTF-16: https://github.com/boostorg/locale/blob/develop/include/boost/locale/utf.hpp#L312C++11 introduced standardized strings with encoding via
char16_t, char32_t
(and laterchar8_t
). Those are "at least" 2 or 4 bytes respectively wide. So usingutf_traits<char16_t>
might fail if that happens to be 4 bytes wide on some platform.Proposed solution: Templatize
utf_traits
over an encoding instead. This can be a plain-old-enum where valuesUTF8, UTF16, UTF32
are set to1, 2, 4
to allow this to be backwards compatible. Then dispatch (via an indirection if required) thecharN_t
types to their known encoding. Not sure if with the current impl one can directly makeutf_traits<char16_t, N>==utf_traits<char16_t, 2>
so indirection might be required.The text was updated successfully, but these errors were encountered: