UTF8 and Unicode #25
Merged
UTF8 and Unicode #25
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* added UTF-8 support as its own standard implementation - http://tools.ietf.org/html/rfc3629 * allows to create UTF-8 abstraction of byte sequences and decode as 32bit value * added some basic unit tests
* added code value description * updated some basic unit tests
* updated byte sequence detection * provided new `byteSequenceLengthIndication` functionality
* split single header into header file and compilation unit * updated expansion functionality to support UTF-8
* provided a new internal helper function to extract a UTF-8 slice of a given source file and the source positions
* added support to represent UTF-8 byte sequence as unicode value and string representation
* primer support of common Unicode planes and block ranges * added functionality to test if UTF-8 characters are inside certain block ranges * provided proper unit tests
This was referenced Sep 7, 2019
emmanuel099
approved these changes
Sep 11, 2019
| EXPECT_EQ( range.plane(), Plane::SUPPLEMENTARY_MULTILINGUAL ); | ||
| break; | ||
| } | ||
| case Block::TRANSPORT_AND_MAP_SYMBOLS: |
There was a problem hiding this comment.
unreachable?
Better use multiple test cases (or parameterized test case) instead of the for loop.
Makes it easier to follow and easier to detect the faulty block in case of an test case error.
There was a problem hiding this comment.
@emmanuel099 good catch, I've created an issue, which addresses this comment and will fix this problem with the suggested solution in a future PR
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Unicodewhich provides an alias forUTF8and defines the Unicode planes, blocks, and rangesUnicodeconsists of helper functions to check of aUTF8is inside one or multiple block ranges