Improve Unicode perf. #1318
Merged
Improve Unicode perf. #1318
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
While working on diagnostics for microsoft#1210 I observed that we were printing the caret ^ in the wrong place when it goes after the input. The way this works is we form the line of text to print, then decode the unicode encoding units, and when we hit the target, we stop and print ^: https://github.com/microsoft/vcpkg-tool/blob/5b8f9c40dd9d6a07ec636590e78c2f49d28624b9/src/vcpkg/base/parse.cpp#L51-L68 however, if the intended location for the ^ is the "end" of the line, we hit this bug: https://github.com/microsoft/vcpkg-tool/blob/5b8f9c40dd9d6a07ec636590e78c2f49d28624b9/src/vcpkg/base/unicode.cpp#L273 The iterator only compares the last_ pointers, but both the "points at the last code point in the input" and "points to the end of the input" state set `next_ == last_`. See: https://github.com/microsoft/vcpkg-tool/blob/5b8f9c40dd9d6a07ec636590e78c2f49d28624b9/src/vcpkg/base/unicode.cpp#L222-L226 This means that the points to the end and points one past the end iterator compare equal, so the loop in parse.cpp stops one position too early. Also adds a bunch of testing for this specific case, for other parts of Utf8Decoder, adds a way to parse the first code point without failing, makes all the operators 'hidden friends', and removes localized strings for bugs-in-vcpkg-itself.
For `vcpkg format-manifest --all` this change results in: After Before 1.3853984 1.4387856 1.3930014 1.5002737 1.4488841 1.4187639 1.3823215 1.4723308 1.4366255 1.4213668 1.3880373 1.4606811 Average 1.405711367 1.45203365 StdDev 0.026624509 0.028973257
…ent-pointer # Conflicts: # include/vcpkg/base/unicode.h # src/vcpkg-test/unicode.cpp # src/vcpkg/base/unicode.cpp
BillyONeal
commented
Jan 5, 2024
| } | ||
|
|
||
| bool utf8_is_valid_string(const char* first, const char* last) noexcept | ||
| { | ||
| utf8_errc err = utf8_errc::NoError; | ||
| for (auto dec = Utf8Decoder(first, last); dec != dec.end(); err = dec.next()) |
There was a problem hiding this comment.
Stealth bug fix: the previous implementation would crash if the first code point was invalid.
BillyONeal
commented
Jan 8, 2024
ras0219-msft
approved these changes
Jan 11, 2024
|
After applying Robert's suggestion I now get:
if I throw out the outlier 4.3 second one:
(These are on the Zypherus G14 / Ryzen 9 7940HS) |
|
Reran on the 7950X and the values are closer. Don't know what to make of that 🤷 |
|
I ran more trials and maybe on the 7950X it makes no difference :/. I still like this change though
|
|
OK I give up trying to characterize it :P
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In #1316 I noticed that callers expect
pointer_to_currentto be cheap, and we were doubling the number of branches by converting to enums first.For
vcpkg format-manifest --allthis change results in:Average 1.405711367 1.45203365
StdDev 0.026624509 0.028973257
(@ras0219-msft edit: -3.2% faster)