[Merged by Bors] - Optimize String.prototype.normalize
#2848
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jedel1043
added
dependencies
jedel1043
requested review from
Razican,
jasonwilliams,
HalidOdat,
RageKnify,
raskad and
nekevss
Test262 conformance changes
|
Codecov Report
@@ Coverage Diff @@
## main #2848 +/- ##
==========================================
- Coverage 50.92% 50.92% -0.01%
==========================================
Files 419 419
Lines 41780 41799 +19
==========================================
+ Hits 21278 21286 +8
- Misses 20502 20513 +11
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
jedel1043
force-pushed
the
fast-normalizers
branch
from
7d739b8
to
f5615bc
Compare
raskad
approved these changes
Apr 22, 2023
|
@jedel1043 I did not look into it much, do you think we could do the same for the |
Yep! There's the |
HalidOdat
approved these changes
Apr 23, 2023
|
bors r+ |
bors bot
pushed a commit
that referenced
this pull request
Apr 23, 2023
We currently use `unicode_normalization` to handle the `String.prototype.normalize` method. However, the crate doesn't support UTF-16 as a first class string, so we had to do some hacks by converting the valid parts of a string to UTF-8, normalizing each one, encoding back to UTF-16 and concatenating everything with the unpaired surrogates within. All of this is obviously suboptimal for performance, which is why I leveraged the `icu_normalizer`, which does support UTF-16 input, to replace our current implementation. Additionally, this allows users to override the default normalization data if the `intl` feature is enabled by providing the required data in the `BoaProvider` data provider.
|
Pull request successfully merged into main. Build succeeded: |
bors
bot
changed the title
String.prototype.normalizeString.prototype.normalize
bors bot
pushed a commit
that referenced
this pull request
Apr 24, 2023
As mentioned in #2848 (comment), this uses our new default ICU4X data to replace `char::is_start` and `char::is_continue` from the `boa_unicode` crate with the [`icu_properties`](https://crates.io/crates/icu_properties) crate. Note that this doesn't deprecate `boa_unicode` yet, since that'll require some discussion about how to proceed with a now unused sub-crate.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We currently use
unicode_normalizationto handle theString.prototype.normalizemethod. However, the crate doesn't support UTF-16 as a first class string, so we had to do some hacks by converting the valid parts of a string to UTF-8, normalizing each one, encoding back to UTF-16 and concatenating everything with the unpaired surrogates within. All of this is obviously suboptimal for performance, which is why I leveraged theicu_normalizer, which does support UTF-16 input, to replace our current implementation.Additionally, this allows users to override the default normalization data if the
intlfeature is enabled by providing the required data in theBoaProviderdata provider.