-
Notifications
You must be signed in to change notification settings - Fork 10.6k
Vectorize UTF16->UTF8 transcoding #83407
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Catfish-Man
wants to merge
20
commits into
swiftlang:main
Choose a base branch
from
Catfish-Man:asciivec-2
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+496
−15
Open
Changes from all commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
6ae47a3
WIP vectorization for UTF16->UTF8
Catfish-Man 246939c
Lots of fixes
Catfish-Man f85efe5
Fun fact: UInt16 is not the same size as UInt8
Catfish-Man 4b84ced
See if the scalar version autovectorizes on arm64 too
Catfish-Man 25ac970
Build fix for experiment
Catfish-Man 931ae62
Remove arm64-specific code
Catfish-Man 8e9f5e0
Adjust for 32 bit
Catfish-Man f0cee25
Stop doing size math, stop duplicating work in some cases, and delete…
Catfish-Man 4b9be8f
Adopt the new implementation in another place, add unsafe annotations…
Catfish-Man f326f61
Actually detect non-ascii in the fallback path
Catfish-Man 9263ce6
Remove pointless failed attempt at being clever
Catfish-Man 3177957
Do it all by hand, since empirically it's a lot faster for runs of no…
Catfish-Man 9d6d225
Merge branch 'main' into asciivec
Catfish-Man 9dc0c96
Add a (slow) scalar fallback path, and add more unsafe annotations
Catfish-Man bb2437d
Fix precondition
Catfish-Man 3105736
Optimize utf8Length to handle values < 0x1000 with the vector path
Catfish-Man af65484
Expand the table to 128 entries and use an InlineArray instead of a t…
Catfish-Man 1c80d60
Fix 32 bit
Catfish-Man 72738d2
Fix comment
Catfish-Man 3358192
Sigh, don't get to use InlineArray after all
Catfish-Man File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Once we're ready to push forward on this again, a test I just ran locally suggests that adding a callback to resize the allocation if needed (guess all-ascii, then resize by 2x, then by 1.5x) is actually quite a bit faster than sizing up front.
We'll see if I can make that work for String, will require a bit of shenanigans probably.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fun option: port String growth-on-append to
reallocthen use that