Vectorize UTF16->UTF8 transcoding #83407
Conversation
… an unnecessary usableFromInline
…, and special case empty buffers
|
@swift-ci please smoke test |
|
@swift-ci please Apple Silicon benchmark |
|
@swift-ci please benchmark |
|
@swift-ci please smoke test |
|
@swift-ci please smoke test |
|
@swift-ci please smoke test |
|
@swift-ci please Apple Silicon benchmark |
|
Just to double check that I didn't break perf when I switched away from InlineArray :) |
| @_transparent func withLengthLUT( | ||
| _ work: (UnsafeRawBufferPointer) -> Int | ||
| ) -> Int { | ||
| let lut: ( |
There was a problem hiding this comment.
We can probably use an InlineArray<128, UInt8> here.
There was a problem hiding this comment.
Oh I didn't read your latest message that said you came from an InlineArray. Any reason you switched away from it?
There was a problem hiding this comment.
Availability issues :/
There was a problem hiding this comment.
But this approach is actually all wrong anyway, and only faster because the codegen for our SIMD types has issues. Steve says he’s going to fix it, then I’ll retest and likely delete the table entirely.
| Values >= 0x4000 are handled via the scalar path to keep the table size down | ||
| */ | ||
| @_transparent func withLengthLUT( | ||
| _ work: (UnsafeRawBufferPointer) -> Int |
There was a problem hiding this comment.
Can we start using spans here?
| repairing: Bool = true | ||
| ) -> (String, repairsMade: Bool)? { | ||
| if input.isEmpty { return ("", repairsMade: false) } | ||
| guard let (utf8Len, isASCII) = unsafe utf8Length( |
There was a problem hiding this comment.
Once we're ready to push forward on this again, a test I just ran locally suggests that adding a callback to resize the allocation if needed (guess all-ascii, then resize by 2x, then by 1.5x) is actually quite a bit faster than sizing up front.
We'll see if I can make that work for String, will require a bit of shenanigans probably.
There was a problem hiding this comment.
Fun option: port String growth-on-append to realloc then use that
2 participants
Same as #83073, but with an additional optimization I wanted to try in parallel
Fixes rdar://141789595