Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

base64 encoding with simd-support #27433

Closed
gfoidl opened this issue Sep 20, 2018 · 4 comments · Fixed by dotnet/corefx#34529
Closed

base64 encoding with simd-support #27433

gfoidl opened this issue Sep 20, 2018 · 4 comments · Fixed by dotnet/corefx#34529
Labels
area-System.Runtime enhancement Product code improvement that does NOT require public API changes/additions tenet-performance Performance related issue
Milestone

Comments

@gfoidl
Copy link
Member

gfoidl commented Sep 20, 2018

I've prototyped base64 encoding with SSE2 / SSSE3 support here.
The input, that can't be processed via simd, is handled with the current default encoded.

Numbers look quite promising:

Method N Mean Error StdDev Scaled ScaledSD
Default 16 39.04 ns 0.7766 ns 0.6884 ns 1.00 0.00
Simd_Sse2 16 30.03 ns 0.4058 ns 0.3168 ns 0.77 0.02
Default 1000 1,072.48 ns 8.5875 ns 8.0327 ns 1.00 0.00
Simd_Sse2 1000 497.02 ns 4.4778 ns 4.1886 ns 0.46 0.01

The code is based and inspired on https://github.com/aklomp/base64 wich is licensed under BSD 2-Clause "Simplified" License. So can this code be used here in corefx under the terms of the license?

Should I pursue this approach and finally submit a PR?

An AVX2 variant was also prototyped, but some intrinsics like PermuteVar8x32 are WIP, so AVX2 wasn't pursued further, but it can be on the landscape too.

In my prototype right now there is only encoding. But for decoding Klomp has code too, so it could be incorporated, so that both encoding and decoding could be done with simd-support.

This is somewhat tracked in https://github.com/dotnet/coreclr/issues/15506#issuecomment-351498311

/cc: @ahsonkhan

@danmoseley
Copy link
Member

@tannergooding

@tannergooding
Copy link
Member

Implementation looks good. It might be nice to wait on taking this (if we do) until after we finish the API cleanup for the HWIntrinsics (some design decisions were made at the last API review, that we need to respond to).

@gfoidl
Copy link
Member Author

gfoidl commented Sep 20, 2018

@tannergooding thanks. So let's wait for the cleanup, then we can come back to productize this.

@gfoidl
Copy link
Member Author

gfoidl commented Dec 1, 2018

FYI in the WIP-project https://github.com/gfoidl/Base64 SSSE3 / AVX2 is used for encoding / decoding, and speedups for encoding up to 12 are possible (for datalength=1000). For decoding ~6x speedup is possbile (has to check for invalid inputs).

Although this project now exists, I still would like to take this here*, so that corefx can benefit from these improvements in base64 encoding / decoding.

The project also supports char-buffers as sink / source, and base64Url encoding, so I see no "conflict" with an implementation here in corefx.

* still aware of https://github.com/dotnet/corefx/issues/32365#issuecomment-423257658 as this comment should just be an informational note, not to produce any pressure / hurry... (especially for myself 😉)

@msftgits msftgits transferred this issue from dotnet/corefx Jan 31, 2020
@msftgits msftgits added this to the 3.0 milestone Jan 31, 2020
@ghost ghost locked as resolved and limited conversation to collaborators Dec 15, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-System.Runtime enhancement Product code improvement that does NOT require public API changes/additions tenet-performance Performance related issue
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants