Changed base64 implementation to Abseil's.#44915
Conversation
Signed-off-by: Ethan Marantz <ebmarantz@gmail.com>
|
/retest |
There was a problem hiding this comment.
Pull request overview
This PR updates Envoy’s core Base64 utilities to use Abseil’s Base64 encode/decode helpers, aiming to preserve existing behavior while reducing custom implementation code (per #12694 and related PRs).
Changes:
- Switched Base64/Base64Url encode/decode to
absl::Base64*andabsl::WebSafeBase64*functions. - Preserved Envoy-specific strict validation (e.g., rejecting invalid trailing bits) via explicit checks before decoding.
- Adjusted no-padding behavior by stripping trailing
=on encode and tolerating missing padding on decode.
…64 implementations. Signed-off-by: Ethan Marantz <ebmarantz@gmail.com>
|
Running on my system, these are what the speed benchmarks show: -------------------------------------------------------------------------------------------
Benchmark Time CPU Iterations UserCounters...
-------------------------------------------------------------------------------------------
BM_AbslBase64Encode/8 20.0 ns 19.9 ns 34555635 bytes_per_second=382.932Mi/s
BM_AbslBase64Encode/64 56.3 ns 56.0 ns 12831974 bytes_per_second=1.0653Gi/s
BM_AbslBase64Encode/512 254 ns 253 ns 2797137 bytes_per_second=1.88573Gi/s
BM_AbslBase64Encode/4096 1856 ns 1834 ns 383248 bytes_per_second=2.08003Gi/s
BM_AbslBase64Encode/32768 14341 ns 14243 ns 48645 bytes_per_second=2.14269Gi/s
BM_AbslBase64Encode/262144 113306 ns 112144 ns 5953 bytes_per_second=2.17703Gi/s
BM_AbslBase64Encode/1048576 456593 ns 451077 ns 1537 bytes_per_second=2.16496Gi/s
BM_LegacyBase64Encode/8 42.4 ns 42.1 ns 16265013 bytes_per_second=181.062Mi/s
BM_LegacyBase64Encode/64 317 ns 314 ns 2227138 bytes_per_second=194.263Mi/s
BM_LegacyBase64Encode/512 2272 ns 2242 ns 325554 bytes_per_second=217.833Mi/s
BM_LegacyBase64Encode/4096 19538 ns 19138 ns 40061 bytes_per_second=204.114Mi/s
BM_LegacyBase64Encode/32768 147673 ns 143314 ns 4863 bytes_per_second=218.053Mi/s
BM_LegacyBase64Encode/262144 1146333 ns 1134801 ns 579 bytes_per_second=220.303Mi/s
BM_LegacyBase64Encode/1048576 4723849 ns 4659935 ns 150 bytes_per_second=214.595Mi/s
BM_AbslBase64Decode/8 33.6 ns 33.3 ns 21628962 bytes_per_second=343.608Mi/s
BM_AbslBase64Decode/64 90.2 ns 89.6 ns 7645094 bytes_per_second=936.221Mi/s
BM_AbslBase64Decode/512 420 ns 418 ns 1677261 bytes_per_second=1.52471Gi/s
BM_AbslBase64Decode/4096 3098 ns 3084 ns 228345 bytes_per_second=1.64983Gi/s
BM_AbslBase64Decode/32768 24430 ns 24329 ns 28856 bytes_per_second=1.67258Gi/s
BM_AbslBase64Decode/262144 198126 ns 197117 ns 3557 bytes_per_second=1.65142Gi/s
BM_AbslBase64Decode/1048576 819272 ns 814537 ns 843 bytes_per_second=1.59856Gi/s
BM_LegacyBase64Decode/8 62.8 ns 62.5 ns 11391560 bytes_per_second=183.199Mi/s
BM_LegacyBase64Decode/64 476 ns 474 ns 1465152 bytes_per_second=177.106Mi/s
BM_LegacyBase64Decode/512 3606 ns 3568 ns 190385 bytes_per_second=182.807Mi/s
BM_LegacyBase64Decode/4096 28209 ns 28001 ns 25335 bytes_per_second=186.097Mi/s
BM_LegacyBase64Decode/32768 222668 ns 221735 ns 3157 bytes_per_second=187.917Mi/s
BM_LegacyBase64Decode/262144 1790706 ns 1781131 ns 395 bytes_per_second=187.148Mi/s
BM_LegacyBase64Decode/1048576 7136770 ns 7104609 ns 98 bytes_per_second=187.672Mi/s
BM_AbslBase64UrlEncode/8 19.8 ns 19.7 ns 35520575 bytes_per_second=387.775Mi/s
BM_AbslBase64UrlEncode/64 54.9 ns 54.6 ns 12820113 bytes_per_second=1.09093Gi/s
BM_AbslBase64UrlEncode/512 250 ns 249 ns 2792840 bytes_per_second=1.91567Gi/s
BM_AbslBase64UrlEncode/4096 1824 ns 1801 ns 382329 bytes_per_second=2.11833Gi/s
BM_AbslBase64UrlEncode/32768 14114 ns 14051 ns 49879 bytes_per_second=2.17199Gi/s
BM_AbslBase64UrlEncode/262144 110826 ns 110293 ns 6301 bytes_per_second=2.21357Gi/s
BM_AbslBase64UrlEncode/1048576 449509 ns 445350 ns 1579 bytes_per_second=2.1928Gi/s
BM_LegacyBase64UrlEncode/8 40.4 ns 39.5 ns 18084066 bytes_per_second=193.075Mi/s
BM_LegacyBase64UrlEncode/64 316 ns 312 ns 2254509 bytes_per_second=195.413Mi/s
BM_LegacyBase64UrlEncode/512 2290 ns 2266 ns 316214 bytes_per_second=215.442Mi/s
BM_LegacyBase64UrlEncode/4096 18420 ns 18085 ns 36982 bytes_per_second=215.999Mi/s
BM_LegacyBase64UrlEncode/32768 141624 ns 140161 ns 4949 bytes_per_second=222.958Mi/s
BM_LegacyBase64UrlEncode/262144 1205516 ns 1180459 ns 625 bytes_per_second=211.782Mi/s
BM_LegacyBase64UrlEncode/1048576 4616359 ns 4591294 ns 153 bytes_per_second=217.804Mi/s
BM_AbslBase64UrlDecode/8 31.7 ns 31.4 ns 22154215 bytes_per_second=333.781Mi/s
BM_AbslBase64UrlDecode/64 94.5 ns 93.8 ns 7348945 bytes_per_second=874.68Mi/s
BM_AbslBase64UrlDecode/512 458 ns 453 ns 1580037 bytes_per_second=1.40527Gi/s
BM_AbslBase64UrlDecode/4096 3252 ns 3224 ns 217977 bytes_per_second=1.5776Gi/s
BM_AbslBase64UrlDecode/32768 25517 ns 25355 ns 28234 bytes_per_second=1.60482Gi/s
BM_AbslBase64UrlDecode/262144 202193 ns 200960 ns 3421 bytes_per_second=1.61983Gi/s
BM_AbslBase64UrlDecode/1048576 832491 ns 824100 ns 891 bytes_per_second=1.58001Gi/s
BM_LegacyBase64UrlDecode/8 60.4 ns 59.6 ns 11777184 bytes_per_second=175.948Mi/s
BM_LegacyBase64UrlDecode/64 457 ns 450 ns 1485901 bytes_per_second=182.246Mi/s
BM_LegacyBase64UrlDecode/512 3554 ns 3482 ns 200294 bytes_per_second=187.06Mi/s
BM_LegacyBase64UrlDecode/4096 27639 ns 27360 ns 25995 bytes_per_second=190.386Mi/s
BM_LegacyBase64UrlDecode/32768 225326 ns 221805 ns 3194 bytes_per_second=187.854Mi/s
BM_LegacyBase64UrlDecode/262144 1784478 ns 1772074 ns 399 bytes_per_second=188.104Mi/s
BM_LegacyBase64UrlDecode/1048576 6917716 ns 6841059 ns 100 bytes_per_second=194.902Mi/s |
Signed-off-by: Ethan Marantz <ebmarantz@gmail.com>
…zing tests for BufferInstance. Added failed test to the unit test. Signed-off-by: ethan <ethan@debian-server.home.arpa>
jmarantz
left a comment
There was a problem hiding this comment.
Can you put the performance data (including the buffer encoding speed) into the PR description?
This is looking great!
4eea06f to
56bd9b3
Compare
Signed-off-by: Ethan Marantz <ebmarantz@gmail.com> Signed-off-by: ethan <ethan@debian-server.home.arpa>
56bd9b3 to
d17da1d
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request refactors the Base64 and Base64Url encoding and decoding logic to utilize Abseil's escaping utilities, simplifying the implementation. It also introduces comprehensive differential fuzzing and performance benchmarks to ensure functional parity with the legacy implementation. Feedback was provided regarding a performance regression in the Base64::encode method for Buffer::Instance, where an extra intermediate allocation was introduced; an optimization was suggested to handle contiguous buffers more efficiently.
…check to skip intermediate copy when buffer contains one slice. Signed-off-by: Ethan Marantz <ebmarantz@gmail.com>
|
With this change, single sliced Buffer::Instance get encoded 5-15% faster than the old code. Multi sliced Buffer::Instance show no change which is expected. |
Signed-off-by: Ethan Marantz <ebmarantz@gmail.com>
|
Please review as a senior maintainer @RyanTheOptimist . |
RyanTheOptimist
left a comment
There was a problem hiding this comment.
Looks awesome. Thanks for doing this!
| @@ -0,0 +1 @@ | |||
| % | |||
There was a problem hiding this comment.
Out of curiosity, why does this change?
There was a problem hiding this comment.
I copied it over from the existing base64 fuzz test's corpus (test/common/common/base64_corpus). I kept base64_corpus and base64_differential_corpus as separate directories to signify that they are different tests with different discovered inputs.
Commit Message: replaces base64 implementation with Abseil's base64 implementation.
Additional Description: Replaces the existing base64 implementation with Abseil’s base64 one. This change maintains identical encoding/decoding behavior.
Benchmarks show
~2x at 8 bytes
~5x at 64 bytes
~8-10x at 512 bytes and above
Full Benchmark
Risk Level: Medium (encoding/decoding is core utils logic and should preserve behavior if tests pass).
Testing: Existing unit tests for base64 encoding/decoding pass. Added differential fuzz testing (test/common/common/base64_fuzz_test.cc) and benchmarks (test/common/common/base64_speed_test.cc) comparing the new base64 implementation against the legacy implementation.
Docs Changes: None
Release Notes: Base64 implementation updated to use Abseil utilities. No behavioral changes expected.
Platform Specific Features: N/A
[Optional Fixes #Issue]: Fixes #12694.
[Optional Fixes commit #PR or SHA]: Related to PR #29360 and PR #12703.