Skip to content

Changed base64 implementation to Abseil's.#44915

Merged
RyanTheOptimist merged 7 commits into
envoyproxy:mainfrom
ebm:change-base64-implementation
May 21, 2026
Merged

Changed base64 implementation to Abseil's.#44915
RyanTheOptimist merged 7 commits into
envoyproxy:mainfrom
ebm:change-base64-implementation

Conversation

@ebm
Copy link
Copy Markdown
Contributor

@ebm ebm commented May 7, 2026

Commit Message: replaces base64 implementation with Abseil's base64 implementation.

Additional Description: Replaces the existing base64 implementation with Abseil’s base64 one. This change maintains identical encoding/decoding behavior.

Benchmarks show
~2x at 8 bytes
~5x at 64 bytes
~8-10x at 512 bytes and above

Full Benchmark
----------------------------------------------------------------------------------------------------------------
Benchmark                                                      Time             CPU   Iterations UserCounters...
----------------------------------------------------------------------------------------------------------------
BM_AbslBase64Encode/8                                       20.5 ns         20.4 ns     33893373 bytes_per_second=374.723Mi/s
BM_AbslBase64Encode/64                                      56.1 ns         55.8 ns     12443880 bytes_per_second=1.06846Gi/s
BM_AbslBase64Encode/512                                      248 ns          247 ns      2822896 bytes_per_second=1.93045Gi/s
BM_AbslBase64Encode/4096                                    1777 ns         1770 ns       395332 bytes_per_second=2.15477Gi/s
BM_AbslBase64Encode/32768                                  14170 ns        14108 ns        50054 bytes_per_second=2.16317Gi/s
BM_AbslBase64Encode/262144                                110850 ns       110328 ns         6340 bytes_per_second=2.21287Gi/s
BM_AbslBase64Encode/1048576                               442523 ns       440467 ns         1589 bytes_per_second=2.21711Gi/s
BM_LegacyBase64Encode/8                                     42.7 ns         42.5 ns     16462454 bytes_per_second=179.434Mi/s
BM_LegacyBase64Encode/64                                     310 ns          308 ns      2324094 bytes_per_second=197.93Mi/s
BM_LegacyBase64Encode/512                                   2202 ns         2193 ns       322767 bytes_per_second=222.7Mi/s
BM_LegacyBase64Encode/4096                                 17763 ns        17687 ns        40018 bytes_per_second=220.852Mi/s
BM_LegacyBase64Encode/32768                               141403 ns       140443 ns         5038 bytes_per_second=222.51Mi/s
BM_LegacyBase64Encode/262144                             1127638 ns      1121530 ns          633 bytes_per_second=222.91Mi/s
BM_LegacyBase64Encode/1048576                            4445197 ns      4424678 ns          155 bytes_per_second=226.005Mi/s
BM_AbslBase64Decode/8                                       25.9 ns         25.8 ns     26666600 bytes_per_second=443.357Mi/s
BM_AbslBase64Decode/64                                      80.2 ns         79.8 ns      8809031 bytes_per_second=1.02672Gi/s
BM_AbslBase64Decode/512                                      398 ns          396 ns      1774014 bytes_per_second=1.607Gi/s
BM_AbslBase64Decode/4096                                    3000 ns         2985 ns       237276 bytes_per_second=1.70479Gi/s
BM_AbslBase64Decode/32768                                  23209 ns        23118 ns        30188 bytes_per_second=1.76019Gi/s
BM_AbslBase64Decode/262144                                185230 ns       184336 ns         3801 bytes_per_second=1.76593Gi/s
BM_AbslBase64Decode/1048576                               742042 ns       738630 ns          953 bytes_per_second=1.76284Gi/s
BM_LegacyBase64Decode/8                                     59.7 ns         59.5 ns     11753923 bytes_per_second=192.427Mi/s
BM_LegacyBase64Decode/64                                     431 ns          428 ns      1568483 bytes_per_second=195.924Mi/s
BM_LegacyBase64Decode/512                                   3209 ns         3196 ns       216661 bytes_per_second=204.13Mi/s
BM_LegacyBase64Decode/4096                                 25388 ns        25295 ns        27671 bytes_per_second=206.001Mi/s
BM_LegacyBase64Decode/32768                               203233 ns       202417 ns         3500 bytes_per_second=205.852Mi/s
BM_LegacyBase64Decode/262144                             1615847 ns      1608513 ns          437 bytes_per_second=207.232Mi/s
BM_LegacyBase64Decode/1048576                            6483992 ns      6455343 ns          109 bytes_per_second=206.548Mi/s
BM_AbslBase64BufferInstanceEncode/8                         35.3 ns         35.1 ns     19569259 bytes_per_second=217.294Mi/s
BM_AbslBase64BufferInstanceEncode/64                        70.5 ns         70.1 ns      9924410 bytes_per_second=870.793Mi/s
BM_AbslBase64BufferInstanceEncode/512                        269 ns          268 ns      2601418 bytes_per_second=1.77916Gi/s
BM_AbslBase64BufferInstanceEncode/4096                      1836 ns         1826 ns       383958 bytes_per_second=2.0896Gi/s
BM_AbslBase64BufferInstanceEncode/32768                    14150 ns        14086 ns        49749 bytes_per_second=2.16657Gi/s
BM_AbslBase64BufferInstanceEncode/262144                  111975 ns       111408 ns         6288 bytes_per_second=2.19141Gi/s
BM_AbslBase64BufferInstanceEncode/1048576                 446107 ns       443873 ns         1580 bytes_per_second=2.20009Gi/s
BM_LegacyBase64BufferInstanceEncode/8                       54.1 ns         53.9 ns     13240954 bytes_per_second=141.619Mi/s
BM_LegacyBase64BufferInstanceEncode/64                       335 ns          334 ns      2099860 bytes_per_second=183.004Mi/s
BM_LegacyBase64BufferInstanceEncode/512                     2355 ns         2346 ns       299256 bytes_per_second=208.131Mi/s
BM_LegacyBase64BufferInstanceEncode/4096                   18410 ns        18336 ns        38182 bytes_per_second=213.041Mi/s
BM_LegacyBase64BufferInstanceEncode/32768                 147610 ns       147060 ns         4762 bytes_per_second=212.498Mi/s
BM_LegacyBase64BufferInstanceEncode/262144               1179349 ns      1174398 ns          596 bytes_per_second=212.875Mi/s
BM_LegacyBase64BufferInstanceEncode/1048576              4742301 ns      4721762 ns          149 bytes_per_second=211.785Mi/s
BM_AbslBase64BufferInstanceEncodeMultiSlice/4096            1911 ns         1903 ns       369925 bytes_per_second=2.00461Gi/s
BM_AbslBase64BufferInstanceEncodeMultiSlice/32768          15227 ns        15165 ns        45886 bytes_per_second=2.01239Gi/s
BM_AbslBase64BufferInstanceEncodeMultiSlice/262144        124408 ns       123841 ns         5669 bytes_per_second=1.9714Gi/s
BM_AbslBase64BufferInstanceEncodeMultiSlice/1048576       499894 ns       497611 ns         1407 bytes_per_second=1.9625Gi/s
BM_LegacyBase64BufferInstanceEncodeMultiSlice/4096         18588 ns        18516 ns        37684 bytes_per_second=210.969Mi/s
BM_LegacyBase64BufferInstanceEncodeMultiSlice/32768       147629 ns       147052 ns         4634 bytes_per_second=212.51Mi/s
BM_LegacyBase64BufferInstanceEncodeMultiSlice/262144     1183149 ns      1177438 ns          594 bytes_per_second=212.325Mi/s
BM_LegacyBase64BufferInstanceEncodeMultiSlice/1048576    4724186 ns      4702789 ns          149 bytes_per_second=212.64Mi/s
BM_AbslBase64UrlEncode/8                                    19.9 ns         19.8 ns     35318734 bytes_per_second=385.207Mi/s
BM_AbslBase64UrlEncode/64                                   54.6 ns         54.4 ns     12961587 bytes_per_second=1.09557Gi/s
BM_AbslBase64UrlEncode/512                                   252 ns          252 ns      2792499 bytes_per_second=1.89564Gi/s
BM_AbslBase64UrlEncode/4096                                 1760 ns         1753 ns       399001 bytes_per_second=2.17577Gi/s
BM_AbslBase64UrlEncode/32768                               14038 ns        13986 ns        50051 bytes_per_second=2.18199Gi/s
BM_AbslBase64UrlEncode/262144                             110773 ns       110276 ns         6348 bytes_per_second=2.2139Gi/s
BM_AbslBase64UrlEncode/1048576                            442787 ns       440739 ns         1589 bytes_per_second=2.21574Gi/s
BM_LegacyBase64UrlEncode/8                                  41.5 ns         41.3 ns     16950384 bytes_per_second=184.617Mi/s
BM_LegacyBase64UrlEncode/64                                  297 ns          296 ns      2355033 bytes_per_second=206.009Mi/s
BM_LegacyBase64UrlEncode/512                                2136 ns         2128 ns       328029 bytes_per_second=229.444Mi/s
BM_LegacyBase64UrlEncode/4096                              16885 ns        16823 ns        41566 bytes_per_second=232.199Mi/s
BM_LegacyBase64UrlEncode/32768                            134893 ns       134392 ns         5212 bytes_per_second=232.529Mi/s
BM_LegacyBase64UrlEncode/262144                          1083154 ns      1078276 ns          650 bytes_per_second=231.852Mi/s
BM_LegacyBase64UrlEncode/1048576                         4356684 ns      4336895 ns          161 bytes_per_second=230.58Mi/s
BM_AbslBase64UrlDecode/8                                    25.3 ns         25.2 ns     27737820 bytes_per_second=415.973Mi/s
BM_AbslBase64UrlDecode/64                                   78.3 ns         78.0 ns      8924236 bytes_per_second=1.02696Gi/s
BM_AbslBase64UrlDecode/512                                   397 ns          395 ns      1767540 bytes_per_second=1.60968Gi/s
BM_AbslBase64UrlDecode/4096                                 2967 ns         2952 ns       242422 bytes_per_second=1.7233Gi/s
BM_AbslBase64UrlDecode/32768                               23697 ns        23524 ns        29233 bytes_per_second=1.72977Gi/s
BM_AbslBase64UrlDecode/262144                             185443 ns       184584 ns         3795 bytes_per_second=1.76354Gi/s
BM_AbslBase64UrlDecode/1048576                            746481 ns       742547 ns          953 bytes_per_second=1.75354Gi/s
BM_LegacyBase64UrlDecode/8                                  60.3 ns         60.0 ns     11947521 bytes_per_second=174.801Mi/s
BM_LegacyBase64UrlDecode/64                                  427 ns          425 ns      1636120 bytes_per_second=193.1Mi/s
BM_LegacyBase64UrlDecode/512                                3163 ns         3151 ns       219959 bytes_per_second=206.698Mi/s
BM_LegacyBase64UrlDecode/4096                              25836 ns        25510 ns        27914 bytes_per_second=204.193Mi/s
BM_LegacyBase64UrlDecode/32768                            203179 ns       202069 ns         3471 bytes_per_second=206.201Mi/s
BM_LegacyBase64UrlDecode/262144                          1621516 ns      1614225 ns          435 bytes_per_second=206.498Mi/s
BM_LegacyBase64UrlDecode/1048576                         6539881 ns      6506282 ns          107 bytes_per_second=204.93Mi/s

Risk Level: Medium (encoding/decoding is core utils logic and should preserve behavior if tests pass).

Testing: Existing unit tests for base64 encoding/decoding pass. Added differential fuzz testing (test/common/common/base64_fuzz_test.cc) and benchmarks (test/common/common/base64_speed_test.cc) comparing the new base64 implementation against the legacy implementation.

Docs Changes: None

Release Notes: Base64 implementation updated to use Abseil utilities. No behavioral changes expected.

Platform Specific Features: N/A

[Optional Fixes #Issue]: Fixes #12694.

[Optional Fixes commit #PR or SHA]: Related to PR #29360 and PR #12703.

Signed-off-by: Ethan Marantz <ebmarantz@gmail.com>
@repokitteh-read-only
Copy link
Copy Markdown

As a reminder, PRs marked as draft will not be automatically assigned reviewers,
or be handled by maintainer-oncall triage.

Please mark your PR as ready when you want it to be reviewed!

🐱

Caused by: #44915 was opened by ebm.

see: more, trace.

@ebm ebm temporarily deployed to external-contributors May 7, 2026 02:01 — with GitHub Actions Inactive
@jmarantz
Copy link
Copy Markdown
Contributor

jmarantz commented May 7, 2026

/retest

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates Envoy’s core Base64 utilities to use Abseil’s Base64 encode/decode helpers, aiming to preserve existing behavior while reducing custom implementation code (per #12694 and related PRs).

Changes:

  • Switched Base64/Base64Url encode/decode to absl::Base64* and absl::WebSafeBase64* functions.
  • Preserved Envoy-specific strict validation (e.g., rejecting invalid trailing bits) via explicit checks before decoding.
  • Adjusted no-padding behavior by stripping trailing = on encode and tolerating missing padding on decode.

Comment thread source/common/common/base64.cc Outdated
Comment thread source/common/common/base64.cc Outdated
…64 implementations.

Signed-off-by: Ethan Marantz <ebmarantz@gmail.com>
@ebm
Copy link
Copy Markdown
Contributor Author

ebm commented May 7, 2026

bazel test //test/common/common:base64_differential_fuzz_test runs the fuzzing test which asserts the old and new base64 implementations produce identical outputs.
bazel run -c opt //test/common/common:base64_speed_test runs the benchmark test which compares the speed of encoding/decoding at different byte mounts.

Running on my system, these are what the speed benchmarks show:

-------------------------------------------------------------------------------------------
Benchmark                                 Time             CPU   Iterations UserCounters...
-------------------------------------------------------------------------------------------
BM_AbslBase64Encode/8                  20.0 ns         19.9 ns     34555635 bytes_per_second=382.932Mi/s
BM_AbslBase64Encode/64                 56.3 ns         56.0 ns     12831974 bytes_per_second=1.0653Gi/s
BM_AbslBase64Encode/512                 254 ns          253 ns      2797137 bytes_per_second=1.88573Gi/s
BM_AbslBase64Encode/4096               1856 ns         1834 ns       383248 bytes_per_second=2.08003Gi/s
BM_AbslBase64Encode/32768             14341 ns        14243 ns        48645 bytes_per_second=2.14269Gi/s
BM_AbslBase64Encode/262144           113306 ns       112144 ns         5953 bytes_per_second=2.17703Gi/s
BM_AbslBase64Encode/1048576          456593 ns       451077 ns         1537 bytes_per_second=2.16496Gi/s
BM_LegacyBase64Encode/8                42.4 ns         42.1 ns     16265013 bytes_per_second=181.062Mi/s
BM_LegacyBase64Encode/64                317 ns          314 ns      2227138 bytes_per_second=194.263Mi/s
BM_LegacyBase64Encode/512              2272 ns         2242 ns       325554 bytes_per_second=217.833Mi/s
BM_LegacyBase64Encode/4096            19538 ns        19138 ns        40061 bytes_per_second=204.114Mi/s
BM_LegacyBase64Encode/32768          147673 ns       143314 ns         4863 bytes_per_second=218.053Mi/s
BM_LegacyBase64Encode/262144        1146333 ns      1134801 ns          579 bytes_per_second=220.303Mi/s
BM_LegacyBase64Encode/1048576       4723849 ns      4659935 ns          150 bytes_per_second=214.595Mi/s
BM_AbslBase64Decode/8                  33.6 ns         33.3 ns     21628962 bytes_per_second=343.608Mi/s
BM_AbslBase64Decode/64                 90.2 ns         89.6 ns      7645094 bytes_per_second=936.221Mi/s
BM_AbslBase64Decode/512                 420 ns          418 ns      1677261 bytes_per_second=1.52471Gi/s
BM_AbslBase64Decode/4096               3098 ns         3084 ns       228345 bytes_per_second=1.64983Gi/s
BM_AbslBase64Decode/32768             24430 ns        24329 ns        28856 bytes_per_second=1.67258Gi/s
BM_AbslBase64Decode/262144           198126 ns       197117 ns         3557 bytes_per_second=1.65142Gi/s
BM_AbslBase64Decode/1048576          819272 ns       814537 ns          843 bytes_per_second=1.59856Gi/s
BM_LegacyBase64Decode/8                62.8 ns         62.5 ns     11391560 bytes_per_second=183.199Mi/s
BM_LegacyBase64Decode/64                476 ns          474 ns      1465152 bytes_per_second=177.106Mi/s
BM_LegacyBase64Decode/512              3606 ns         3568 ns       190385 bytes_per_second=182.807Mi/s
BM_LegacyBase64Decode/4096            28209 ns        28001 ns        25335 bytes_per_second=186.097Mi/s
BM_LegacyBase64Decode/32768          222668 ns       221735 ns         3157 bytes_per_second=187.917Mi/s
BM_LegacyBase64Decode/262144        1790706 ns      1781131 ns          395 bytes_per_second=187.148Mi/s
BM_LegacyBase64Decode/1048576       7136770 ns      7104609 ns           98 bytes_per_second=187.672Mi/s
BM_AbslBase64UrlEncode/8               19.8 ns         19.7 ns     35520575 bytes_per_second=387.775Mi/s
BM_AbslBase64UrlEncode/64              54.9 ns         54.6 ns     12820113 bytes_per_second=1.09093Gi/s
BM_AbslBase64UrlEncode/512              250 ns          249 ns      2792840 bytes_per_second=1.91567Gi/s
BM_AbslBase64UrlEncode/4096            1824 ns         1801 ns       382329 bytes_per_second=2.11833Gi/s
BM_AbslBase64UrlEncode/32768          14114 ns        14051 ns        49879 bytes_per_second=2.17199Gi/s
BM_AbslBase64UrlEncode/262144        110826 ns       110293 ns         6301 bytes_per_second=2.21357Gi/s
BM_AbslBase64UrlEncode/1048576       449509 ns       445350 ns         1579 bytes_per_second=2.1928Gi/s
BM_LegacyBase64UrlEncode/8             40.4 ns         39.5 ns     18084066 bytes_per_second=193.075Mi/s
BM_LegacyBase64UrlEncode/64             316 ns          312 ns      2254509 bytes_per_second=195.413Mi/s
BM_LegacyBase64UrlEncode/512           2290 ns         2266 ns       316214 bytes_per_second=215.442Mi/s
BM_LegacyBase64UrlEncode/4096         18420 ns        18085 ns        36982 bytes_per_second=215.999Mi/s
BM_LegacyBase64UrlEncode/32768       141624 ns       140161 ns         4949 bytes_per_second=222.958Mi/s
BM_LegacyBase64UrlEncode/262144     1205516 ns      1180459 ns          625 bytes_per_second=211.782Mi/s
BM_LegacyBase64UrlEncode/1048576    4616359 ns      4591294 ns          153 bytes_per_second=217.804Mi/s
BM_AbslBase64UrlDecode/8               31.7 ns         31.4 ns     22154215 bytes_per_second=333.781Mi/s
BM_AbslBase64UrlDecode/64              94.5 ns         93.8 ns      7348945 bytes_per_second=874.68Mi/s
BM_AbslBase64UrlDecode/512              458 ns          453 ns      1580037 bytes_per_second=1.40527Gi/s
BM_AbslBase64UrlDecode/4096            3252 ns         3224 ns       217977 bytes_per_second=1.5776Gi/s
BM_AbslBase64UrlDecode/32768          25517 ns        25355 ns        28234 bytes_per_second=1.60482Gi/s
BM_AbslBase64UrlDecode/262144        202193 ns       200960 ns         3421 bytes_per_second=1.61983Gi/s
BM_AbslBase64UrlDecode/1048576       832491 ns       824100 ns          891 bytes_per_second=1.58001Gi/s
BM_LegacyBase64UrlDecode/8             60.4 ns         59.6 ns     11777184 bytes_per_second=175.948Mi/s
BM_LegacyBase64UrlDecode/64             457 ns          450 ns      1485901 bytes_per_second=182.246Mi/s
BM_LegacyBase64UrlDecode/512           3554 ns         3482 ns       200294 bytes_per_second=187.06Mi/s
BM_LegacyBase64UrlDecode/4096         27639 ns        27360 ns        25995 bytes_per_second=190.386Mi/s
BM_LegacyBase64UrlDecode/32768       225326 ns       221805 ns         3194 bytes_per_second=187.854Mi/s
BM_LegacyBase64UrlDecode/262144     1784478 ns      1772074 ns          399 bytes_per_second=188.104Mi/s
BM_LegacyBase64UrlDecode/1048576    6917716 ns      6841059 ns          100 bytes_per_second=194.902Mi/s

ebm and others added 2 commits May 8, 2026 23:39
Signed-off-by: Ethan Marantz <ebmarantz@gmail.com>
…zing tests for BufferInstance. Added failed test to the unit test.

Signed-off-by: ethan <ethan@debian-server.home.arpa>
Copy link
Copy Markdown
Contributor

@jmarantz jmarantz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you put the performance data (including the buffer encoding speed) into the PR description?

This is looking great!

Comment thread source/common/common/base64.cc Outdated
Comment thread source/common/common/base64.cc Outdated
Comment thread source/common/common/base64.cc Outdated
Comment thread source/common/common/base64.cc Outdated
Comment thread source/common/common/base64.cc Outdated
Comment thread source/common/common/base64.cc Outdated
Comment thread test/common/common/base64_legacy.cc
Comment thread test/common/common/base64_legacy.h
@ebm ebm force-pushed the change-base64-implementation branch 2 times, most recently from 4eea06f to 56bd9b3 Compare May 10, 2026 06:01
Signed-off-by: Ethan Marantz <ebmarantz@gmail.com>
Signed-off-by: ethan <ethan@debian-server.home.arpa>
@ebm ebm force-pushed the change-base64-implementation branch from 56bd9b3 to d17da1d Compare May 10, 2026 07:15
@jmarantz
Copy link
Copy Markdown
Contributor

/gemini review

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the Base64 and Base64Url encoding and decoding logic to utilize Abseil's escaping utilities, simplifying the implementation. It also introduces comprehensive differential fuzzing and performance benchmarks to ensure functional parity with the legacy implementation. Feedback was provided regarding a performance regression in the Base64::encode method for Buffer::Instance, where an extra intermediate allocation was introduced; an optimization was suggested to handle contiguous buffers more efficiently.

Comment thread source/common/common/base64.cc
…check to skip intermediate copy when buffer contains one slice.

Signed-off-by: Ethan Marantz <ebmarantz@gmail.com>
@ebm
Copy link
Copy Markdown
Contributor Author

ebm commented May 10, 2026

With this change, single sliced Buffer::Instance get encoded 5-15% faster than the old code. Multi sliced Buffer::Instance show no change which is expected.

Signed-off-by: Ethan Marantz <ebmarantz@gmail.com>
@ebm ebm marked this pull request as ready for review May 11, 2026 03:02
@ebm
Copy link
Copy Markdown
Contributor Author

ebm commented May 11, 2026

@yanavlasov

@kyessenov
Copy link
Copy Markdown
Contributor

Please review as a senior maintainer @RyanTheOptimist .
/wait

Copy link
Copy Markdown
Contributor

@RyanTheOptimist RyanTheOptimist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks awesome. Thanks for doing this!

@@ -0,0 +1 @@
%
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, why does this change?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I copied it over from the existing base64 fuzz test's corpus (test/common/common/base64_corpus). I kept base64_corpus and base64_differential_corpus as separate directories to signify that they are different tests with different discovered inputs.

@RyanTheOptimist RyanTheOptimist merged commit 6f7f7f7 into envoyproxy:main May 21, 2026
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

replace envoy base64 code with abseil library code

5 participants