Base64 update with absl::Base64#12703
Conversation
Signed-off-by: pingsun <pingsun@google.com>
Signed-off-by: pingsun <pingsun@google.com>
Signed-off-by: pingsun <pingsun@google.com>
|
If we can't completely replace and remove Base64::encode/decode, it might not worth for the replacement. Can you just replace the implementation of |
Sure, Thanks for the great idea. I previously thought that we will finally remove the lib of the current Base64 so I just updated the code where it has been used directly. I think your idea is a even better, I will try to replace the implementation of Base64. |
|
I have some trouble implementing the The expected value for decode("Zh==") in the test is EMPTY_STRING. |
|
The current implementation: std::string Base64::decode(const std::string& input) {
if (input.length() % 4) {
return EMPTY_STRING;
}
return decodeWithoutPadding(input);
}
std::string Base64::decodeWithoutPadding(absl::string_view input) {
if (input.empty()) {
return EMPTY_STRING;
}
std::string decoded;
bool success = absl::Base64Unescape(input, &decoded);
if (success) {
return decoded;
} else {
return EMPTY_STRING;
}
}
std::string Base64Url::decode(const std::string& input) {
if (input.empty()) {
return EMPTY_STRING;
}
std::string decoded;
bool success = absl::WebSafeBase64Unescape(input, &decoded);
if (success) {
return decoded;
} else {
return EMPTY_STRING;
}
} |
|
Hi @lizan, I think it might not be possible to remove the original implementation of Base64 Encode/Decode, especially considering the usage of |
|
@ASOPVII if we can't replace it yeah I would say let's just close this for now? |
Sure, I agree~ |
Commit Message: replaces base64 implementation with Abseil's base64 implementation. Additional Description: Replaces the existing base64 implementation with Abseil’s base64 one. This change maintains identical encoding/decoding behavior. Benchmarks show ~2x at 8 bytes ~5x at 64 bytes ~8-10x at 512 bytes and above <details> <summary>Full Benchmark</summary> <pre> ---------------------------------------------------------------------------------------------------------------- Benchmark Time CPU Iterations UserCounters... ---------------------------------------------------------------------------------------------------------------- BM_AbslBase64Encode/8 20.5 ns 20.4 ns 33893373 bytes_per_second=374.723Mi/s BM_AbslBase64Encode/64 56.1 ns 55.8 ns 12443880 bytes_per_second=1.06846Gi/s BM_AbslBase64Encode/512 248 ns 247 ns 2822896 bytes_per_second=1.93045Gi/s BM_AbslBase64Encode/4096 1777 ns 1770 ns 395332 bytes_per_second=2.15477Gi/s BM_AbslBase64Encode/32768 14170 ns 14108 ns 50054 bytes_per_second=2.16317Gi/s BM_AbslBase64Encode/262144 110850 ns 110328 ns 6340 bytes_per_second=2.21287Gi/s BM_AbslBase64Encode/1048576 442523 ns 440467 ns 1589 bytes_per_second=2.21711Gi/s BM_LegacyBase64Encode/8 42.7 ns 42.5 ns 16462454 bytes_per_second=179.434Mi/s BM_LegacyBase64Encode/64 310 ns 308 ns 2324094 bytes_per_second=197.93Mi/s BM_LegacyBase64Encode/512 2202 ns 2193 ns 322767 bytes_per_second=222.7Mi/s BM_LegacyBase64Encode/4096 17763 ns 17687 ns 40018 bytes_per_second=220.852Mi/s BM_LegacyBase64Encode/32768 141403 ns 140443 ns 5038 bytes_per_second=222.51Mi/s BM_LegacyBase64Encode/262144 1127638 ns 1121530 ns 633 bytes_per_second=222.91Mi/s BM_LegacyBase64Encode/1048576 4445197 ns 4424678 ns 155 bytes_per_second=226.005Mi/s BM_AbslBase64Decode/8 25.9 ns 25.8 ns 26666600 bytes_per_second=443.357Mi/s BM_AbslBase64Decode/64 80.2 ns 79.8 ns 8809031 bytes_per_second=1.02672Gi/s BM_AbslBase64Decode/512 398 ns 396 ns 1774014 bytes_per_second=1.607Gi/s BM_AbslBase64Decode/4096 3000 ns 2985 ns 237276 bytes_per_second=1.70479Gi/s BM_AbslBase64Decode/32768 23209 ns 23118 ns 30188 bytes_per_second=1.76019Gi/s BM_AbslBase64Decode/262144 185230 ns 184336 ns 3801 bytes_per_second=1.76593Gi/s BM_AbslBase64Decode/1048576 742042 ns 738630 ns 953 bytes_per_second=1.76284Gi/s BM_LegacyBase64Decode/8 59.7 ns 59.5 ns 11753923 bytes_per_second=192.427Mi/s BM_LegacyBase64Decode/64 431 ns 428 ns 1568483 bytes_per_second=195.924Mi/s BM_LegacyBase64Decode/512 3209 ns 3196 ns 216661 bytes_per_second=204.13Mi/s BM_LegacyBase64Decode/4096 25388 ns 25295 ns 27671 bytes_per_second=206.001Mi/s BM_LegacyBase64Decode/32768 203233 ns 202417 ns 3500 bytes_per_second=205.852Mi/s BM_LegacyBase64Decode/262144 1615847 ns 1608513 ns 437 bytes_per_second=207.232Mi/s BM_LegacyBase64Decode/1048576 6483992 ns 6455343 ns 109 bytes_per_second=206.548Mi/s BM_AbslBase64BufferInstanceEncode/8 35.3 ns 35.1 ns 19569259 bytes_per_second=217.294Mi/s BM_AbslBase64BufferInstanceEncode/64 70.5 ns 70.1 ns 9924410 bytes_per_second=870.793Mi/s BM_AbslBase64BufferInstanceEncode/512 269 ns 268 ns 2601418 bytes_per_second=1.77916Gi/s BM_AbslBase64BufferInstanceEncode/4096 1836 ns 1826 ns 383958 bytes_per_second=2.0896Gi/s BM_AbslBase64BufferInstanceEncode/32768 14150 ns 14086 ns 49749 bytes_per_second=2.16657Gi/s BM_AbslBase64BufferInstanceEncode/262144 111975 ns 111408 ns 6288 bytes_per_second=2.19141Gi/s BM_AbslBase64BufferInstanceEncode/1048576 446107 ns 443873 ns 1580 bytes_per_second=2.20009Gi/s BM_LegacyBase64BufferInstanceEncode/8 54.1 ns 53.9 ns 13240954 bytes_per_second=141.619Mi/s BM_LegacyBase64BufferInstanceEncode/64 335 ns 334 ns 2099860 bytes_per_second=183.004Mi/s BM_LegacyBase64BufferInstanceEncode/512 2355 ns 2346 ns 299256 bytes_per_second=208.131Mi/s BM_LegacyBase64BufferInstanceEncode/4096 18410 ns 18336 ns 38182 bytes_per_second=213.041Mi/s BM_LegacyBase64BufferInstanceEncode/32768 147610 ns 147060 ns 4762 bytes_per_second=212.498Mi/s BM_LegacyBase64BufferInstanceEncode/262144 1179349 ns 1174398 ns 596 bytes_per_second=212.875Mi/s BM_LegacyBase64BufferInstanceEncode/1048576 4742301 ns 4721762 ns 149 bytes_per_second=211.785Mi/s BM_AbslBase64BufferInstanceEncodeMultiSlice/4096 1911 ns 1903 ns 369925 bytes_per_second=2.00461Gi/s BM_AbslBase64BufferInstanceEncodeMultiSlice/32768 15227 ns 15165 ns 45886 bytes_per_second=2.01239Gi/s BM_AbslBase64BufferInstanceEncodeMultiSlice/262144 124408 ns 123841 ns 5669 bytes_per_second=1.9714Gi/s BM_AbslBase64BufferInstanceEncodeMultiSlice/1048576 499894 ns 497611 ns 1407 bytes_per_second=1.9625Gi/s BM_LegacyBase64BufferInstanceEncodeMultiSlice/4096 18588 ns 18516 ns 37684 bytes_per_second=210.969Mi/s BM_LegacyBase64BufferInstanceEncodeMultiSlice/32768 147629 ns 147052 ns 4634 bytes_per_second=212.51Mi/s BM_LegacyBase64BufferInstanceEncodeMultiSlice/262144 1183149 ns 1177438 ns 594 bytes_per_second=212.325Mi/s BM_LegacyBase64BufferInstanceEncodeMultiSlice/1048576 4724186 ns 4702789 ns 149 bytes_per_second=212.64Mi/s BM_AbslBase64UrlEncode/8 19.9 ns 19.8 ns 35318734 bytes_per_second=385.207Mi/s BM_AbslBase64UrlEncode/64 54.6 ns 54.4 ns 12961587 bytes_per_second=1.09557Gi/s BM_AbslBase64UrlEncode/512 252 ns 252 ns 2792499 bytes_per_second=1.89564Gi/s BM_AbslBase64UrlEncode/4096 1760 ns 1753 ns 399001 bytes_per_second=2.17577Gi/s BM_AbslBase64UrlEncode/32768 14038 ns 13986 ns 50051 bytes_per_second=2.18199Gi/s BM_AbslBase64UrlEncode/262144 110773 ns 110276 ns 6348 bytes_per_second=2.2139Gi/s BM_AbslBase64UrlEncode/1048576 442787 ns 440739 ns 1589 bytes_per_second=2.21574Gi/s BM_LegacyBase64UrlEncode/8 41.5 ns 41.3 ns 16950384 bytes_per_second=184.617Mi/s BM_LegacyBase64UrlEncode/64 297 ns 296 ns 2355033 bytes_per_second=206.009Mi/s BM_LegacyBase64UrlEncode/512 2136 ns 2128 ns 328029 bytes_per_second=229.444Mi/s BM_LegacyBase64UrlEncode/4096 16885 ns 16823 ns 41566 bytes_per_second=232.199Mi/s BM_LegacyBase64UrlEncode/32768 134893 ns 134392 ns 5212 bytes_per_second=232.529Mi/s BM_LegacyBase64UrlEncode/262144 1083154 ns 1078276 ns 650 bytes_per_second=231.852Mi/s BM_LegacyBase64UrlEncode/1048576 4356684 ns 4336895 ns 161 bytes_per_second=230.58Mi/s BM_AbslBase64UrlDecode/8 25.3 ns 25.2 ns 27737820 bytes_per_second=415.973Mi/s BM_AbslBase64UrlDecode/64 78.3 ns 78.0 ns 8924236 bytes_per_second=1.02696Gi/s BM_AbslBase64UrlDecode/512 397 ns 395 ns 1767540 bytes_per_second=1.60968Gi/s BM_AbslBase64UrlDecode/4096 2967 ns 2952 ns 242422 bytes_per_second=1.7233Gi/s BM_AbslBase64UrlDecode/32768 23697 ns 23524 ns 29233 bytes_per_second=1.72977Gi/s BM_AbslBase64UrlDecode/262144 185443 ns 184584 ns 3795 bytes_per_second=1.76354Gi/s BM_AbslBase64UrlDecode/1048576 746481 ns 742547 ns 953 bytes_per_second=1.75354Gi/s BM_LegacyBase64UrlDecode/8 60.3 ns 60.0 ns 11947521 bytes_per_second=174.801Mi/s BM_LegacyBase64UrlDecode/64 427 ns 425 ns 1636120 bytes_per_second=193.1Mi/s BM_LegacyBase64UrlDecode/512 3163 ns 3151 ns 219959 bytes_per_second=206.698Mi/s BM_LegacyBase64UrlDecode/4096 25836 ns 25510 ns 27914 bytes_per_second=204.193Mi/s BM_LegacyBase64UrlDecode/32768 203179 ns 202069 ns 3471 bytes_per_second=206.201Mi/s BM_LegacyBase64UrlDecode/262144 1621516 ns 1614225 ns 435 bytes_per_second=206.498Mi/s BM_LegacyBase64UrlDecode/1048576 6539881 ns 6506282 ns 107 bytes_per_second=204.93Mi/s </pre> </details> Risk Level: Medium (encoding/decoding is core utils logic and should preserve behavior if tests pass). Testing: Existing unit tests for base64 encoding/decoding pass. Added differential fuzz testing (test/common/common/base64_fuzz_test.cc) and benchmarks (test/common/common/base64_speed_test.cc) comparing the new base64 implementation against the legacy implementation. Docs Changes: None Release Notes: Base64 implementation updated to use Abseil utilities. No behavioral changes expected. Platform Specific Features: N/A [Optional Fixes #Issue]: Fixes #12694. [Optional Fixes commit #PR or SHA]: Related to PR #29360 and PR #12703. --------- Signed-off-by: Ethan Marantz <ebmarantz@gmail.com> Signed-off-by: ethan <ethan@debian-server.home.arpa> Co-authored-by: ethan <ethan@debian-server.home.arpa>
Related Issue: #12694
Description:
This pull request helps to replace the current
Base64::encode/decodewithabsl::Base64.The current code base provide base64 encoding and decoding with or without padding. Abseil has supported to do that. To be more specific:
Base64::encode(src_data, src_len, padding=true)can be replaced withabsl::Base64Escape(std::string src).Base64::encode(src_data, src_len, padding=false)can be replaced withabsl::Base64WebSafeEscape(std::string src).Base64Url::encode(src_data, src_len)can be replaced withabsl::Base64WebSafeEscape(std::string src).Base64::decode(std::string src)can be replaced withabsl::Base64Unescape(src, *dst).Base64::decodeWithoutPadding(std::string src)can be replaced withabsl::Base64WebSafeUnescape(src, *dst).Base64Url::decode(std::string src)can be replaced withabsl::Base64WebSafeEscape(std::string src).The doc of absl::Base64: https://github.com/abseil/abseil-cpp/blob/master/absl/strings/escaping.h