Use UTF-8 for MySQL JSON binlog value decoding - #39140
Merged
Merged
Conversation
terrymanu
previously approved these changes
Jul 17, 2026
terrymanu
left a comment
Member
There was a problem hiding this comment.
Summary
Review Result: Mergeable
Reason: The root cause is fixed at both byte-to-string conversion sites, and the regression test exercises UTF-8 key and value decoding with correct byte lengths. Scoped verification also passes with a US-ASCII default charset. This is a code-scope result only; CI was not reviewed by request.
Evidence
- Issue #39139 identifies default-charset decoding in both object keys and string values. MySQL’s binary JSON specification defines both keys and strings as utf8mb4 data.
MySQLJsonValueDecoder.javanow decodes keys withStandardCharsets.UTF_8; the shared string decoder does the same for top-level, object, array, and nested string values.MySQLJsonValueDecoderTest.javacovers a multibyte key and value. Its fixtures explicitly encode key bytes and string bytes and lengths using UTF-8.- ASCII behavior, public APIs, configuration, dependencies, lifecycle, and control flow are unchanged. The explicit charset selection adds no material performance or security risk.
- The release-note entry accurately describes the user-visible fix.
Review Details
- Review Focus: Code Correctness Review. CI not reviewed by request.
- Reviewed Scope: All three GitHub-listed files in the MySQL protocol module and release notes at head
a155673ac878bce61a99ec6ba12ba9d32b186dc3; base SHAe249deba59d3583dc5464292672c5c73a92c2832; local merge-baseaf91b1c34cb041466449eec61ae711ec1f245d92. The local triple-dot file list matched GitHub/pulls/39140/files. - Not Reviewed Scope: GitHub Actions/check-runs, full-repository build, E2E scenarios, and unrelated modules.
- Verification:
- Focused
MySQLJsonValueDecoderTestrun: exit0; 42 tests passed. - The same test with
-DargLine=-Dfile.encoding=COMPAT: exit0; 42 tests passed, with the forked JVM reportingfile.encoding=US-ASCII. - Scoped
spotless:check checkstyle:check: exit0; 0 Spotless changes required and 0 Checkstyle violations. - Logs:
<system-temp>/shardingsphere-pr39140-{test,compat-test,style}.*.log. - GitHub PR, file, commit, comment, and review endpoints were accessible; no public-evidence gap remains.
- Focused
- Release Note / User Docs: Release note present and verified. Additional user documentation is not required because the fix changes no configuration, API/SPI, migration, or operational procedure.
terrymanu
approved these changes
Jul 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #39139.
Changes proposed in this pull request:
decodeString) and object keys (decodeKeyEntry) withStandardCharsets.UTF_8instead ofnew String(byte[]), which relies on the JVM default charset and corrupts non-ASCII data on non-UTF-8-default JVMs.MySQLPacketPayload, which always reads bytes with an explicit charset.MySQLJsonValueDecoderTestcase decoding a multibyte UTF-8 key and value ({"名前":"café"}) from explicit UTF-8 bytes, asserting the utf8mb4 contract. On a UTF-8-default JVM it acts as a regression/contract guard; it also catches corruption on non-UTF-8-default JVMs.Before committing this PR, I'm sure that I have checked the following options:
./mvnw clean install -B -T1C -Dmaven.javadoc.skip -Dmaven.jacoco.skip -e.