Malformed JSON command output when non-ASCII strings are present #4634
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is a simple safety check. Since we default to true it is currently opt-in. Backport: giant, firefly Signed-off-by: Sage Weil <sage@redhat.com> (cherry picked from commit 6b982e4)
When the python CLI is given non-ASCII characters, it converts them to \uHHHH escapes in JSON. json_spirit parses these internally into 16 bit characters, which could only work if json_spirit were built to use std::wstring, which it isn't; it's using std::string, so the high byte ends up being zero'd, leaving the low byte which is effectively garbage. This hack^H^H^H^H change makes json_spirit convert to utf8 internally instead, which can be stored just fine inside a std::string. Note that this implementation still assumes \uHHHH escapes are four hex digits, so it'll only cope with characters in the Basic Multilingual Plane. Still, that's rather a lot more characters than it could cope with before ;) (For characters outside the BMP, Python seems to generate escapes in the form \uHHHHHHHH, i.e. 8 hex digits, which the current implementation doesn't expect to see) Fixes: ceph#7387 Signed-off-by: Tim Serong <tserong@suse.com> (cherry picked from commit 8add15b)
|
I know backport PRs are supposed to contain a single commit only, but in this case 5369e10 (8add15 in master) depends on 47eecaf (6b982e in master), and the latter hasn't been backported yet and I couldn't find a tracker issue for it, either. |
Fixes: ceph#11574 Signed-off-by: Kefu Chai <kchai@redhat.com> (cherry picked from commit 6b68b27)
|
This last commit 010065e (6b68b2 in master) fixes a FTBFS introduced by the fix for http://tracker.ceph.com/issues/7387 There is a separate tracker issue http://tracker.ceph.com/issues/11574 open for it, but since it depends on this commit, it makes sense to combine. |
|
Looking at this in the light of morning I see that Sage's and Tim's commits are unrelated and really should be kept separate. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
http://tracker.ceph.com/issues/7387