Fix JsonReader advancing before throwing exception due to malformed JSON #1743
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously JsonReader.doPeek() would have already advanced or modified the
state of the reader when malformed JSON was encountered and an exception
was thrown. Therefore repeatedly calling methods which peek would result
in inconsistent exceptions and could even make the JSON "valid" again by
simply advancing past the malformed JSON.
Now the state of the reader is only modified when valid JSON is encountered.
Note that there are a few remaining cases where this issue still occurs,
which will be fixed in subsequent commits.
This change required adding more JsconScope constants because if there is
enough space (or a long comment) between tokens the skipped content could
not have been stored in the buffer, e.g. `{"a": ... :1}`.
Here a value is expected after the `:`. However there are a lot of
whitespaces followed by a second `:` (i.e. malformed JSON).
With the previous JsonScopes it would not have been possible to represent
this state correctly. Therefore a subsequent peek would have consumed the
second `:` making the JSON "valid" again.
…scape Previously JsonReader.nextString() and skipValue() consumed the '\' of an escape sequence before throwing an exception for invalid escape sequences. Therefore a subsequent method call would have read a "valid" string. Now the stream position is only advanced when the escape sequence is valid.
yuedaxia76
pushed a commit
to yuedaxia76/gson
that referenced
this pull request
Jul 27, 2020
For a malformed unicode escape sequence the exception message thrown be the method erroneously only included the first 4 chars which is `\uXX`, missing the last 2 hex chars.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #1735
Previously the JsonReader methods
doPeek()(internal method called for example bypeek()),nextString()andskipValue()advanced in the stream before throwing an exception due to malformed JSON.This allowed them to "skip" malformed JSON making the JSON valid again for subsequent calls, which is likely undesired.
This pull request changes it so that if malformed JSON is encountered all subsequent method calls will throw the same exception.
Additionally it introduces more
JsonScopeconstants because it was not possible with the existing ones to handle cases where a variable amount of characters is skipped, e.g. whitespaces between:after an object property name and before the property value.