Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.
Sign upencoding/json: the Decoder.Decode API lends itself to misuse #36225
Comments
This comment has been minimized.
This comment has been minimized.
|
I don't think that's a bug, or misuse, I actually depend on it in a project where there's a json header then there's other data. This change would break a lot of working code. |
This comment has been minimized.
This comment has been minimized.
|
The only The call to the |
This comment has been minimized.
This comment has been minimized.
|
A |
This comment has been minimized.
This comment has been minimized.
|
@OneOfOne, I'm not proposing that we change the From my observation, the source of misuse is due to:
The first issue could be alleviated by adding: func UnmarshalReader(r io.Reader, v interface{}) errorwhere The second issue could be alleviated by having an options pattern that doesn't require the creation of a streaming @bcmills, a
|
This comment has been minimized.
This comment has been minimized.
That would work too. So that gives three patterns that should suppress the warning:
(1) and (2) suppress the warning if the user has read enough of the input to detect either EOF or an earlier syntax error. (3) suppresses the warning if a non-JSON blob follows the JSON value. |
This comment has been minimized.
This comment has been minimized.
Or a Decoder option to mark it as being single-object only. That way fixing existing instances of this problem (of which there are many, including all over my own code) involves just adding a line, instead of refactoring. |
This comment has been minimized.
This comment has been minimized.
I like how concise using a decoder often is:
It would be nice if the fix didn't require breaking up the line. I like how @dsnet's UnmarshalReader suggestion offers something similarly concise:
We could also use an alternative Decode method:
|
This comment has been minimized.
This comment has been minimized.
To give further evidence that this is a common issue, I'm one of the I'm personally in favour of I think a vet check would also be useful, because many existing users might not realise that this common json decoding pattern is buggy. Coupled with |
This comment has been minimized.
This comment has been minimized.
|
Maybe a method like |
This comment has been minimized.
This comment has been minimized.
|
Shouldn't this be a concrete proposal (with possible variations)? The issue description doesn't seem actionable. |
This comment has been minimized.
This comment has been minimized.
|
@networkimprov usually, a proposal brings forward a specific solution to be implemented. It seems like the main point of this issue was to bring up the problem, so the proposal could come later if needed. |
This comment has been minimized.
This comment has been minimized.
|
Time and again, I've seen issues closed and further discussion directed to golang-nuts/dev because the item isn't actionable. |
This comment has been minimized.
This comment has been minimized.
|
I agree with a new method. It re-use options for |
This comment has been minimized.
This comment has been minimized.
|
I suggest the name DecodeFull, it is more consistent with ReadFull. |
This comment has been minimized.
This comment has been minimized.
|
I agree that |
This comment has been minimized.
This comment has been minimized.
|
Remove this entirely, don't let close-to–duplicate functionality lying around. Especially one that invites misuse. |
This comment has been minimized.
This comment has been minimized.
|
I don't want to bikeshed names too hard, but my suggestion would be (Also, this is a function name in C#'s LINQ, where it signifies that an IEnumerable contains a single thing, and throws if not, which feels similar.) |
This comment has been minimized.
This comment has been minimized.
|
Removing Let's avoid bikeshedding on names; it brings unnecessary noise to the discussion. Bikeshedding can occur in future CLs that implement this assuming we should do anything about this. To summarize the discussion so far, there seems to be 3 (non-mutually exclusive) approaches:
An issue with approaches 1 or 2 is that it assumes that Personally, I like approach 2 and 3. |
This comment has been minimized.
This comment has been minimized.
|
I think the io.EOF issue is a showstopper, unless a new function/method takes a size argument, which could be zero or -1 for read-to-eof. Maybe add the list of solutions to the issue text, and retitle it "proposal: ..." ? I think that places it on the proposal review committee's agenda... |
This comment has been minimized.
This comment has been minimized.
If the code doing the decoding knows the length of the data, then they could use an I suspect, though, that the more common case is that the code doesn't know, and doesn't care: It just wants to decode until the first json object is done. If it so happens that you can detect that there's extra (non-whitespace?) data, because it came along with a read you issued anyway, then sure, return an error. But I think most users would be perfectly happy to be a bit sloppy here and have only best-effort "extra data" error detection, as evidenced by the multitude of us happily, obliviously using json.Decoder for years and years. So I guess I would advocate "fixing" this problem by defining it away. FWIW, I am also a fan of approaches 2 and 3. |
I'm observing the existence of several production servers that are buggy because the
json.Decoder.DecodeAPI lends itself to misuse.Consider the following:
json.NewDecoderis often used because the user has anio.Readeron hand or wants to configure some of the options onjson.Decoder. However, the common case is that the user only wants to decode a single JSON value. As it stands the API does not make the common case easy sinceDecodeis designed with the assumption that the user will continue to decode more JSON values, which is rarely the case.The code above executes just fine without reporting an error and silently allows the decoder to silently accept bad input without reporting any problems.