Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/json: decoding JSON into anonymous struct works in go 1.8.7 but fails silently in go 1.9.4 #23861

Closed
nkhumphreys opened this issue Feb 15, 2018 · 9 comments

Comments

Projects
None yet
6 participants
@nkhumphreys
Copy link

commented Feb 15, 2018

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

1.8.7 and 1.9.4

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOOS="darwin"

What did you do?

When decoding json into an anonymous structure, the data is present when using an app built with go version 1.8.7 but empty when using go 1.9.7

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

https://play.golang.org/p/vPsyOX7wBea

Here is an example of how I discovered this using the req.Body, details in the README show the difference between the go 1.8 and 1.9 output. With help from eric_hill on IRC #go-nuts we narrowed it down the the above play link.

https://github.com/nkhumphreys/goreqbody

What did you expect to see?

localStr should be populated with the data decoded from the JSON

What did you see instead?

localStr was empty

@bradfitz

This comment has been minimized.

Copy link
Member

commented Feb 15, 2018

With help from eric_hill on IRC #go-nuts we narrowed it down the the above play link.

You two both forgot to check for errors.

That might not necessarily matter, but it's a distracting bug report, since everybody will immediately assume the answer is in the ignored error.

@nkhumphreys

This comment has been minimized.

Copy link
Author

commented Feb 15, 2018

I can update the play link and check the error if you like, but there is not one. :)

https://play.golang.org/p/3ZxteJYqZMp

@bradfitz

This comment has been minimized.

Copy link
Member

commented Feb 15, 2018

Well, we can investigate, but there's no fixing Go 1.9 at this point, and Go 1.10 is out, like, today, so probably not changing there either.

I see nothing in the Go 1.9 release notes about json or reflect.

Please test new Go releases early & often! :)

@bradfitz bradfitz changed the title decoding JSON into anonymous struct works in go 1.8.7 but fails silently in go 1.9.4 encoding/json: decoding JSON into anonymous struct works in go 1.8.7 but fails silently in go 1.9.4 Feb 15, 2018

@bradfitz bradfitz added this to the Go1.11 milestone Feb 15, 2018

@nkhumphreys

This comment has been minimized.

Copy link
Author

commented Feb 15, 2018

Just an FYI: The same behaviour as go 1.9 is exhibited in go 1.10rc2

@pciet

This comment has been minimized.

Copy link
Contributor

commented Feb 16, 2018

An exported field works: https://play.golang.org/p/0kdRTJY_Xp5

Capitalizing the type works: https://play.golang.org/p/4szuu5Zydg3

From https://golang.org/pkg/encoding/json/#Marshal:

Anonymous struct fields are usually marshaled as if their inner exported fields were fields in the outer struct, subject to the usual Go visibility rules amended as described in the next paragraph.

@0xmohit

This comment has been minimized.

Copy link
Contributor

commented Feb 16, 2018

The behavior changed in b817359.

@bradfitz

This comment has been minimized.

Copy link
Member

commented Feb 16, 2018

Hey @dsnet , is that expected? A pointer to a string is considered unexported?

@dsnet

This comment has been minimized.

Copy link
Member

commented Feb 16, 2018

Yes, this is expected behavior.

According to the language spec:

An identifier may be exported to permit access to it from another package. An identifier is exported if both:

  1. the first character of the identifier's name is a Unicode upper case letter (Unicode class "Lu"); and
  2. the identifier is declared in the package block or it is a field name or method name.

All other identifiers are not exported.

Thus, embedding a builtin type results in a field name that is not exported (since all builtin have lowercase names). Since this is unexported, the json package documents that it does not handle this.

The json rule you quote does not apply here:

Anonymous struct fields are usually marshaled as if their inner exported fields were fields in the outer struct, subject to the usual Go visibility rules amended as described in the next paragraph.

The example you provided has an anonymous string, not a struct.

@dsnet dsnet closed this Feb 16, 2018

@dsnet

This comment has been minimized.

Copy link
Member

commented Feb 16, 2018

Furthermore, in Go1.10 and on, it wouldn't be possible for json to even handle this without using unsafe since reflect reports this field as unexported.

The reason why reflect on Go1.9 and below could actually mutate this field was due to a compiler bug, fixed by @mdempsky in CL/60410.

@golang golang locked and limited conversation to collaborators Feb 16, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.