reflect: resolve incompatibility between gc and gccgo for unexported embedded structs #12367
Both are not correct w.r.t. the way Go works: an unexported embedded struct should not be settable as a whole, however, exported fields and methods of unexported embedded structs may be promoted and be visible.
Make unexported embedded structs unsettable, but allow the reflect library to access exported fields and methods of such structs. Such fields could also be modified.
Rationale: this prevents blanket changes to values in structs that are generally not settable, while allowing access to fields that might be promoted.
The approach might still allow access to fields that are not accessible in the language, namely exported fields and methods that are not promoted because they are masked by or conflict with fields or methods of the same name. However, access to such fields is also necessary. Packages like encoding/xml and encoding/json allow users to map fields to alternative names using tags effectively breaking ties and making fields visible again. Disallowing access to blocked fields now would be too much of a disruptive change and break the Go 1 compatibility promise.
This will break packages like xml and json, but only minimally. The typical check for whether a field is exported, f.PkgPath != "", would have to be amended to f.PkgPath != "" && !f.Anonymous.
adopt gccgo behavior
Unacceptable: it would be too disruptive to packages like xml and json and would break documented behavior, breaking the Go 1 compatibility promise. It would mean reflect cannot give access to fields that are visible in Go itself.
adopt gc behavior
Less problematic, but it would make it very easy to access structs that are indicated as hidden. Users could bypass initialization code etc, raising various security issues. The proposed solution does not eliminate this, but it does considerably mitigate it by limiting access to fields that are deemed exported and settable if the embedded type were accessed directly.
package reflect provides a different API for accessing promoted fields and methods.
Provide a higher-level API for packages like xml and json for mapping structured data to struct fields
This would potentially give greater consistency and would make it easier for developers to write package like json and xml. However, it would be a huge change and similarly does not address the concerns of what to do with the current API. Like the current proposal, it would still allow access to non-promoted fields and methods. It could be implemented on top of the proposed solution.
The text was updated successfully, but these errors were encountered:
I think the statement here can be made much more precise. Here is my understanding.
In the gc toolchain,
In the gccgo toolchain,
The fundamental problem here is that a Go program's source code can refer to t1.X (implicitly t1.t2.X) directly, while the reflect API cannot. The reflect API requires stepping into each field explicitly. The gc implementation allows reflection to set t1.X by recording the field t2 as exported (wrong): that makes it possible to set t1.t2.X (arguably right) but also all of t1.t2 (wrong). The gccgo implementation records t2 as unexported (right), which disallows setting t1.t2.X (arguably wrong) and also disallows setting t1.t2 (right).
The proposal is that both implementations record t2 as unexported, and then change reflect to record whether a value like v2 is read-only only because it is an unexported embedded field. In that case, accessing an exported element within would clear the read-only bit. The effect would be that t2 is record as unexported (right), t1.t2.X is settable (arguably right), t1.t2 is not settable (right), and of course t1.t2.u is also not settable (right).
That is, under this proposal:
That satisfies encoding/xml and encoding/json, disallows most of what is incorrectly allowed in the gc toolchain today, and allows what is arguably incorrectly disallowed in the gccgo toolchain today.
Both encoding/json and encoding/xml assume today that if f.PkgPath != nil then the field is unexported and must be ignored. That test needs to be revised to f.PkgPath != nil && !f.Anonymous, so that they do try to walk into the embedded fields to look for unexported fields contained within. Any similar external packages would need the same ~1 line change. The new condition is correct both before and after the proposed change, so it could be made by any affected packages independent of the eventual Go 1.6 release.
See the actual change:
Note that the tests added in the json and xml CLs will currently break for gccgo.
Packages just checking for f.PkgPath to determine if a field is exported should implement a fix similar to those in 14011 and 14012 now to prevent breaking when CL 14010 is checked in.
This CL changes reflect to allow access to exported fields and methods in unexported embedded structs for gccgo and after gc has been adjusted to disallow access to embedded unexported structs. Adresses #12367, #7363, #11007, and #7247. Change-Id: If80536eab35abcd25300d8ddc2d27d5c42d7e78e Reviewed-on: https://go-review.googlesource.com/14010 Reviewed-by: Russ Cox <email@example.com>
…, docs - json: inline skipWhitespace: this may give some slight performance improvement by eliding the function call on the hot-path. - handle exported members of unexported anonymous fields See golang/go#12367 for more info - document that WriteExt, ConvertExt may take a pointer value. To avoid potentially expensive copying, we may pass in the address to a struct or array
skipping it when walking over values in an struct, to make sure that we don't skip validly accessible (exported) embedded values in an unexported field. This new checking logic is the new recommended strategy for reflective struct walking as of the Go 1.6 release. Related/References: - Go 1.6 reflect release note: https://golang.org/doc/go1.6#reflect - Issue: https://golang.org/issue/12367 (golang/go#12367) - CL: https://golang.org/cl/14085 (https://go-review.googlesource.com/#/c/14085/) - Commit: golang/go@afe9837