encoding/json/v2: remove `unknown` tag option and DiscardUnknownMembers

[dsnet]

Proposal Details

(This is a counter-proposal to #76444).

The json/v2 package provides too much functionality with unknown fields:

• A Go struct field may be marked as unknown to indicate that it holds unknown members. Note that this is nearly identical to the existing inline tag option, but additionally indicates intent for holding unknown members.
• DiscardUnknownMembers discards unknown members when marshaling by ignoring any data in Go struct fields marked as unknown.
• RejectUnknownMembers rejects unknown members when unmarshaling (even if a Go struct field is marked as unknown) by reporting an error.

I propose removing the unknown tag option and the DiscardUnknownMembers option. Note that RejectUnknownMembers still needs to exist to maintain backwards compatibility with the jsonv1.Decoder.DisallowUnknownFields feature.

We could always re-introduce the concept of Go struct fields marked as unknown in a future release of json/v2, but then we would need to think about how all related options might interoperate (see #76444). Fortunately, adding new features is always backwards compatible, but removing or changing them is not.

[gabyhelp]

Related Issues

• encoding/json/v2: make DiscardUnknownMembers and RejectUnknownMembers affect both marshaling and unmarshaling #76444
• proposal: encoding/json: preserve unknown fields #22533
• proposal: encoding/json: reject unknown fields in Decoder #15314 (closed)
• encoding/json/v2: proposal: require known fields to exist when unmarshaling #76458
• proposal: Allow specifying a field which takes all unexpected fields when parsing JSON #42417 (closed)
• proposal: encoding/json: add "Skip when Unmarshalling" tag #19423 (closed)
• encoding/json, encoding/xml: option to treat unknown fields as an error #6901 (closed)
• proposal: Struct field tags for directional Marshal/Unmarshal #18662 (closed)
• encoding/json: show nested fields path if DisallowUnknownFields leads to an error #58649

Related Code Changes

• encoding/json: add Decoder.DisallowUnknownFields option to reject unknown keys

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[dsnet]

We discussed this in the json/v2 working group and decided to adopt this proposal over #76444.

The difference between inline and unknown is subtle. They operate in nearly the exact same way, except unknown carries an extra bit of information that the field contains unknown fields as opposed to known fields.

The only relevant distinction is with RejectUnknownMembers and DiscardUnknownMembers options, where the former rejects unknown JSON members even if a Go field marked with unknown exists, while the later ignores the presence of a Go field marked with unknown when marshaling. This subtle distinction is difficult even for experienced Go programmers to understand.

For the initial release of json/v2, we will remove the concept of the unknown tag option and the DiscardUnknownMembers option. These can always be added back in the future in a backwards compatible manner. At present, we don't have evidence that this feature is needed. The inline tag option (new in v2) already gets you most of you would want with unknown member handling and that's already a significant new feature relative to v1.

As a workaround for existing users, they can migrate existing usages of unknown to inline. They can manually inspect the Go struct field marked with inline and report an error after unmarshaling (to have similar behavior as RejectUnknownMembers), or clear the field before marshaling (to have similar behavior as DiscardUnknownMembers). Essentially, it places the burden of considering of what is considered "unknown" on the type author, rather than being a first-class feature of the "json" package.

Of particular note, the RejectUnknownMembers option still exists and operates the same way (except that it has no special semantics with regard to the unknown tag option since we would have removed that). If you never used the unknown tag option or DiscardUnknownMembers option, then removal of this feature does not affect you in any way.

[gopherbot]

Change https://go.dev/cl/741320 mentions this issue: encoding/json/v2: remove unknown tag option and DiscardUnknownMembers