encoding/json: potential compatibility issues with "inline" tag on existing types

[prattmic]

Go 1.27 adds a new JSON "inline" struct tag as part of #71497. This tag specifies that the JSON object representation of this field is to be promoted as if it were specified in the parent struct. It is the JSON equivalent of Go struct embedding. A Go embedded field is implicitly inlined unless an explicit JSON name is specified.

In Go 1.27, this tag is supported by both the encoding/json and encoding/json/v2 APIs. Support in encoding/json is intentional to ensure that packages using new tags on their types don't implicitly require all of their dependents switch to the encoding/json/v2 API.

Even though this tag doesn't exist in 1.26, it turns out that it has widespread use throughout public Go code, particularly within the Kubernetes community. https://grep.app/search?q=%60json%3A%22%2Cinline%22%60 shows ~29k results for uses on the "inline" tag.

I do not know the original source of this tag in Kubernetes. It seems to have originated at some point and then got copied around. In fact, @liggitt has been removing uses of this tag since it didn't actually do anything (kubernetes/kubernetes#138260).

Regardless, Kubernetes uses encoding/json and many of these tags still exist, so there is a risk of behavior change when the 1.27 toolchain upgrade occurs and encoding/json gains support for this tag.

29k uses sounds dire, but the vast majority of uses apply the tag to an embedded field. Embedded fields are already inlined, so the "inline" tag has no impact to these fields.

I analyzed modules in the Go module proxy with at 3 transitive importers for uses of the "inline" tag on named fields. The full set of 2639 findings are here.

While I have not looked at every finding, some things I've found so far:

• A large proportion of uses come from github.com/openai/openai-go or copies of this code. This module uses a custom json package (github.com/openai/openai-go/internal/apijson) that supports an "inline" tag with semantics similar to Go 1.27. It does not appear that these types are ever used with encoding/json. Rudimentary filtering of the findings to exclude these packages reduces the list to 694 findings. The full filtered list is here.
• https://go-mod-viewer.appspot.com/knative.dev/pkg@v0.0.0-20260602142205-ac97e43f6622/apis/volatile_time.go#L34 applies an "inline" tag, but also implements a MarshalJSON method that manually implements the inline behavior. I've seen this pattern on several different types. This should be fine in 1.27 since MarshalJSON will render "inline" useless.
• https://go-mod-viewer.appspot.com/github.com/chainreactors/fingers@v1.2.1/fingers/fingers.go#L18 is affected as far as I can tell. This module has a LoadFingers function that unmarshals this type with encoding/json, which 1.27 will do differently. Ironically I can't find any tests in this module that actually care about the inlined fields, so I don't have confirmation that it is broken.
• The most frequent case I see are cases like https://go-mod-viewer.appspot.com/github.com/rancher/opni-monitoring@v0.4.0/pkg/sdk/api/v1beta1/gateway_types.go#L36, where marshaling of this type is definitely affected by 1.27, but the code is too abstract for me to easily determine whether the type is ever used with encoding/json.

All told, I have not yet found any cases that are definitively broken by a 1.27 behavior change, but I do not have high confidence that there are none.

I'm creating this issue to discuss what we should do about this. Currently I see four options:

1. Simply do nothing because we don't believe that this will cause ecosystem disruption.
2. Rename the "inline" tag to something with no existing use in the ecosystem to avoid potential conflicts.
3. Drop the "inline" tag entirely, and reconsider it for 1.28.
4. Add a compatibility GODEBUG that gates the "inline" tag behind the language version. See the full description of this below.

cc @dsnet @aclements @ChrisHines @neild @liggitt

[prattmic]

My proposal for a compatibility GODEBUG is:

We define a new json127tags GODEBUG option. This option defaults to 0 if language version is 1.26 or lower, and 1 if the language version is 1.27 or higher.

If GODEBUG=json127tags=0, then encoding/json completely ignores the new struct tags (case:ignore, case:strict, inline), as in 1.26 today. The non-”inline” new tags are included for consistency.

If GODEBUG=json127tags=1, then encoding/json supports these tags, as implemented at tip today.

For encoding/json/v2, if GODEBUG=json127tags=0, the presence of one of these tags results in a run time error which notes that these tags require language version 1.27.

If GODEBUG=json127tags=1, then encoding/json/v2 supports these tags, as implemented at tip today.

The basic idea here is to emulate as if these tags were actual language features. We can't fail the build on their use, but we can have v2 report an error on use.

When upgrading to a 1.27 toolchain, bad “inline” tags can’t cause compatibility issues. Any issues are deferred until the main module chooses to upgrade to the 1.27 language version.

This actually has a nice property beyond the “inline” compatibility concern. Today I may upgrade to a 1.27 toolchain and start using the new tags that I find useful. I am not required to increase the language version, so my module remains at language version 1.26. Users of my module may be using a 1.26 toolchain, and thus not support the new tags. My types will thus not marshal as expected. If the new tags are guarded behind language version 1.27, then I must upgrade my module’s language version, which will in turn require my users to do the same.

I think this makes a solid argument for putting any new tags behind the language version. This is a new idea, we did not do this for “omitzero”, even though the same argument applies there. With typed struct tags (#74472) this problem goes away as the new toolchain will be required to resolve the tag’s symbol.

One downside of this GODEBUG idea is that compatibility GODEBUGs change behavior across the entire program, not on a per-module basis. So while bad “inline” tags won’t break things when upgrading the toolchain, they will break things when upgrading the language version. The bad tag may be deep in a transitive dependency with a lower language version, so it could be difficult to find the problematic type.

The bisect tool supports a -godebug flag to add a bisect string to the GODEBUG. encoding/json could use the bisect string to decide which types enable “inline” support so the bisect tool can pinpoint the problematic type. This would help, but it is still a pretty annoying and niche option.

[dims]

I do not know the original source of this tag in Kubernetes. It seems to have originated at some point and then got copied around.

It started here i think:

• kubernetes/kubernetes@601f6bb
• Parse settings client-side (addresses #67) kubernetes/kubernetes#79

it was a mirror of the functional yaml:",inline"

[prattmic]

Thanks @dims, that's helpful (also an impressively early Kubernetes PR!). If I understand that PR correctly, the idea is that the types can be given as JSON, since gopkg.in/v1/yaml can unmarshal JSON (JSON is valid YAML). But the tag looks like a misunderstanding, as gopkg.in/v1/yaml appears to always use the yaml tag even if the input happens to be JSON.

[neild]

I don't really like the idea of a version-gated GODEBUG, but I'm not adamantly opposed. A GODEBUG has a few problems that worry me:

• Package A uses a Go 1.27 inline tag. Package B uses a legacy inline tag. You want to use packages A and B. You can't; this is an unresolvable conflict.
• Action at a distance: I have a package that uses a legacy inline tag. My go.mod has a language version of 1.26. All my tests pass. If my package is imported by a program built with a higher language version, my package breaks.
• Possibility of getting the ecosystem stuck on a lower version. If the advice is that if you use Kubernetes modules, you must use a language version of 1.26, do we get an ecosystem of modules that don't dare set anything higher?

I think my preferences are to change the tag name to "embed" (which hopefully doesn't have any conflicts) or disable it for 1.27 and hopefully add it back in as a structured tag in 1.28. This assumes we manage to get structured tags in 1.28, of course, but signs are hopeful for that at this time.

[prattmic]

The name "embed" does have a lot going for it, as "inline" already describes itself with "It is the JSON equivalent of Go struct embedding. A Go embedded field is implicitly inlined unless an explicit JSON name is specified."

My quick search attempt did not find any uses of the name. It did find a use of inline with a comment saying embed. 😃 https://grep.app/search?regexp=true&q=%60json%3A%22.*%2C.*embed

[ChrisHines]

I have no problem with switching to "embed". FWIW, during the design discussions around this feature I often forgot what the semantics of the "inline" tag were and had to be reminded by others. Aligning the tag with the Go embedding feature makes it easier for a Gopher like me to remember. :)

[gabyhelp]

Related Issues

• encoding/json: fields with "inline" tag and unsupported type are silently dropped #79921 (closed)
• proposal: encoding/json: add "inline" struct tag #6213

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[dsnet]

Some thoughts:

• I'd rather not remove support for inline for Go 1.27.
• I don't feel strongly about whether to gate this being a GODEBUG.
• Switching to embed sounds fine to me. In general, I'd prefer to use JSON terminology for features, but there is nothing in the RFCs that's remotely like this. Given that embed operates most closely to Go's embed feature and arguably is semantically closely tied with it, naming it after Go terminology makes sense. The term inline lacks any shared heritage with both Go and JSON.

Given that https://go.dev/cl/788864 makes it such that invalid usages of inline still operate like normal fields, do we still anticipate more compatibility issues?

[prattmic]

Given that https://go.dev/cl/788864 makes it such that invalid usages of inline still operate like normal fields, do we still anticipate more compatibility issues?

The cases I'm worried about are ones like https://go-mod-viewer.appspot.com/github.com/rancher/opni-monitoring@v0.4.0/pkg/sdk/api/v1beta1/gateway_types.go#L36, where the use of "inline" is valid, but was placed there by mistake and never mattered because 1.26 ignores the tag.

[liggitt]

The cases I'm worried about are ones like https://go-mod-viewer.appspot.com/github.com/rancher/opni-monitoring@v0.4.0/pkg/sdk/api/v1beta1/gateway_types.go#L36, where the use of "inline" is valid, but was placed there by mistake and never mattered because 1.26 ignores the tag.

Exactly ... the only good outcomes require someone to notice the tag wasn't doing what they intended and fixing it (either by dropping the inline tag to keep compat with their existing serializations, or by modifying their Go struct to actually embed to get the serialization behavior they wanted). That particular example did end up embedding the field later if you trace through to the current repos for that type.

Silently changing serialization behavior of a widely-used-but-inert-tag when switching a module from building with Go 1.26 to Go 1.27 will almost certainly break someone.

For my own projects (e.g. Kubernetes), we added linters to flag use of inline with non-embedded struct fields, so I'm not personally invested either way, but going with a tag like embed we think has ~zero existing use would definitely be safer for the ecosystem.

[dsnet]

Thanks @prattmic and @liggitt for the explanation. I propose that we switch to using name embed for the feature.

I scanned the latest version of all modules known by the Go module proxy:

• There are ~119M usages of json field tags
• There are ~649k occurrences (0.5%) of the inline field tag
• There are 0 detected occurrences of the embed field tag

For some history, when I ran this analysis back in December of 2021:

• There were ~18M usages of json field tags
• There were ~52k occurrences (0.3%) of the inline field tag