proposal: encoding/json: export tagOptions, parseTag

[sparr]

https://cs.opensource.google/go/go/+/master:src/encoding/json/tags.go

Multiple other packages reimplement similar functionality, including efforts to match the behavior of encoding/json:

• https://github.com/pelletier/go-toml/blob/v2/marshaler.go#L768-L797 and https://github.com/pelletier/go-toml#struct-tags-have-been-merged
• https://github.com/go-yaml/yaml/blob/v3/yaml.go#L557-L572

Exporting this functionality would allow those and similar packages to have convenient access to identical behavior to the json package. It would even allow them to safely parse the json tag, if they so choose.

[apparentlymart]

The fact that non-JSON packages are trying to use the JSON package's tags seems to suggest that there's a missing abstraction here, for defining the relationship between properties in a serialization format and Go struct fields in a format-agnostic way.

While I expect it would be limited only to formats that have an infoset similar to JSON's, might it be better to expose a new struct tag scheme that is format-agnostic and then make encoding/json support it, rather than effectively "making it official" that the JSON package struct tags are the format-agnostic tags? 🤔

To make this alternative a bit more concrete for discussion, consider:

struct Example {
    Foo string `marshal:"foo"`
}

In the above, marshal is my hypothetical name for the format-agnostic struct tag type, with a parser for that format in its own dedicated package that encoding/json then depends on. I assume it would also support omitempty like the json struct tags do. Whether it should also support the other oddities like the string option is debatable; I've not researched how common that need is across other serialization formats. (e.g. in real-world YAML the exact type of a scalar is often just implied rather than explicit anyway.)

(I could also see just blessing "json" as the name for the general-purpose format as unfortunate but pragmatic, given that it paves an existing cowpath, but it is somewhat confusing for newcomers. I wonder what others think. I think accepting this proposal boils down to blessing "json" as the official format-agnostic name for "JSON-like" serialization formats -- whatever "JSON-like" might mean.)

[sparr]

https://gist.github.com/sparr/008ebfab065bd00c3dd903fa43b9ad55

I actually drafted something along those lines but decided to submit this first as lower hanging fruit.

[sparr]

To be clear, this proposal is about parsing what's inside a tag, the "name,flag,flag,flag" part. Nothing in the functions I'm proposing exposing is specific to the json: tag name. Those other packages could use it to parse the json tag, but they could also use it to parse the toml and yaml tags that they currently parse using their own parser implementations.

[ianlancetaylor]

tagOptions and parseTag are pretty simple and don't have anything to do with JSON. If several different packages are implementing the same thing, that is an argument for providing this functionality somewhere in the standard library, but I don't see why encoding/json would be the right place for it.

We already have reflect.StructTag with Get and Lookup methods. We could perhaps extend that a bit, if somebody wants to think about what that might look like.

[sparr]

If we're going to expose these, I might also recommend benchmarking Contains current implementation (repeated strings.Cut) vs using a regular expression. The regex might look simpler and be faster.

I guess I should actually create that other proposal since that seems closer to what the feedback here is suggesting anyway.