cmd/vet: warn about structs marked json omitempty

[josharian]

Marking a struct field of type time.Time as json omitempty has no effect. You must use a *time.Time for it to work (which is wire compatible). Vet should detect this situation and perhaps suggest *time.Time instead.

• Frequency: According to @dsnet, this is a common source of package json confusion. This situation has spawned a handful of proposals: proposal: encoding/json, encoding/xml: support zero values of structs with omitempty #11939, proposal: encoding/json: "nonil" struct tag to marshal nil slices and maps as non-null #27589, encoding/json: add omitzero option #45669.
• Precision: There is no sensible interpretation of omitempty on a time.Time field.
• Severity: Depends on how important the omitempty behavior is. In the case that I just encountered, it was moderate severity. In the most likely scenario, it is resource waste (bandwidth, storage).

This might be reasonably extended to any struct type; see the linked omitzero proposals for more discussion. If an omitzero proposal is accepted, vet could switch to suggesting the use of omitzero rather than suggesting a pointer type.

cc @dsnet @robpike @dominikh

[dsnet]

Scanning all modules, I found ~73k fields where the type was either time.Time or *time.Time with a json:omitempty tag. Approximately 33% were of the former (incorrect) and 66% were of the latter (correct) form.

That's pretty significant evidence that this is a highly frequent problem.

[dominikh]

Is there any reason to limit this to time.Time instead of applying to all structs?

[dsnet]

I advocate expanding it to all structs.

[guodongli-google]

Yes this shall be extending any non-primitive and non-reference types, mainly struct.

Interestingly, the official document specifies:

Programs using times should typically store and pass them as values, not pointers. That is, time variables and struct fields should be of type time.Time, not *time.Time.

which actually indicates that a conflicting check that is supposed to report usage of *time.Time in struct fields.
For any time.Time fields, omitempty has no effects. What other harm it can bring other than wasting some storage?

[prattmic]

I don't write much JSON code, so take my thoughts with a grain of salt, but this problem seems to indicate that there should be some form of json:omitzero to omit fields that are equal to the type's zero value [1]. It seems rather painful to require use of pointers just to allow omission on the wire.

[1] Admittedly this is still a problem for time.Time, which prefers time.Time.IsZero() over comparison with var zero time.Time.

[dsnet]

this problem seems to indicate that there should be some form of json:omitzero to omit fields that are equal to the type's zero value

I agree there should be some type of omitzero tag that does what the user intends here, but that seems orthogonal to a vet warning that flags incorrect usages of omitempty with a Go struct type.

[guodongli-google]

this problem seems to indicate that there should be some form of json:omitzero to omit fields that are equal to the type's zero value

I agree there should be some type of omitzero tag that does what the user intends here, but that seems orthogonal to a vet warning that flags incorrect usages of omitempty with a Go struct type.

It is a little bit confusing what "empty" means. Currently:

The "omitempty" option specifies that the field should be omitted from the encoding if the field has an empty value, defined as false, 0, a nil pointer, a nil interface value, and any empty array, slice, map, or string.

Regarding a composite object, what does "empty" or "zero" mean? Consider these cases:

1. type T struct {}; var o T: obviously o is an empty object.
2. type T struct { x int; y *int }; var o T;: o is not empty but it is zero.
3. type T struct { x int}; o := map[int]T{0: T{}};: o is not empty but is it zero?

Users seems to assume that (2) is "empty" when using "omitempty".

[dsnet]

I think this issue getting off topic. There's a lot of valid questions with regard to omitzero, omitempty, and their exact semantics, but that discussion belongs in #11939. We should separate "how can json make this better in the future" from "how can we detect incorrect usages today".

[timothy-king]

One concern I have is how to educate users. It seems like the most likely cause is the user misunderstanding of how structs interact with encoding/json with omitempty being a symptom. Vet's message have historically been quite terse. I am hesitant to give suggestions like "rewrite time.Time to *time.Time here" without giving an explanation as to why (and can cause other bugs during the transition). Similarly a message like "struct field %s is not emitted by encoding/json so the omitempty tag is redundant" seems like it would often [usually?] result in the omitempty tag being removed which does not help the underlying confusion.

Do folks have a suggestion for what to tell the user in the diagnostic? What I really want to do is point users at a longer discussion of the issue (like 1-2 paragraphs describing the issue and possible solutions), but to the best of my knowledge this has not done this before in vet.

[gopherbot]

Change https://go.dev/cl/388574 mentions this issue: go/analysis/passes/structtag: check option "omitempty" on fields of struct types.