Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: cmd/vet: warn about time.Time struct fields marked json omitempty #51261

Open
josharian opened this issue Feb 18, 2022 · 10 comments
Open

proposal: cmd/vet: warn about time.Time struct fields marked json omitempty #51261

josharian opened this issue Feb 18, 2022 · 10 comments
Labels
Projects
Milestone

Comments

@josharian
Copy link
Contributor

@josharian josharian commented Feb 18, 2022

Marking a struct field of type time.Time as json omitempty has no effect. You must use a *time.Time for it to work (which is wire compatible). Vet should detect this situation and perhaps suggest *time.Time instead.

  • Frequency: According to @dsnet, this is a common source of package json confusion. This situation has spawned a handful of proposals: #11939, #27589, #45669.
  • Precision: There is no sensible interpretation of omitempty on a time.Time field.
  • Severity: Depends on how important the omitempty behavior is. In the case that I just encountered, it was moderate severity. In the most likely scenario, it is resource waste (bandwidth, storage).

This might be reasonably extended to any struct type; see the linked omitzero proposals for more discussion. If an omitzero proposal is accepted, vet could switch to suggesting the use of omitzero rather than suggesting a pointer type.

cc @dsnet @robpike @dominikh

@dsnet
Copy link
Member

@dsnet dsnet commented Feb 18, 2022

Scanning all modules, I found ~73k fields where the type was either time.Time or *time.Time with a json:omitempty tag. Approximately 33% were of the former (incorrect) and 66% were of the latter (correct) form.

That's pretty significant evidence that this is a highly frequent problem.

@ianlancetaylor ianlancetaylor added this to Incoming in Proposals Feb 18, 2022
@dominikh
Copy link
Member

@dominikh dominikh commented Feb 19, 2022

Is there any reason to limit this to time.Time instead of applying to all structs?

@dsnet
Copy link
Member

@dsnet dsnet commented Feb 19, 2022

I advocate expanding it to all structs.

@guodongli-google
Copy link

@guodongli-google guodongli-google commented Feb 22, 2022

Yes this shall be extending any non-primitive and non-reference types, mainly struct.

Interestingly, the official document specifies:

Programs using times should typically store and pass them as values, not pointers. That is, time variables and struct fields should be of type time.Time, not *time.Time.

which actually indicates that a conflicting check that is supposed to report usage of *time.Time in struct fields.
For any time.Time fields, omitempty has no effects. What other harm it can bring other than wasting some storage?

@prattmic
Copy link
Member

@prattmic prattmic commented Feb 22, 2022

I don't write much JSON code, so take my thoughts with a grain of salt, but this problem seems to indicate that there should be some form of json:omitzero to omit fields that are equal to the type's zero value [1]. It seems rather painful to require use of pointers just to allow omission on the wire.

[1] Admittedly this is still a problem for time.Time, which prefers time.Time.IsZero() over comparison with var zero time.Time.

@dsnet
Copy link
Member

@dsnet dsnet commented Feb 22, 2022

this problem seems to indicate that there should be some form of json:omitzero to omit fields that are equal to the type's zero value

I agree there should be some type of omitzero tag that does what the user intends here, but that seems orthogonal to a vet warning that flags incorrect usages of omitempty with a Go struct type.

@guodongli-google
Copy link

@guodongli-google guodongli-google commented Feb 22, 2022

this problem seems to indicate that there should be some form of json:omitzero to omit fields that are equal to the type's zero value

I agree there should be some type of omitzero tag that does what the user intends here, but that seems orthogonal to a vet warning that flags incorrect usages of omitempty with a Go struct type.

It is a little bit confusing what "empty" means. Currently:

The "omitempty" option specifies that the field should be omitted from the encoding if the field has an empty value, defined as false, 0, a nil pointer, a nil interface value, and any empty array, slice, map, or string.

Regarding a composite object, what does "empty" or "zero" mean? Consider these cases:

  1. type T struct {}; var o T: obviously o is an empty object.
  2. type T struct { x int; y *int }; var o T;: o is not empty but it is zero.
  3. type T struct { x int}; o := map[int]T{0: T{}};: o is not empty but is it zero?

Users seems to assume that (2) is "empty" when using "omitempty".

@dsnet
Copy link
Member

@dsnet dsnet commented Feb 22, 2022

I think this issue getting off topic. There's a lot of valid questions with regard to omitzero, omitempty, and their exact semantics, but that discussion belongs in #11939. We should separate "how can json make this better in the future" from "how can we detect incorrect usages today".

@timothy-king
Copy link
Contributor

@timothy-king timothy-king commented Feb 23, 2022

One concern I have is how to educate users. It seems like the most likely cause is the user misunderstanding of how structs interact with encoding/json with omitempty being a symptom. Vet's message have historically been quite terse. I am hesitant to give suggestions like "rewrite time.Time to *time.Time here" without giving an explanation as to why (and can cause other bugs during the transition). Similarly a message like "struct field %s is not emitted by encoding/json so the omitempty tag is redundant" seems like it would often [usually?] result in the omitempty tag being removed which does not help the underlying confusion.

Do folks have a suggestion for what to tell the user in the diagnostic? What I really want to do is point users at a longer discussion of the issue (like 1-2 paragraphs describing the issue and possible solutions), but to the best of my knowledge this has not done this before in vet.

@gopherbot
Copy link

@gopherbot gopherbot commented Mar 1, 2022

Change https://go.dev/cl/388574 mentions this issue: go/analysis/passes/structtag: check option "omitempty" on fields of struct types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Proposals
Incoming
Development

No branches or pull requests

7 participants