New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: encoding/json, encoding/xml: support zero values of structs with omitempty #11939

Open
joeshaw opened this Issue Jul 30, 2015 · 39 comments

Comments

Projects
None yet
@joeshaw
Contributor

joeshaw commented Jul 30, 2015

Support zero values of structs with omitempty in encoding/json and encoding/xml.

This bites people a lot, especially with time.Time. Open bugs include #4357 (which has many dups) and #10648. There may be others.

Proposal

Check for zero struct values by adding an additional case to the isEmptyValue function:

case reflect.Struct:
        return reflect.Zero(v.Type()).Interface() == v.Interface()

This will solve the vast majority of cases.

(Optional) Introduce a new encoding.IsZeroer interface, and use this to check for emptiness:

Update: I am dropping this part of the proposal, see below.

type IsZeroer interface {
        IsZero() bool
}

Visit this playground link and note that the unmarshaled time.Time value does not have a nil Location field. This prevents the reflection-based emptiness check from working. IsZero() already exists on time.Time, has the correct semantics, and has been adopted as a convention by Go code outside the standard library.

An additional check can be added to the isEmptyValue() functions before checking the value's Kind:

if z, ok := v.Interface().(encoding.IsZeroer); ok {
        return z.IsZero()
}

Compatibility

The encoding.IsZeroer interface could introduce issues with existing non-struct types that may have implemented IsZero() without consideration of omitempty. If this is undesirable, the encoding.IsZeroer interface check could be moved only within the struct case:

case reflect.Struct:
        val := v.Interface()
        if z, ok := val.(encoding.IsZeroer); ok {
                return z.IsZero()
        }
        return reflect.Zero(v.Type()).Interface() == val

Otherwise, this change is backward-compatible with existing valid uses of omitempty. Users who have applied omitempty to struct fields incorrectly will get their originally intended behavior for free.

Implementation

I (@joeshaw) have implemented and tested this change locally, and will send the CL when the Go 1.6 tree opens.

@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Jul 30, 2015

@mikioh mikioh added the Proposal label Aug 13, 2015

@gopherbot

This comment has been minimized.

Show comment
Hide comment
@gopherbot

gopherbot commented Aug 25, 2015

CL https://golang.org/cl/13914 mentions this issue.

@gopherbot

This comment has been minimized.

Show comment
Hide comment
@gopherbot

gopherbot commented Aug 28, 2015

CL https://golang.org/cl/13977 mentions this issue.

@joeshaw

This comment has been minimized.

Show comment
Hide comment
@joeshaw

joeshaw Sep 18, 2015

Contributor

The empty struct approach is implemented in CL 13914 and the IsZeroer interface is implemented in CL 13977.

In order for them to be reviewable separately they conflict a bit -- mostly in the documentation -- but I will fix for one if the other is merged.

Contributor

joeshaw commented Sep 18, 2015

The empty struct approach is implemented in CL 13914 and the IsZeroer interface is implemented in CL 13977.

In order for them to be reviewable separately they conflict a bit -- mostly in the documentation -- but I will fix for one if the other is merged.

@adg adg added Proposal and removed Proposal labels Sep 25, 2015

@joeshaw

This comment has been minimized.

Show comment
Hide comment
@joeshaw

joeshaw Oct 19, 2015

Contributor

In the CLs @rsc said,

I'd really like to stop adding to these packages. I think we need to leave well enough alone at some point.

I see what he's getting at. CL 13977, which implements the IsZeroer interface is clearly an enhancement and adds API to the standard library that needs to be maintained forever. So, I am abandoning that CL and that part of the proposal.

However, I still feel strongly about omitempty with empty structs, and I want to push for CL 13914 to land for Go 1.6.

I use the JSON encoding in Go a lot, as my work is mostly writing services that communicate with other services, in multiple languages, over HTTP. The fact that structs don't obey omitempty is a frequent source of confusion (see #4357 and its many dups and references, and #10648) and working around it is really annoying. Other programming languages do not conform to Go's ideal "zero value" idea, and as a result encoding a zero value is semantically different in JSON than omitting it or encoding it as null. People run into this most commonly with time.Time. (There is also the issue that decoding a zero time.Time does not result in an empty struct, see #4357 (comment) for background on that.)

I think it should be considered a bug that Go does not support omitempty for these types, and although it adds a small amount of additional code, it fixes a bug.

Contributor

joeshaw commented Oct 19, 2015

In the CLs @rsc said,

I'd really like to stop adding to these packages. I think we need to leave well enough alone at some point.

I see what he's getting at. CL 13977, which implements the IsZeroer interface is clearly an enhancement and adds API to the standard library that needs to be maintained forever. So, I am abandoning that CL and that part of the proposal.

However, I still feel strongly about omitempty with empty structs, and I want to push for CL 13914 to land for Go 1.6.

I use the JSON encoding in Go a lot, as my work is mostly writing services that communicate with other services, in multiple languages, over HTTP. The fact that structs don't obey omitempty is a frequent source of confusion (see #4357 and its many dups and references, and #10648) and working around it is really annoying. Other programming languages do not conform to Go's ideal "zero value" idea, and as a result encoding a zero value is semantically different in JSON than omitting it or encoding it as null. People run into this most commonly with time.Time. (There is also the issue that decoding a zero time.Time does not result in an empty struct, see #4357 (comment) for background on that.)

I think it should be considered a bug that Go does not support omitempty for these types, and although it adds a small amount of additional code, it fixes a bug.

@rsc rsc modified the milestones: Proposal, Unplanned Oct 24, 2015

@rsc rsc changed the title from proposal: encoding: Support zero values of structs with omitempty in encoding/json and encoding/xml to proposal: encoding/json, encoding/xml: support zero values of structs with omitempty Oct 24, 2015

@jeromenerf

This comment has been minimized.

Show comment
Hide comment
@jeromenerf

jeromenerf Mar 27, 2016

This proposal is marked as unplanned, yet the related bug report #10648 is marked as go1.7.
Is it still being worked /thought on?

jeromenerf commented Mar 27, 2016

This proposal is marked as unplanned, yet the related bug report #10648 is marked as go1.7.
Is it still being worked /thought on?

@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc Mar 28, 2016

Contributor

To my knowledge, it is not being worked on. Honestly this seems fairly low
priority and will likely miss Go 1.7.

On Sun, Mar 27, 2016 at 12:22 PM Jérôme Andrieux notifications@github.com
wrote:

This proposal is marked as unplanned, yet the related bug report #10648
#10648 is marked as go1.7.
Is it still being worked /thought on?


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#11939 (comment)

Contributor

rsc commented Mar 28, 2016

To my knowledge, it is not being worked on. Honestly this seems fairly low
priority and will likely miss Go 1.7.

On Sun, Mar 27, 2016 at 12:22 PM Jérôme Andrieux notifications@github.com
wrote:

This proposal is marked as unplanned, yet the related bug report #10648
#10648 is marked as go1.7.
Is it still being worked /thought on?


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#11939 (comment)

@jeromenerf

This comment has been minimized.

Show comment
Hide comment
@jeromenerf

jeromenerf Mar 28, 2016

OK.

This is more of a convenience than a priority indeed.

It can be a pain point when dealing with libs that don't support "embedded structs" as pointer though.

jeromenerf commented Mar 28, 2016

OK.

This is more of a convenience than a priority indeed.

It can be a pain point when dealing with libs that don't support "embedded structs" as pointer though.

@Perelandric

This comment has been minimized.

Show comment
Hide comment
@Perelandric

Perelandric Mar 29, 2016

I wonder if a low-impact alternative to the IsZeroer interface would be to allow one to return an error called json.CanOmit (or similar) from an implementation of the Marshaler interface. That way the dev is in control of determining what constitutes a zero value, and it doesn't impact other code.

It's not a perfect solution, since one can't add methods to types defined in another package, but this can be worked around to a degree.

Taking the time.Time example:

type MyTime struct {
  time.Time
}

// Implement the Marshaler interface
func (mt MyTime) MarshalJSON() ([]byte, error) {
  res, err := json.Marshal(mt.Time)

  if err == nil && mt.IsZero() {
    return res, json.CanOmit // Exclude zero value from fields with `omitempty`
  }
  return res, err
}

I haven't looked into implementation, but on the surface it would seem like a low-overhead solution, assuming the only work would be to check if an error returned equals json.CanOmit on fields where omitempty was included.

Using errors as a flag is not without precedent in the standard library, e.g. filepath#WalkFunc allows one to return filepath.SkipDir to skip recursion into a directory.

Perelandric commented Mar 29, 2016

I wonder if a low-impact alternative to the IsZeroer interface would be to allow one to return an error called json.CanOmit (or similar) from an implementation of the Marshaler interface. That way the dev is in control of determining what constitutes a zero value, and it doesn't impact other code.

It's not a perfect solution, since one can't add methods to types defined in another package, but this can be worked around to a degree.

Taking the time.Time example:

type MyTime struct {
  time.Time
}

// Implement the Marshaler interface
func (mt MyTime) MarshalJSON() ([]byte, error) {
  res, err := json.Marshal(mt.Time)

  if err == nil && mt.IsZero() {
    return res, json.CanOmit // Exclude zero value from fields with `omitempty`
  }
  return res, err
}

I haven't looked into implementation, but on the surface it would seem like a low-overhead solution, assuming the only work would be to check if an error returned equals json.CanOmit on fields where omitempty was included.

Using errors as a flag is not without precedent in the standard library, e.g. filepath#WalkFunc allows one to return filepath.SkipDir to skip recursion into a directory.

@joeshaw

This comment has been minimized.

Show comment
Hide comment
@joeshaw

joeshaw May 12, 2016

Contributor

@Perelandric I mentioned a possible sentinel error value in https://golang.org/cl/13914 but I didn't get feedback on the idea or an opportunity to implement it before the Go 1.7 freeze. After Russ's comments on my original CL (and showing the unexpected difficulty in implementing this) I think that's the better way to go.

Contributor

joeshaw commented May 12, 2016

@Perelandric I mentioned a possible sentinel error value in https://golang.org/cl/13914 but I didn't get feedback on the idea or an opportunity to implement it before the Go 1.7 freeze. After Russ's comments on my original CL (and showing the unexpected difficulty in implementing this) I think that's the better way to go.

@gopherbot

This comment has been minimized.

Show comment
Hide comment
@gopherbot

gopherbot commented May 15, 2016

CL https://golang.org/cl/23088 mentions this issue.

@adg

This comment has been minimized.

Show comment
Hide comment
@adg

adg Jul 19, 2016

Contributor

While it's clear that we can do this, it's not clear that we want to. I'd like to see a formal proposal document that weighs the advantages and drawbacks of this feature addition. In particular, I am concerned about compatibility and maintenance burden.

Contributor

adg commented Jul 19, 2016

While it's clear that we can do this, it's not clear that we want to. I'd like to see a formal proposal document that weighs the advantages and drawbacks of this feature addition. In particular, I am concerned about compatibility and maintenance burden.

@joeshaw

This comment has been minimized.

Show comment
Hide comment
@joeshaw

joeshaw Jul 19, 2016

Contributor

thanks Andrew. I worked on this a little bit at GopherCon. I will look into putting together a formal proposal.

Contributor

joeshaw commented Jul 19, 2016

thanks Andrew. I worked on this a little bit at GopherCon. I will look into putting together a formal proposal.

@albrow

This comment has been minimized.

Show comment
Hide comment
@albrow

albrow Aug 2, 2016

Contributor

@joeshaw we ran into this issue at my place of work and I'm eagerly awaiting your proposal. Feel free to contact me if you would like any help. Email is on my profile.

Contributor

albrow commented Aug 2, 2016

@joeshaw we ran into this issue at my place of work and I'm eagerly awaiting your proposal. Feel free to contact me if you would like any help. Email is on my profile.

@Perelandric

This comment has been minimized.

Show comment
Hide comment
@Perelandric

Perelandric Sep 8, 2016

@joeshaw Is the proposal you're considering based on the sentinel object idea, or are you considering a different approach? Do you think you'll have time for this before the next release?

Perelandric commented Sep 8, 2016

@joeshaw Is the proposal you're considering based on the sentinel object idea, or are you considering a different approach? Do you think you'll have time for this before the next release?

@joeshaw

This comment has been minimized.

Show comment
Hide comment
@joeshaw

joeshaw Sep 9, 2016

Contributor

@Perelandric Yes, I think the sentinel object idea is the most straightforward way to go.

Other options include:

  • The IsZeroer interface (but I think this has potential backward compatibility issues)
  • "Backspacing" over objects that serialize to {} (but I think this requires too big a change to the JSON encoder code, and doesn't handle json.Marshaler implementers like time.Time)

I don't think I will be able to do this (proposal + implementation) before Go 1.8. If someone else wants to take it on for 1.8, I will gladly pass along my knowledge and partial implementation.

Contributor

joeshaw commented Sep 9, 2016

@Perelandric Yes, I think the sentinel object idea is the most straightforward way to go.

Other options include:

  • The IsZeroer interface (but I think this has potential backward compatibility issues)
  • "Backspacing" over objects that serialize to {} (but I think this requires too big a change to the JSON encoder code, and doesn't handle json.Marshaler implementers like time.Time)

I don't think I will be able to do this (proposal + implementation) before Go 1.8. If someone else wants to take it on for 1.8, I will gladly pass along my knowledge and partial implementation.

@Perelandric

This comment has been minimized.

Show comment
Hide comment
@Perelandric

Perelandric Sep 10, 2016

Thanks @joeshaw. I created an implementation using the sentinel error for the encoding/json package and will start to work on the proposal in a bit. I think I'll focus primarily on this approach.

The Marshaler interface in encoding/xml is different from that in encoding/json, and seems as though a custom zero-value can already be established without needing to return anything special. Did you find that to be true?

After I make a little more progress, I'll post a link to a branch in case you, @albrow or anyone else wishes to review and contribute.

If you have any additional thoughts or info in the meantime, please let me know. Thank you!

Perelandric commented Sep 10, 2016

Thanks @joeshaw. I created an implementation using the sentinel error for the encoding/json package and will start to work on the proposal in a bit. I think I'll focus primarily on this approach.

The Marshaler interface in encoding/xml is different from that in encoding/json, and seems as though a custom zero-value can already be established without needing to return anything special. Did you find that to be true?

After I make a little more progress, I'll post a link to a branch in case you, @albrow or anyone else wishes to review and contribute.

If you have any additional thoughts or info in the meantime, please let me know. Thank you!

@Perelandric

This comment has been minimized.

Show comment
Hide comment
@Perelandric

Perelandric Sep 19, 2016

Change of heart on this. If there's resistance to adding to packages, then this won't fly. Maybe someone else wishes to advocate for this.

Perelandric commented Sep 19, 2016

Change of heart on this. If there's resistance to adding to packages, then this won't fly. Maybe someone else wishes to advocate for this.

@pschultz

This comment has been minimized.

Show comment
Hide comment
@pschultz

pschultz Dec 14, 2016

Mentioned in one of the duplicate issues was the idea to let MarshalJSON return (nil, nil) to skip the field. Borrowing your earlier example:

type MyTime struct { time.Time }

func (mt MyTime) MarshalJSON() ([]byte, error) {
    if mt.IsZero() {
        return nil, nil // Exclude zero value from fields with `omitempty`
    }

    return json.Marshal(mt.Time)
}

In Go 1.7, returning nil is not a valid implementation for MarshalJSON and leads to "unexpected end of JSON input" errors. This approach doesn't require any visible change to the encoding package (not even adding an error value).

For what it's worth, I just intuitively wrote a MarshalJSON method like that, expecting a field to be omitted from the JSON output.

pschultz commented Dec 14, 2016

Mentioned in one of the duplicate issues was the idea to let MarshalJSON return (nil, nil) to skip the field. Borrowing your earlier example:

type MyTime struct { time.Time }

func (mt MyTime) MarshalJSON() ([]byte, error) {
    if mt.IsZero() {
        return nil, nil // Exclude zero value from fields with `omitempty`
    }

    return json.Marshal(mt.Time)
}

In Go 1.7, returning nil is not a valid implementation for MarshalJSON and leads to "unexpected end of JSON input" errors. This approach doesn't require any visible change to the encoding package (not even adding an error value).

For what it's worth, I just intuitively wrote a MarshalJSON method like that, expecting a field to be omitted from the JSON output.

@joeshaw

This comment has been minimized.

Show comment
Hide comment
@joeshaw

joeshaw Dec 14, 2016

Contributor

@pschultz That approach seems reasonable to me, but it can't be used with omitempty.

The reason you get that error is because the JSON encoder checks for the validity of the JSON coming out of MarshalJSON and the result of returning a nil byte slice is (something like) "key":,. If returning a nil byte slice indicated that it should be omitted that'd be a different way to omit something from being encoded in the JSON than omitempty. (That might be fine, it seems ok to me.)

The benefit of the error value is that it could fit in with the existing omitempty because you'd return something like []byte(""), ErrOmitEmpty and it'd obey omitempty yet still return a valid value ("") if not set.

Contributor

joeshaw commented Dec 14, 2016

@pschultz That approach seems reasonable to me, but it can't be used with omitempty.

The reason you get that error is because the JSON encoder checks for the validity of the JSON coming out of MarshalJSON and the result of returning a nil byte slice is (something like) "key":,. If returning a nil byte slice indicated that it should be omitted that'd be a different way to omit something from being encoded in the JSON than omitempty. (That might be fine, it seems ok to me.)

The benefit of the error value is that it could fit in with the existing omitempty because you'd return something like []byte(""), ErrOmitEmpty and it'd obey omitempty yet still return a valid value ("") if not set.

@Perelandric

This comment has been minimized.

Show comment
Hide comment
@Perelandric

Perelandric Dec 14, 2016

@pschultz: A nil return wouldn't be able to be used as a flag for omitempty without causing undesired errors when omitempty is not present, since the implementation of MarshalJSON doesn't know when omitempty is actually there.

I don't know if having nil as an alternative to omitempty would be the best either. Seems like the user of a type should be the one deciding when it is omitted. IMO, the implementation of MarshalJSON should always return some useful representation, or an error when that's impossible.

Perelandric commented Dec 14, 2016

@pschultz: A nil return wouldn't be able to be used as a flag for omitempty without causing undesired errors when omitempty is not present, since the implementation of MarshalJSON doesn't know when omitempty is actually there.

I don't know if having nil as an alternative to omitempty would be the best either. Seems like the user of a type should be the one deciding when it is omitted. IMO, the implementation of MarshalJSON should always return some useful representation, or an error when that's impossible.

oneumyvakin added a commit to oneumyvakin/jirardeau that referenced this issue Jan 17, 2017

@lokhman

This comment has been minimized.

Show comment
Hide comment
@lokhman

lokhman Mar 15, 2018

I created a simple library that patches isEmptyValue function in memory and declares Zeroer interface: https://github.com/lokhman/json-zeroer. Use it at your own risk :)

lokhman commented Mar 15, 2018

I created a simple library that patches isEmptyValue function in memory and declares Zeroer interface: https://github.com/lokhman/json-zeroer. Use it at your own risk :)

@gopherbot

This comment has been minimized.

Show comment
Hide comment
@gopherbot

gopherbot Mar 22, 2018

Change https://golang.org/cl/102158 mentions this issue: encoding/json: encoding/xml: interface to omit marshalling empty structs

gopherbot commented Mar 22, 2018

Change https://golang.org/cl/102158 mentions this issue: encoding/json: encoding/xml: interface to omit marshalling empty structs

wayneashleyberry added a commit to wayneashleyberry/null that referenced this issue Apr 26, 2018

Update README.md
Updates the issue tracker link, golang/go#4357 has been closed and the newer proposal is here: golang/go#11939

dmitshur added a commit to andygrunwald/go-gerrit that referenced this issue May 6, 2018

Use Timestamp type for all time fields.
This is a breaking API change. It improves usability of time fields
by automatically parsing them into a time.Time-like type.

Pointers are used for optional fields, so that the ,omitempty option
correctly omits them when they have zero value (i.e., nil).
This can't be done with values at this time (see golang/go#11939).

Resolves #55.

dmitshur added a commit to andygrunwald/go-gerrit that referenced this issue May 6, 2018

Use Timestamp type for all time fields.
This is a breaking API change. It improves usability of time fields
by automatically parsing them into a time.Time-like type.

Pointers are used for optional fields, so that the ,omitempty option
correctly omits them when they have zero value (i.e., nil).
This can't be done with values at this time (see golang/go#11939).

Resolves #55.

andygrunwald added a commit to andygrunwald/go-gerrit that referenced this issue May 6, 2018

Use Timestamp type for all time fields.
This is a breaking API change. It improves usability of time fields
by automatically parsing them into a time.Time-like type.

Pointers are used for optional fields, so that the ,omitempty option
correctly omits them when they have zero value (i.e., nil).
This can't be done with values at this time (see golang/go#11939).

Resolves #55.
@wayneashleyberry

This comment has been minimized.

Show comment
Hide comment
@wayneashleyberry

wayneashleyberry Jun 7, 2018

Contributor

I'd really like to help out as this has also been a fairly large source of confusion in my organisation, but until this (or another) proposal is accepted it doesn't seem like there's anything to do?

Contributor

wayneashleyberry commented Jun 7, 2018

I'd really like to help out as this has also been a fairly large source of confusion in my organisation, but until this (or another) proposal is accepted it doesn't seem like there's anything to do?

@joeshaw

This comment has been minimized.

Show comment
Hide comment
@joeshaw

joeshaw Jun 13, 2018

Contributor

@wayneashleyberry The work to be done here is to put together a proposal.

As I see it there are at least 3 different possible approaches:

I don't see any way to implement this without adding some surface to the encoding/json package, unfortunately, whether it's a new interface type or a sentinel error value.

I like the IsZeroer interface, but as @bradfitz points out in https://go-review.googlesource.com/c/go/+/13977#message-54fb3469327f16170962c57896824a4cdd5cd541 that could be pretty expensive and I've not done any performance analysis on this. There are potential backward compatibility issues (or at least potentially surprising behavior) surrounding people's misunderstanding of the omitempty tag with structs.

I also like the sentinel value, but that only works for types that implement json.Marshaler. But for the most commonly-desired types like time.Time that's fine, but it doesn't solve the asymmetry of the fact that omitempty doesn't work out of the box on structs. And this sentinel error value would be orthogonal to omitempty, so it is potentially confusing in that respect.

So what this needs is someone to do some further analysis, implement these (and/or take my lame implementations further), see which is best and put forth a formal proposal. (But that's not going to be me, as my failure to do this in the past two years shows 😄)

Contributor

joeshaw commented Jun 13, 2018

@wayneashleyberry The work to be done here is to put together a proposal.

As I see it there are at least 3 different possible approaches:

I don't see any way to implement this without adding some surface to the encoding/json package, unfortunately, whether it's a new interface type or a sentinel error value.

I like the IsZeroer interface, but as @bradfitz points out in https://go-review.googlesource.com/c/go/+/13977#message-54fb3469327f16170962c57896824a4cdd5cd541 that could be pretty expensive and I've not done any performance analysis on this. There are potential backward compatibility issues (or at least potentially surprising behavior) surrounding people's misunderstanding of the omitempty tag with structs.

I also like the sentinel value, but that only works for types that implement json.Marshaler. But for the most commonly-desired types like time.Time that's fine, but it doesn't solve the asymmetry of the fact that omitempty doesn't work out of the box on structs. And this sentinel error value would be orthogonal to omitempty, so it is potentially confusing in that respect.

So what this needs is someone to do some further analysis, implement these (and/or take my lame implementations further), see which is best and put forth a formal proposal. (But that's not going to be me, as my failure to do this in the past two years shows 😄)

@mibk

This comment has been minimized.

Show comment
Hide comment
@mibk

mibk Jun 14, 2018

Contributor

Are there any experience reports concerning types other than time.Time? If this issue is mostly about time.Time, maybe we should focus on that rather than come up with a more general solution covering mostly only hypothetical issues.

Contributor

mibk commented Jun 14, 2018

Are there any experience reports concerning types other than time.Time? If this issue is mostly about time.Time, maybe we should focus on that rather than come up with a more general solution covering mostly only hypothetical issues.

@wayneashleyberry

This comment has been minimized.

Show comment
Hide comment
@wayneashleyberry

wayneashleyberry Jun 14, 2018

Contributor

@mibk I often run into this issue using sql.NullString, mysql.NullTime etc.

Contributor

wayneashleyberry commented Jun 14, 2018

@mibk I often run into this issue using sql.NullString, mysql.NullTime etc.

@mibk

This comment has been minimized.

Show comment
Hide comment
@mibk

mibk Jun 14, 2018

Contributor

@wayneashleyberry I see. But these cases could be handled using the third approach:

Whenever any struct would marshal to {}, omit it. This is an idea @rsc brought up in https://go-review.googlesource.com/c/go/+/13914#message-89b368b14fcdc0a03089aa34dc8185710144947f

which seems like a good idea regardless this issue. The third approach, IMO, also has the biggest potential to make it to the standard library.

Contributor

mibk commented Jun 14, 2018

@wayneashleyberry I see. But these cases could be handled using the third approach:

Whenever any struct would marshal to {}, omit it. This is an idea @rsc brought up in https://go-review.googlesource.com/c/go/+/13914#message-89b368b14fcdc0a03089aa34dc8185710144947f

which seems like a good idea regardless this issue. The third approach, IMO, also has the biggest potential to make it to the standard library.

@Xe

This comment has been minimized.

Show comment
Hide comment
@Xe

Xe Jul 2, 2018

Is this still being discussed for implementation? This just bit me today.

Xe commented Jul 2, 2018

Is this still being discussed for implementation? This just bit me today.

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Jul 2, 2018

Member

@Xe, the JSON and XML issues are on hold until at least Go 1.12. @rsc is focused on vgo at the moment.

Member

bradfitz commented Jul 2, 2018

@Xe, the JSON and XML issues are on hold until at least Go 1.12. @rsc is focused on vgo at the moment.

@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc Jul 23, 2018

Contributor

I hope to work on this for Go 1.12.

Contributor

rsc commented Jul 23, 2018

I hope to work on this for Go 1.12.

@wayneashleyberry

This comment has been minimized.

Show comment
Hide comment
@wayneashleyberry

wayneashleyberry Jul 24, 2018

Contributor

That just made my week @rsc, thank you so much 🙏

Contributor

wayneashleyberry commented Jul 24, 2018

That just made my week @rsc, thank you so much 🙏

@Xe

This comment has been minimized.

Show comment
Hide comment
@Xe

Xe Aug 1, 2018

@bradfitz would a CL to fix this issue be accepted?

Xe commented Aug 1, 2018

@bradfitz would a CL to fix this issue be accepted?

@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc Aug 13, 2018

Contributor

@Xe please hold off. We have not figured out the design yet. As I indicated above, I want to work on this for Go 1.12, probably in mid-September.

Contributor

rsc commented Aug 13, 2018

@Xe please hold off. We have not figured out the design yet. As I indicated above, I want to work on this for Go 1.12, probably in mid-September.

@dwoodlins

This comment has been minimized.

Show comment
Hide comment
@dwoodlins

dwoodlins Oct 12, 2018

@rsc Is it helpful to add a slightly different use case for this issue?

We use a custom ID type that is represented as a [16]byte. Our type implements the MarshalJSON() method to encode the byte array as a more readable string. I believe the same issue occurs when using UUIDs from either popular uuid library. A zero ID is represents as [16]byte{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0}, but that json serializes to [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0], so even hacks with unsafe.Pointer don't work.

If I'm reading this correctly, the third proposed solution (whenever any struct would marshal to {}, omit it.) would not work for this use case, at least not without requiring a wrapper struct. I'm personally a fan of using IsZero since we already use that extensively, but the sentinel value also seems to work.

dwoodlins commented Oct 12, 2018

@rsc Is it helpful to add a slightly different use case for this issue?

We use a custom ID type that is represented as a [16]byte. Our type implements the MarshalJSON() method to encode the byte array as a more readable string. I believe the same issue occurs when using UUIDs from either popular uuid library. A zero ID is represents as [16]byte{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0}, but that json serializes to [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0], so even hacks with unsafe.Pointer don't work.

If I'm reading this correctly, the third proposed solution (whenever any struct would marshal to {}, omit it.) would not work for this use case, at least not without requiring a wrapper struct. I'm personally a fan of using IsZero since we already use that extensively, but the sentinel value also seems to work.

@rsc rsc added the Proposal-Hold label Oct 17, 2018

@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc Oct 17, 2018

Contributor

On hold for JSON sweep.

Contributor

rsc commented Oct 17, 2018

On hold for JSON sweep.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment