New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: encoding/json: add omitnil option #22480

Open
blixt opened this Issue Oct 28, 2017 · 12 comments

Comments

Projects
None yet
8 participants
@blixt
Contributor

blixt commented Oct 28, 2017

Note: This proposal already has as a patch from 2015 by @bakineggs, but it appears to have fallen between the cracks.

I have the following case:

type Join struct {
    ChannelId string      `json:"channel_id"`
    Accounts  []Ident     `json:"accounts,omitempty"`
    History   []TextEntry `json:"history,omitempty"`
}

This struct is used for message passing and the slices are only relevant (and set to non-nil) in some cases. However, since encoding/json does not differentiate between a nil slice and an empty slice, there will be legitimate cases where a field is excluded when it's not expected to (e.g., the History slice is set, but empty).

I reiterate the proposal by Dan in his patch referred above to support an omitnil option which allows this differentiation for slices and maps.

Note for hypothetical Go 2.0: This is already how omitempty works for pointers to Go's basic types (e.g., (*int)(nil) is omitted while pointer to 0 is not). For Go 2.0 the behavior of omitempty could change to omit both nil and 0 when specified, and then only nil would be omitted when omitnil is specified.

@robyoder

This comment has been minimized.

robyoder commented Jan 4, 2018

Yes please! This bit me today. We like to use omitempty to minimize the fields we need to include in a response. But we do need to differentiate between a nil slice and an empty slice, so we had to remove omitempty for slices and always include the field, so now it's an array or null. 🙁

@rsc

This comment has been minimized.

Contributor

rsc commented Jan 22, 2018

Partial dup of #11939, which we may get to someday. Mostly JSON is done but better handling of zeros is on the long-term wish list.

@rsc

This comment has been minimized.

Contributor

rsc commented Jan 22, 2018

Closing in favor of #11939, which now mentions this one.

@rsc rsc closed this Jan 22, 2018

@robyoder

This comment has been minimized.

robyoder commented Jan 29, 2018

@rsc how is this even a partial dupe of that issue? Its focus is defining additional zero values to be counted as "empty" by omitempty. This issue is focused on the fact that there is no way to differentiate between nil and empty slices with omitempty. How does #11939 address that?

@blixt

This comment has been minimized.

Contributor

blixt commented Jan 30, 2018

@rsc That proposal you referred seems to make omitempty exclude even more values from being encoded. This proposal is for doing the opposite – keep values that are today hidden from the output!

To clarify this proposal, here's an example. I want to be able to omit the Contents property for the Black Box (as it works today), but keep it for the Empty Box:

package main

import "encoding/json"
import "fmt"

type Box struct {
	Label    string
	Contents []string `json:",omitempty"`
}

func main() {
	var d []byte
	d, _ = json.Marshal(Box{Label: "Black Box"})
	fmt.Println(string(d))
	d, _ = json.Marshal(Box{Label: "Empty Box", Contents: []string{}})
	fmt.Println(string(d))
	d, _ = json.Marshal(Box{Label: "Banana Box", Contents: []string{"Banana", "Another banana"}})
	fmt.Println(string(d))
}

Here's the output I want, after changing omitempty to omitnil:

{"Label":"Black Box"}
{"Label":"Empty Box","Contents":[]}
{"Label":"Banana Box","Contents":["Banana","Another banana"]}

Actual output:

{"Label":"Black Box"}
{"Label":"Empty Box"}
{"Label":"Banana Box","Contents":["Banana","Another banana"]}
@blixt

This comment has been minimized.

Contributor

blixt commented Jan 30, 2018

Further clarification: The patch mentioned above adds support to encoding/json to differentiate on []string(nil) vs. []string{} (through a new keyword omitnil so backwards compatibility is not broken), allowing nil slices to be excluded without excluding empty slices.

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Jan 30, 2018

I guess I'll reopen this, but to be honest it sounds like a bad idea to me. Distinguishing between a nil slice and an empty slice is sufficiently confusing that it is generally a mistake. I don't think that the additional complexity in the already very complex encoding/json package is worth it.

@robyoder

This comment has been minimized.

robyoder commented Jan 30, 2018

@ianlancetaylor well, the encoder distinguishes. It encodes an empty slice as [] and a nil slice as null. But including a null property in the JSON is kinda pointless. Including [] is not.

@bakineggs

This comment has been minimized.

bakineggs commented Jan 30, 2018

In go, you shouldn't need to distinguish between a nil slice and an empty slice because they're generally treated the same. In JSON and many languages in which people parse JSON, null arrays and empty arrays are treated differently.

Ideally, the consumers of your JSON will be able to handle output that has either a null array, an empty array, or a missing key. In the real world, customers often demand that you supply output exactly how they want it. Product managers insist that engineers do the things that customers who pay boatloads of money want, so that's why I had to write the omitnil patch and run a product I built for a previous employer on a fork of go that includes this patch (and the poor souls who inherited that product have to continue maintaining that fork). It appears that my situation was not unique and others have to deal with similar demands, so it would be helpful if they didn't also have to maintain their own forks of go.

@rsc

This comment has been minimized.

Contributor

rsc commented Feb 5, 2018

On hold for #11939.

@rsc rsc added the Proposal-Hold label Feb 5, 2018

@jucardi

This comment has been minimized.

jucardi commented Nov 27, 2018

+1 to this request. I have a struct with a a field

Metadata interface{} `json:"metadata,omitempty,omitnil"`

but if the value is nil, it still serializes it as "metadata": null

@tariq1890

This comment has been minimized.

tariq1890 commented Dec 16, 2018

+1 to this request

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment