easyjson unmarshals empty slice [] into nil slice instead of empty slice, which in turn will be marshalled to null instead of empty slice.

[WUMUXIAN]

Hi, I've noticed some strange behaviour recently. See below for the details.

First of all, I define the following type:

type Attributes map[string]interface{}

This allows me to store a key-value set with the value being anything I like.

I then give this some values like this:

var attributes = Attributes{
    "key1": "value1",
    "key2": 2,
    "key3": []string{"v31", "v32"},
   "key4":  []string{},
}

If I marshal this into JSON, it looks like:

bytes, _ = easyjson.Marshal(attributes)
fmt.Println("Marshaled JSON:", string(bytes))

output json:
{
  "key1": "value1",
  "key2": 2,
  "key3": [
    "v31",
    "v32"
  ],
  "key4": []
}

If I store this JSON string somewhere, e.g. database and use it later. When I read it out and unmarshal it back to Attributes, it looks like this:

easyjson.Unmarshal(bytes, &attributes)
fmt.Println("easyjson marshalled attributes:", attributes)
fmt.Println("attributes.key4 is nil?", attributes["key4"].([]interface{}) == nil)

output:
easyjson marshalled attributes: map[key4:[] key1:value1 key2:2 key3:[v31 v32]]
attributes.key4 is nil? true

Now the value of key4 becomes nil, instead of empty slice. This is problematic since if I marshal it again, it becomes "null" in the JSON:

bytes, _ = easyjson.Marshal(attributes)
fmt.Println("Marshaled JSON Again:", string(bytes))

output:
Marshaled JSON Again: {"key2":2,"key3":["v31","v32"],"key4":null,"key1":"value1"}

If I use encoding/json library instead of easyjson, it works ok. See below:

bytes, _ = json.Marshal(attributes)
fmt.Println("Marshaled JSON:", string(bytes))

json.Unmarshal(bytes, &attributes)
fmt.Println("easyjson marshalled attributes:", attributes)
fmt.Println("attributes.key4 is nil?", attributes["key4"].([]interface{}) == nil)

bytes, _ = json.Marshal(attributes)
fmt.Println("Marshaled JSON Again:", string(bytes))

output:

{"type":"Feature","properties":{"name":"Connexis","address":"1 Fusionopolis Way"},"geometry":{"type":"Point","coordinates":[1.29878,103.787668]}}
Marshaled JSON: {"key1":"value1","key2":2,"key3":["v31","v32"],"key4":[]}
easyjson marshalled attributes: map[key4:[] key1:value1 key2:2 key3:[v31 v32]]
attributes.key4 is nil? false
Marshaled JSON Again: {"key1":"value1","key2":2,"key3":["v31","v32"],"key4":[]}

It's consistent, empty slice is always empty slice, not nil, and null.

[shmel1k]

https://github.com/mailru/easyjson/blob/master/jwriter/writer.go#L17

You can create your own jwriter.Writer with this flags.

[shmel1k]

@rvasily can you close?

[james-lawrence]

@shmel1k I don't think that is a valid answer. an empty array shouldn't be marshalled as null.

[shmel1k]

@james-lawrence https://github.com/mailru/easyjson/blob/master/tests/basic_test.go#L176

If I remember correctly, this behavior is supported. You can marshal as null or as an empty slice. The comment above explains this.

[WUMUXIAN]

@shmel1k, thanks for the explanation. Is there a better way to do this? Without tampering the source code. Because creating my own jwriter.Writer will require changing code in helper.go in the package right?

[WUMUXIAN]

@shmel1k .
Hi I just realised that actually the problem I had and illustrated above is not relevant to the json Marshalling nor the jwriter. It's about the Unmarshalling.

If a map is defined as map[string]interface{} and if it's Unmarshalled from a json like this:

{
  "key4": []
}

The map should have a key-value pair with this (And this is what happens when using json.Unmarshal:

• "key": "key4"
• "value": []interface{}{}

However when using easyjson.Unmarshal, it becomes:

• "key": "key4"
• "value": []interface{}{nil}

In a nutshell, the unmashalling should Marshal '[]' to an empty slice, instead of a nil slice. Because otherwise, even I set the Flags in the jwriter, it simply doesn't help. The value for "key4" will be null if I Marshal the map back to JSON.

The following changes to the source code will solution this problem:

change the line at

https://github.com/mailru/easyjson/blob/master/jlexer/lexer.go#L1154

from

var ret []interface{}

to

ret := []interface{}{}

This will align the easyjson.Unmarshal behaviour with the json.Unmarshal when dealing with empty slice value in a map.

[shmel1k]

Yeah, I agree that you're right :)

You can send pull-request for this fix

[WUMUXIAN]

Cool. I have made a pull request, #190
Please review, thanks.

[shmel1k]

I do not have permissions to merge, so ask @rvasily

[smb158]

I know this PR is kind of old but it fixes the issue my team is facing right now. Is there anyway we can get this merged in @rvasily? Thanks very much for your time!

[rvasily]

#190 merged