Ability to not encode key/values with nil value

[midas]

I have searched the code base and documentation and have not found an option for this behavior. Does this feature already exist and I am missing it. If not, is it something you can envision adding to Poison? I would be willing to work on the feature if you are open to a pull request.

Thanks for your work on this project!

[devinus]

Poison encodes nil as null:

iex(1)> Poison.encode(%{foo: nil})
{:ok, "{\"foo\":null}"}

[midas]

I understand. What I would like to do is not serialize foo because it has a nil value. So for instance:

Poison.encode(%{foo: nil, bar: "baz"})
{:ok, "{\"bar\":\"baz"}}

There is really no need to serialize the nil values.

Thanks!

[devinus]

@midas What's your use case? A nil value does represents something.

[midas]

My case is a mobile app in which bandwidth is usually very constrained (not a common smart phone use case). Thus, limiting the representation sent down the pipe to only attributes with non-nil values is desirable and not premature optimization.

While I agree with you a nil value does represent something, a communication protocol can agree that the exclusion of a known attribute is understood to be null, which is what most users of JSON assume.

I understand that I could use maps (first recursively filtering them of nil values) instead of the structs deriving functionality from Poison, but it would be nice to use the same data structures for encoding and decoding and have the option to not serialize null values in order to keep my code base DRYer. Additionally, every JSON serialization library I have used have all provided this functionality, so it seems like a good feature for Poison. I do not think I will be the only person to desire the feature. Thanks!

[devinus]

@midas Link me to some examples of other libraries providing the feature so I can check out their API.

[midas]

Representable does not render nil values by default.

https://github.com/apotonick/representable#false-and-nil-values

Representable also has the ability to dynamically filter rendering, which is useful in other use cases, but could also be used for my use case.

https://github.com/apotonick/representable#skip-rendering

Rabl uses a configuration option called exclude_nil_values.

https://github.com/nesquena/rabl#configuration

ActiveModel serializers allows you to override the attributes method and return only those attributes you wish to serialize. You can filter the nil values here.

Gson does not render null values by default. In order to get it to render null you call a method in the chain: Gson gson = new GsonBuilder().serializeNulls().create();

https://github.com/google/gson/blob/master/UserGuide.md#null-object-support

Thanks!

[ericmj]

The only json library you linked seems to be Gson, the others are object mappers, serializers or whatever they are called.

In elixir you should be able to filter nils from your data structures in very few lines of code so I don't see the need for this being implemented in Poison which is a json library optimized for performance. Every option like this that needs to be supported means we will get less performance out of Poison.

[midas]

OK. Thanks for taking the time to consider.

[mustafaturan]

I know this is old but anyone who needs to remove all nil values, the approach I implemented below will solve the problem with Regex module.

Regex.replace(~r/\"([^\"]+)\":null(,?)/, Poison.encode!(%{foo: nil, bar: "baz"}), "")
=> "{\"bar\":\"baz\"}"

[defsprite]

I think doing this via string processing is fairly brittle, so here is a better alternative for posterity:

  def to_compact_map(map) do
    map
    |> Enum.reject(fn({_, v}) -> v == nil end)
    |> Enum.into(%{})
  end

(This does not work for nested maps though)

[meoyawn]

This is the code I came up with for recursive nil pruning:

defguard is_struct(term) when is_map(term) and :erlang.is_map_key(:__struct__, term)

@spec reducer({any, any}, map) :: map
defp reducer({k, v}, map) when is_map(v), do: Map.put(map, k, prune_nils(v))
defp reducer({_k, v}, map) when is_nil(v), do: map
defp reducer({k, v}, map), do: Map.put(map, k, v)

@spec prune_nils(map) :: map
def prune_nils(s) when is_struct(s), do: s |> Map.from_struct() |> Enum.reduce(%{}, &reducer/2)
def prune_nils(m) when is_map(m), do: Enum.reduce(m, %{}, &reducer/2)

[x-ji]

Thanks @meoyawn . This doesn't seem to deal with lists that contain maps though, e.g.

%{"k" => [%{"k1" => 1, "k2" => nil}, %{"k1" => 1, "k2" => nil}]}

I added the following two clauses

  def prune_nils(l) when is_list(l), do: Enum.map(l, fn entry -> prune_nils(entry) end)
  def prune_nils(v), do: v

which I hope work.