Serializing/Deserializing simple maps should be better supported

[GoogleCodeExporter]

Attempting to simply serialize a map results in fairly useless output:

// dead simple map
Map<String, Object> m = new ArrayOrderedMap<String, Object>();
m.put("id", 123);
m.put("thing", "AZ");

// serialize (annoyingly requires the typetoken thing)
String encoded = gson.toJson(data, new TypeToken<Map<String, Object>>()
{}.getType());

encoded = {"id":{},"thing":{}}

It should have been:
encoded = {"id":123,"thing":"AZ"}

This should really be able to handle the simple example of a map of
primitive/simple objects much better. The same type of thing happens when
attempting to read back in the data.

With a more realistic example it is even worse because another map placed
inside the first map results in an exception like so:
com.google.gson.JsonParseException: The JsonSerializer
com.google.gson.DefaultTypeAdapters$MapTypeAdapter@b27bb5 failed to
serialized object {name=aaron, date=Mon Sep 15 11:58:33 BST 2008, num=456,
array=[Ljava.lang.String;@fe3238} given the type class java.lang.Object
    at
com.google.gson.JsonSerializerExceptionWrapper.serialize(JsonSerializerException
Wrapper.java:61)
    at
com.google.gson.JsonSerializationVisitor.visitUsingCustomHandler(JsonSerializati
onVisitor.java:177)
    at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:144)
    at
com.google.gson.JsonSerializationContextDefault.serialize(JsonSerializationConte
xtDefault.java:47)
    at
com.google.gson.DefaultTypeAdapters$MapTypeAdapter.serialize(DefaultTypeAdapters
.java:301)
    at
com.google.gson.DefaultTypeAdapters$MapTypeAdapter.serialize(DefaultTypeAdapters
.java:293)
    at
com.google.gson.JsonSerializerExceptionWrapper.serialize(JsonSerializerException
Wrapper.java:48)
    at
com.google.gson.JsonSerializationVisitor.visitUsingCustomHandler(JsonSerializati
onVisitor.java:177)
    at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:144)
    at
com.google.gson.JsonSerializationContextDefault.serialize(JsonSerializationConte
xtDefault.java:47)
    at com.google.gson.Gson.toJson(Gson.java:272)
    at com.google.gson.Gson.toJson(Gson.java:228)
    at
org.sakaiproject.entitybroker.impl.EntityEncodingManager.encodeData(EntityEncodi
ngManager.java:586)
    at
org.sakaiproject.entitybroker.impl.EntityEncodingManagerTest.testEncode(EntityEn
codingManagerTest.java:243)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.jav
a:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at junit.framework.TestCase.runTest(TestCase.java:154)
    at junit.framework.TestCase.runBare(TestCase.java:127)
    at junit.framework.TestResult$1.protect(TestResult.java:106)
    at junit.framework.TestResult.runProtected(TestResult.java:124)
    at junit.framework.TestResult.run(TestResult.java:109)
    at junit.framework.TestCase.run(TestCase.java:118)
    at junit.framework.TestSuite.runTest(TestSuite.java:208)
    at junit.framework.TestSuite.run(TestSuite.java:203)
    at
org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestR
eference.java:130)
    at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner
.java:460)
    at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner
.java:673)
    at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java
:386)
    at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.jav
a:196)
Caused by: java.lang.IllegalArgumentException: Map objects need to be
parameterized unless you use a custom serializer. Use the
com.google.gson.reflect.TypeToken to extract the ParameterizedType.
    at com.google.gson.TypeInfoMap.<init>(TypeInfoMap.java:34)
    at
com.google.gson.DefaultTypeAdapters$MapTypeAdapter.serialize(DefaultTypeAdapters
.java:298)
    at
com.google.gson.DefaultTypeAdapters$MapTypeAdapter.serialize(DefaultTypeAdapters
.java:293)
    at
com.google.gson.JsonSerializerExceptionWrapper.serialize(JsonSerializerException
Wrapper.java:48)
    ... 31 more

Original issue reported on code.google.com by azeckoski on 15 Sep 2008 at 11:00

[GoogleCodeExporter]

Extra info on the deserializing:
json = {"id":123,"thing":"AZ"}

Map<String, Object> decoded = gson.fromJson(data, new TypeToken<Map<String,
Object>>() {}.getType());

produces a map with: {id=java.lang.Object@e6612c, thing=java.lang.Object@d704f0}
(seems to be instances of Object with no data)

Original comment by azeckoski on 15 Sep 2008 at 1:29

[GoogleCodeExporter]

First off, I'd like to start with some background information.  When you are 
defining
types (or local variables) that have type parameters, the JVM drops the actual 
type
parameters and associates everything as "Object".  This is known as "type 
erasure". 
In order for a Java Program to retrieve the actual type parameters at run-time, 
you
need to leverage the TypeToken object (this methodology was established by 
GUICE ---
see
http://google-guice.googlecode.com/svn/trunk/javadoc/com/google/inject/TypeLiter
al.html).

Originally, we implemented Gson so that it could "serialize" these kinds of 
objects
without requiring the use of TypeToken; however, deserializing it back into the 
real
object requires it since the JSON output has no type information in it.  As 
well,
with this approach, it meant you were serializing the "real" object which meant 
that
some fields on the real object would be added to the JSON output.  Therefore, 
if you
had a List<A> and added both A and B objects (i.e. B extends A) some objects 
would
expose extra fields in the output.  We decided to take the more explicit route 
and
force the client to provide us the type parameters of the top-level object that 
is
being serialized.

The common use of a Map or List is that you populate the List with the same 
object
types.  Passing in Object means that you can add any instance of a class that 
you
desire to the data structure.  I know there are exceptions to this best 
practice, but
we do not want to implement this corner case scenario.  Instead, if you really 
do
want to use a list of Objects, then do as the exception message says and write a
"custom" (de)serializer (you can bind it specifically to a Map<String, Object> 
and
have the default Gson map serializer handle everything else).

As for a Map of Maps (i.e. Map<String, Map<String, Integer>>) this is already
supported and works well as long as you pass in the actual type object (i.e. new
TypeToken<Map<String, Map<String, Integer>>>() {}.getType())

Here's an example:
public static void main(String[] args) {
  Type mapType = new TypeToken<Map<String, Map<String, Integer>>>() {}.getType();
  Map<String, Map<String, Integer>> map = new HashMap<String, Map<String, Integer>>();
  Map<String, Integer> value1 = new HashMap<String, Integer>();
  value1.put("lalala", 78);
  value1.put("haha", 9999);
  map.put("id", value1);

  Map<String, Integer> value2 = new HashMap<String, Integer>();
  value2.put("nahhd", 121112);
  value2.put("uuywss", 19987);
  map.put("thing", value2);

  Map<String, Integer> value3 = new HashMap<String, Integer>();
  map.put("other", value3);

  Gson gson = new Gson();
  String json = gson.toJson(map, mapType);
  System.out.println(json);

  Map<String, Map<String, Integer>> deserializedMap = gson.fromJson(json, mapType);
  System.out.println(deserializedMap);
}

=========== OUTPUT ===========
{"thing":456,"id":123}
{"other":{},"thing":{"nahhd":121112,"uuywss":19987},"id":{"lalala":78,"haha":999
9}}
{other={}, thing={nahhd=121112, uuywss=19987}, id={lalala=78, haha=9999}}


For now, I am closing this off as "Working as Designed".  Maybe I am not 
completely
following your issue and if that is the case, please start up a new discussion 
in our
Gson discussion group.

Thanks,
Joel

Original comment by joel.leitch@gmail.com on 16 Sep 2008 at 8:57

• Changed state: Invalid

[GoogleCodeExporter]

We solved our problem by using a different library but I wanted to put a 
comment here
anyway.

So what happens if I want to do this?
Map<String, Number>
or
Map<String, Serializable>

(it seems to fail)

It seems that this is designed to only work for the basic case where I have 
really
simple and non-nested structures where all the beans are easily instantiable 
and not
superclasses. It is a shame that this is considered working as designed.

Original comment by azeckoski on 18 Sep 2008 at 9:01

[GoogleCodeExporter]

I am glad to hear that you found something that works for you, but it's too bad 
you
are unable to use Gson.  I'd still like to follow up on this issue because it is
user's like yourself that will help to advance this library.

First off, are you serializing and deserializing an object of type Map<String,
Number>?  If it is serialization only, than that is a much "easier" problem to 
solve
because we have the runtime types.  As for "deserializing" this kind of object, 
we
have provided our clients with the concept of a custom "Type Adapter".  You 
should be
able to write a type as follows to get it to work with "Number":

  public static class NumberTypeAdapter 
      implements JsonSerializer<Number>, JsonDeserializer<Number>,
InstanceCreator<Number> {

    public JsonElement serialize(Number src, Type typeOfSrc, JsonSerializationContext
context) {
      return new JsonPrimitive(src);
    }

    public Number deserialize(JsonElement json, Type typeOfT,
JsonDeserializationContext context)
        throws JsonParseException {
      JsonPrimitive jsonPrimitive = json.getAsJsonPrimitive();
      if (jsonPrimitive.isNumber()) {
        return jsonPrimitive.getAsNumber();
      } else {
        throw new IllegalStateException("Expected a number field, but was " + json);
      }
    }

    public Number createInstance(Type type) {
      return 1L;
    }
  }

  public static void main(String[] args) {
    Map<String, Number> map = new HashMap<String, Number>();    
    map.put("int", 123);
    map.put("long", 1234567890123456789L);
    map.put("double", 1234.5678D);
    map.put("float", 1.2345F);
    Type mapType = new TypeToken<Map<String, Number>>() {}.getType();

    Gson gson = new GsonBuilder().registerTypeAdapter(Number.class, new
NumberTypeAdapter()).create();
    String json = gson.toJson(map, mapType);
    System.out.println(json);

    Map<String, Number> deserializedMap = gson.fromJson(json, mapType);
    System.out.println(deserializedMap);
  }

========== OUTPUT ==========
{"double":1234.5678,"float":1.2345,"int":123,"long":1234567890123456789}
{double=1234.5678, float=1.2345, int=123, long=1234567890123456789}


We should probably just include the above type adapter as a default in Gson and 
I
will discuss this with Inderjeet.  There is a bug, however, since you actually 
have
to specify a "instance creator" for this type of object (i.e. primitive), but I 
will
have that fixed by the next release.  You should be able to write something 
similar
as above for "Serializable".

I hope this information is helpful and thanks for the all the feedback on this 
library.

Original comment by joel.leitch@gmail.com on 27 Sep 2008 at 8:32

[GoogleCodeExporter]

I modify the MapTypeAdapter to match the jdk14's Map
======================================================
public JsonElement serialize(Map src, Type typeOfSrc, JsonSerializationContext 
context) {
      JsonObject map = new JsonObject();
      //Type childType = new TypeInfoMap(typeOfSrc).getValueType();
      for (Iterator iterator = src.entrySet().iterator(); iterator.hasNext(); ) {
        Map.Entry entry = (Map.Entry) iterator.next();
        Object obj = entry.getValue();
        JsonElement valueElement = context.serialize(obj, obj.getClass());
        ---------------------------------------------------------------------modified
        map.add(entry.getKey().toString(), valueElement);
      }
      return map;
    }
--------------------------------------------------------------------------------

and then the map class can be used like this:
  HashMap aaaa = new HashMap();
  aaaa.put("aa", 1212);
  aaaa.put("bb", "fasdfa");
  System.out.println(gson.toJson(aaaa));
==========output=================
{"bb":"fasdfa","aa":1212}
=================================

It can run, good or bad? because there are lot's of  jdk14's source code in many
project's.

Original comment by zhaojinz...@gmail.com on 21 Oct 2008 at 9:53

[GoogleCodeExporter]

Thanks for providing the code snippet. This will not work properly in case of 
genericized maps since in those cases it is important to use the type specified 
in 
the field declaration instead of the actual type. I have made similar fixes for 
Issue 
54 and 58 that I will apply in this case as well. 

Original comment by inder123 on 21 Oct 2008 at 3:32

[GoogleCodeExporter]

I have fixed this issue in r277 

Now, you should be able to serialize raw maps. The deserialization continues to 
require parameterized type. 

Original comment by inder123 on 21 Oct 2008 at 10:41

• Changed state: Fixed

[GoogleCodeExporter]

Thank you for fixing this issue: 
In java land, you really shouldn't be instantiating Map<String, Object> but 
since
we're dealing with JSON world, it actually makes a lot of sense. 
Consider Map<String, Object> map;
map.put("field1", 123);
map.put("field2", "myfield2");

What is gson.toJson(map)???
It's a javascript object o where o.field1 is the number 123 and o.field2 is the
string myfield2!

Original comment by SystemIn...@gmail.com on 22 Jan 2010 at 9:43

[GoogleCodeExporter]

json convert to map<Integer,MyClass> it have problem !
how to  do ?

Original comment by demog...@gmail.com on 6 Mar 2010 at 6:43

[GoogleCodeExporter]

If you do json eval() in javascript or python, you get a dictionary. Inside the 
dictionary, it has String/Number or nested dictionaries. eval doesnt expect 
these type declarations.
I would expect the same on static language as well - Maps with default Number 
(lossless datatype like Double) datatype for deserialization.

Original comment by mani.dor...@gtempaccount.com on 25 Jun 2010 at 6:12

[GoogleCodeExporter]

Hi demograp,

Did you forget to implement a default constructor for MyClass ?
It was my case, and I solved it doing this.

Hope it helps.

Original comment by kstruil...@gmail.com on 6 Aug 2010 at 2:45

[GoogleCodeExporter]

I faced similar problems. Easiest solution for me was wrapping the desired map 
in a wrapper object and passing that to gson. I guess in the end that just 
boils down to be the same as providing the TypeToken, but it is a much more 
straightforward solution for those who want a quick fix.

Original comment by dska...@gmail.com on 1 Sep 2011 at 12:23