JsonTypeInfo is ignored when serializing a list of annotated object

[rabolfazl]

It seems that JsonTypeInfo is somehow ignored when serializing a list of annotated object. The following is some piece of code describing the situation.

@JsonTypeInfo(use = JsonTypeInfo.Id.CLASS, include = JsonTypeInfo.As.PROPERTY, property = "clazz")
public class ProjectElement extends BaseObject {
    private Long id = 10L;

    public Long getId() { return id; }
    public void setId(Long id) { this.id = id; }
}

public abstract class AbstractTask extends ProjectElement {
}

public class Task extends AbstractTask {
}

The following two pieces of code produce the results shown:

Task aTask = new Task();
objectMapper.writeValueAsString(aTask)

Outputs: {"id":10, "clazz":"Task"}

Task aTask = new Task();
List aList = new ArrayList();
aList.add(aTask);
objectMapper.writeValueAsString(aList);

Outputs: [{"id":10}]

"clazz" is not produced in the second code. why?

[cowtowncoder]

Yes and no: this is due Java Type Erasure. It is missing because element type information is not available to Jackson; all it sees is List<?>, and from that base type (used for finding @JsonTypeInfo) is not available.
Base type must be statically accessed, to use uniforma settings, unlike actual content serializer.

So question then is how to pass the extra information needed to detect the type.
There are two main alternatives:

1. Use a helper class like class TaskList extends ArrayList<Task> { } to "save" the type; if so, generic element type is available from super-class (one of oddities of type erasure)
2. Construct ObjectWriter with specific type: mapper.writerWithType(listTypeConstructedViaTypeFactory).writeValueAsString()

Note that type erasure is only problematic for root values (value directly passed to ObjectMapper). Because of this, I recommend not using Lists or Maps (or any generic types) as root values. They can be made to work, but are more hassle.

[rabolfazl]

Thanks a lot. I think I made a mistake adding an issue for it. Sorry.
But just one more thing to ask. Why not real object class (as returned by
object.getClass()) is used to detect proper serializer and checking the
super hierarchy for JsonTypeInfo? I mean why serializer does not do
something like this (hypothetically):

String s = "[";
for (Object o: theListToSerialize) {
     s +=

objectMapper.writeValueAsStringWithJsonTypeInfoConsideration(o,
getJsonTypeInfoFor(o.getClass()) + ",";
}
s += "]";

2013/10/28 Tatu Saloranta notifications@github.com

Yes and no: this is due Java Type Erasure. It is missing because element
type information is not available to Jackson; all it sees is List<?>, and
from that base type (used for finding @JsonTypeInfo) is not available.
Base type must be statically accessed, to use uniforma settings, unlike
actual content serializer.

So question then is how to pass the extra information needed to detect the
type.
There are two main alternatives:

1. Use a helper class like class TaskList extends ArrayList { }to "save" the type; if so, generic element type is available from
   super-class (one of oddities of type erasure)
2. Construct ObjectWriter with specific type:
   mapper.writerWithType(listTypeConstructedViaTypeFactory).writeValueAsString()

Note that type erasure is only problematic for root values (value directly
passed to ObjectMapper). Because of this, I recommend not using Lists or
Maps (or any generic types) as root values. They can be made to work, but
are more hassle.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/336#issuecomment-27228643
.

[cowtowncoder]

Np, it is a common problem unfortunately -- and it's not obvious from outset whose fault it is.

The reason is that when deserializing you will not know "real type" -- this is what @JsonTypeInfo is for -- so it is crucial to use exactly same base type. So while it would work for serialization (although much less efficiently, having to do additional lookups), it would not necessarily work for deserialization.

[Xnyle]

I just dug this up and accepted the answers (sort of): When adding enableDefaultTyping to the mapper, it suddenly adds type infos to the serialized stream, so why can't it use the same mechanism it uses to detect the base type in that case?

I mean not just adding the detected baseType but use that type to then search for the annotation on this type.

[cowtowncoder]

There are two problems: correctness for deserialization, and performance. Or perhaps just one, that of performance for both sides. Additional per-element lookups are significant overhead, and would take more time than actual reading, writing and data-binding of data. So at least by default this does not seem compelling. I am also not confident that changing behavior to allow potentially differing type information on serialization (possibly rare case, but I have noticed users using different @JsonTypeInfo on different parts of same type tree) would not cause additional new failure cases.

But all in all, I think I will add a note somewhere that says that Jackson recommends that users DO NOT try directly serializing any java.util.Collections or java.util.Maps; that is, that the root value would always be a POJO. Or at least never for polymorphic types.
Ultimately this is the problem, and no amount of work-arounds makes this usage reliable and intuitive. In addition to problems with JSON, it will also not work at all with XML.
So avoiding such usage will avoid majority of issues regarding type handling.

[Xnyle]

Performance and Deserialization is not really important for my use case: Javascipt frameworks like Angular or ExtJS which have some nice features where you may directly pipe stuff into sorted, filtered lists and little goblins do all the work for you. But this is working a lot better out of the box if the root node is an array. If not, you have to manually unwrap it yourself. The guys designing those stuff of course don't think about Java limitations.

Manually unwrapping is of course not a big problem, but still a bit annoying if you have to do it over and over again.

I can't use any of the available enableDefaultTyping options because then sublists get wrapped in an array which then breaks things at other places.

Couldn't the default behavior used when enableDefaultTyping is activated be enabled via an additional mapper option enableJsonTypeInfoProcessingOnRootContainers you explicitly have to enable? Then it wouldn't break anything.

If this is a feature that could be implemented by changing a few lines of code, I would really like to have it. If not I keep wrapping stuff :)

[Xnyle]

Or what also would help - at least for my use cases - enableNonRecursiveDefaultTyping

[cowtowncoder]

I guess I am not 100% sure what exactly is the problem in your case -- due to long history here, I may be assuming things.

But one thing about default typing is that determination of what kind of values should use type id is fully configurable. So it may be possible that you either need to use different standard criteria (one of pre-defined ones), or implement your own rule. Determination is never done on basis of hierarchy (there isn't enough information available to do that), but purely based on class in question.
So I suspect you may be able to figure out exact rules that work for you: for example, make sure Lists in question do not use type or so.

It might make sense to just file a new issue, and explain the problem with simple example; I realize that it is related to problem reported here, but it sounds like you have some additional structural differences.

[Xnyle]

Maybe I'm missing something, if I use DefaultTyping.JAVA_LANG_OBJECT which I understand shouldn't do anything to lists, I get this:

[ {
"@Class" : "my.Class",
"id" : "baa9a140-1ee7-4a88-9169-22eea6700934",
"someMap" : {
"someArray" : [ "java.util.ArrayList", [ "elementOfList" ] ]
}
}]

The root list is as I need it, but the someArray sublist inside the map gets wrapped. I don't want that and also don't understand why it is happening.

Is there an example how to "implement my own rule" that I may then use?

[cowtowncoder]

It does not do anything to the List, but it does apply it to things that it only knows as Object, from declared type.

So, for example, POJO like:

class POJO {
    public List<String> values;
}

would not get any type info.

But if you are passing root-level List, then it is typically only known as List<?>, and as such contents are indeed affected.

You can explicitly pass in type to consider for value, something like:

mapper.writerWithType(new TypeReference<List<String>>() {} ).writeValueAsString(list);

and then type is recognized as List<String>, in which case type information will not be added.

I guess your example goes bit deeper, but the same principle applies: declared type must be enough to determine base type, used for determining whether polymorphic handling applies (and if so, with what configuration).

One further caveat is with generic types. Jackson can handle type parameter resolution when there is enough information in most cases, but Java type erasure means that type is not always available when you think it is. Specifically, having parametric type like:

class Wrapper<T> {
    public List<T> values;
}

will NOT have any runtime type parameters, if instantiated with:

Wrapper<POJO> wrapper = new Wrapper<POJO>();

However, a subtype like:

class POJOWrapper extends Wrapper<POJO> { }

DOES have parameterization available. So only in latter case would List be known as List<POJO> (and not as List<?> as in first case).

If you can share specific class declaration(s) involved, I may be able to help add type information (either via ObjectWriter or in declarations).

[Xnyle]

Thanks, I understand what you are telling, but I don't understand how that applies to my example:

[/_this is in Java defined as List (not ArrayList) it is not wrapped as [ "java.util.ArrayList", actualList] */
{
"@Class" : "my.Class", /I only need this to be annotated, not types of Lists/
"id" : "baa9a140-1ee7-4a88-9169-22eea6700934",
"someMap" : { /definded as HashMap, so I understand why there is no type annotation/
"someArray" : [
/_this is also defined as List this time it is wrapped as [ "java.util.ArrayList", actualList] */
"java.util.ArrayList", [ "elementOfList" ] ]
}
}]

What I need is type annotations where possible without messing around with the datastructure.
What I get is two types of annotations:
At elements in non parameterized Lists (I like this one)
For the list itself if it is an abstract type (unless it's the root?) (This destroys my datastructure by wrapping the contents in an array) This is also happening when using DefaultTyping.JAVA_LANG_OBJECT, if I understand the doc correctly it should only happen when using OBJECT_AND_NON_CONCRETE ?

[cowtowncoder]

I can only help if you provide the class description, and piece of code you are using.
On terminology, I try to use "Type id" to mean additional JSON stuff (property or wrapper), just because annotations have specific meaning in Java (like using @JsonTypeInfo on class).

[Xnyle]

What I serialize is defined like this:
List rootList = new ArrayList<>();
HashMap<String,Object> map = new HashMap<>()
HashMap<String,Object> map2 = new HashMap<>()
rootList.add(map );
map.put("someMap",map2);
List subList = new ArrayList<>();
subList.add("elementOfList");
map2.put("someArray",subList);

Then I create an ObjectMapper and call mapper.enableDefaultTyping(DefaultTyping.JAVA_LANG_OBJECT);
mapper.writeValueAsString(rootList);

The output is the one above.

I can't easily change any of the rootLists and subLists type definitions, So I would like to come up with a solution where the Mapper creates "Type ids" only as properties but doesn't change the lists ([ "java.util.ArrayList", actualList]).

BTW: Thanks for your help

[cowtowncoder]

Ok. So, the root value you give will be consider to have type ArrayList<?>; which is not java.lang.Object, and there should NOT be type id (class name) serialized for it.
However, all contents will be of unknown type ('?'), which can only be considered to be java.lang.Object (as per Jackson's choice of uniform Collection values). Hence all values will get type id (with one exception: String -- I'll ignore that for now).

If you defined, say, root type like so:

mapper.writerForType(new TypeReference<List<Map<String,List<?>>>() { })
   .writeValueAsString();

type information would NOT be included for Map values, as they are now known to be of type Map<String,List<?>>; and so forth.
So if you can do this it can be used to guide type inclusion to know where to add type id.

But if you can not specify more specific typing, Jackson type id handling just might not work in your case, and you would possibly need to implement custom serializer, deserializer.

[Xnyle]

I guess I'll have to do that as maps/lists/simple values occur mixed in maps/lists.
May I create a Serializer for List.class and that will work for all Lists? I only found examples for non abstract types.

Where is the wrapping actually done? I looked at the code of CollectionSerializer which calls JsonGenerator for start and end tags, but I couldn't find anything about that in JsonGeneratorImpl

[cowtowncoder]

Registration varies a bit; for serialization, look up will be using actual runtime type (unless configured explicitly to use declared static type). For deserialization declared type is needed; and you will have same dualism so that you will actually need to register deserializer for java.lang.Object.
But registration in both cases is for 'raw' type, so all Lists will use registration for same handler (no difference between List<String>, List<POJO> etc).
If you register Module, you can define custom Serializers and Deserializers object which are called to find any and all (de)serializers. This way you can dynamically attach (de)serializers for all kinds of types.

Another possibility, which might end up being less work would be to do 2-phase processing: first read as JsonNode, then traverse, extract (and remove) type id, call mapper.convertValue(valueNode, targetType). While some work it would probably end up simpler than using custom deserializers, because you have full access to all information.

Actually you can also combine approaches; deserializer can call parser.readValueAsTree() and operate on tree instead of streaming parser.

[Xnyle]

Thanks again :)

I never need to deserialize anything, I would like to create a new module with a modified CollectionSerializer stolen from jackson-databind. I just didn't find the place where the wrapping is currently done:

https://github.com/FasterXML/jackson-databind/blob/master/src/main/java/com/fasterxml/jackson/databind/ser/std/CollectionSerializer.java#L91

I guess it is done somewere in JsonGenerator?

Serialization is based on runtime type. I understand :)

But my question was will module.addSerializer(Collection.class, new StolenCollectionSerializer()) override all the default serializers for all subtypes of Collection? Or an example:

module.addSerializer(Collection.class, new MyCollectionSerializer())
module.addSerializer(List.class, new MyListSerializer())

Which one will be elected for lets say runtime type ArrayList?

[cowtowncoder]

No, JsonGenerator does not get involved with data-binding in any way, it is strictly streaming-only.

As addSerializer, have a look at SimpleModule (and from there SimpleSerializers). It is very simple, and while it does handle sub-types, will not work for Collections as initialization and handling of structured types is a bit involved.

So in your case you will need to implement Module (or sub-class methods of SimpleModule), implement Serializers callback (there is something like Serializers.Base or .Std or such), and fill in callback. In that method you are free to use any logic you want to match type.
In case of Collections, further, you will need to resolve handlers (JsonSerializer, and, if you want to properly support polymorphic values, TypeSerializer as well). That is, unless you only limit handling for types where you do not need to delegate any calls.

At this point let's switch over to Jackson dev list -- github issues are fine for bug reports etc, but in this case this is kind of documentation that others may find useful and be able to able with.

[goldenrat]

Don't you think this will work ?

public abstract class AbstractTask {
     @JsonProperty("clazz")
     public String getClassName() {
         return this.getClass().getName();
     }
}

This will force the even with generics you will always have clazz in your key

[cowtowncoder]

@goldenrat That will simply add type-erased Class, which does not contain generic type information.

[jgribonvald]

I think I'm having the same problem, but how to solve it ?
here are the details of my problem : http://stackoverflow.com/questions/37663404/jackson-xml-problems-on-serializing

[cowtowncoder]

@jgribonvald Please either discuss this on user mailing list, or create a new issue (if it's for XML, for jackson-dataformat-xml; if you can confirm it also affects JSON, here on jackson-databind).
Comments on closed issued should be limited to exactly same problem, and often similar symptoms may be produced by different underlying root causes. In addition even if issues are related release notes are easiest to follow when history of issue does not get complicated with resolution for one version but some followup work later on.

Thanks!