Unable to Serialize Subclasses of "Document"

[robcthegeek]

Problem

I have a class that I want to store in Document DB. I subclassed this from Document because my Repository/Data Access layer cares about such things.

To be clear - it's something like:

public class MyDocument : Document {
    public string Custom { get; set; }
}

// After creating a document, I want to be able to do things like:
var doc = new MyDocument { Custom = "Something" };
var id = doc.Id;
var link = doc.SelfLink;
var custom = doc.Custom;

Within the code - all this appeared to be working fine, but when it came to serialization, my Custom property was being dropped.

I started to dig around and found that Document is a subclass of JsonSerializable - where there seems to be something amiss.

So - I got the code out to isolate it - the instance created from FromObject looks OK (you can see the Custom property) - but when serialized, the information is lost.
NOTE: We know this isn't a problem with JSON.NET - that handles subclasses fine.

Observe - I created the following in LINQpad (just add references to Azure.Documents.Client and Newtonsoft.Json and copy/paste):

void Main()
{
    var testDoc = new MyDocument {
        Custom = "Some Custom Property"
    };

    var test = FromObject(testDoc);
    test.Dump("Object Created from Document.FromObject - 'Custom' property visible:");
    JObject.FromObject(test).ToString().Dump("Serialized object - Y U NO SAVE?");
}

public class MyDocument : Document {
    public string Custom { get; set; }
}

internal static Document FromObject(object document) {
    if (document == null)
        return (Document) null;

    if (typeof (Document).IsAssignableFrom(document.GetType()))
        return (Document) document;

    JObject jobject = JObject.FromObject(document);
    Document document1 = new Document();
    // Private fields be damned!
    typeof(Document).GetField("propertyBag").SetValue(document1, jobject);
    // document1.propertyBag = jobject;
    return document1;
}

SUGGEST: I was really surprised to see this happening. Get Document to a POCO so we can subclass freely. It's serialization should be handled elsewhere.

PS: I'm not blocked - I can work around this.

[robcthegeek]

For those without LINQpad:

[ghost]

Hi, yes, this has been confirmed as a bug in the .NET SDK. It has been fixed and a new NuGet should be published soon. The workaround for now (which is not ideal) is to remove the inheritance from Document ... then the serialization works but then you have to deal with not having SelfLink, ETag, etc. on the object.

[robcthegeek]

Hi @ryancrawcour - thanks for the reply. Good to hear it's already been identified and fixed.

In the meantime - I've just created an IDocument interface with those values. When the object gets de-serialised, it all comes back correctly, so I'm back in business.

OOC - is there any reason the SDK's aren't published on ? It's just a API wrapper right? Would have happily put together a PR for you!

Thanks again!

[ghost]

It is definitely the intent to publish the full SDK on GitHub.

[ghost]

Unfortunately it appears the issue lies with the JSON.NET serialization process. Because our Document base class is a dynamic object, JSON.NET throws away all static properties that have not been decorated with [JsonProperty].

Back to the drawingboard to find a fix.

[robcthegeek]

I'm not so sure - I think the core problem is we have a base class of (what should be) a POCO messing with how it's serialised. I've never had problems serialising stuff with JSON.NET (and I've been forced to do some whacky stuff in the past).

I really think we just need to pull the serialisation up to it's own class. Before sending a resource - we send it to the serialiser as-is. JSON.NET will handle it fine, then we're golden.

Happy to hack if you get the code pushed to

[JSkimming]

Just weighing in on the discussion (Full disclosure: @robcthegeek and I are working together on this 😄)

I don't believe this is a JSON.NET serialization issue, the blow illustrates this (using LINQPad)

void Main()
{
    string serializeObject =
        JsonConvert.SerializeObject(new Test {Prop1 = "prop 1", Prop2 = "prop 2"});
    serializeObject.Dump();
}

class Test
{
    [JsonProperty("prop1")]
    public string Prop1 {get; set;}

    public string Prop2 {get; set;}
}

The output being below (NOTE the custom name is observed, and the non decorated property is serialized):

{"prop1":"prop 1","Prop2":"prop 2"}

As Rob referred to, putting serialization into the base class is, IMO, a poor separation of concerns. While I understand that having a base class is useful in providing functionality, the SelfLink for example, there are other ways of providing this. For instance, it could be provided by implementing an interface, or even by convention, i.e. by defining properties of a specific type and name, they will be populated.

Overall, I am wary of insisting on base classes for functionality, we only get 1 base class, so it needs to be chosen wisely.

[ghost]

The problem comes in because Document (our base class) implements an interface somewhere that makes it a DynamicObject. When you inherit from Document, your object becomes a DynamicObject as well.

Now, for some reason JSON.NET disregards all non-dynamic properties that are not decorated with JsonProperty. You don't have to set a name or anything, but you have to decorate the property. If you don't JSON.NET skips the property when it serializes to string. This is by design, according to the author.

None of the other JSON serializers do this. I checked about 6 others.

It is easy to reproduce. (not from LinqPad cause I don't own a license to it and therefore can't add NuGet packages)

class Program
{
    static void Main(string[] args)
    {
        var foo = new MyClass { MyDecoaratedProperty="something", MyUndecoratedProperty="something else"};
        Console.WriteLine(JsonConvert.SerializeObject(foo));
        Console.ReadLine();
    }
}

class MyClass : System.Dynamic.DynamicObject
{
    public string MyUndecoratedProperty { get; set; }
    [JsonProperty]
    public string MyDecoaratedProperty { get; set; }
}

This outputs:
{"MyDecoaratedProperty":"something"}

It has thrown away my non-decorated non-dynamic property

[ghost]

For now the easiest is to extend Resource, not Document. Resource is not a dynamic object and therefore doesn't suffer the same JSON.NET fate as Document does.

[felschr]

I have set JsonProperties on all my members that I want to serialize.
But I still had a strange issue that caused my objects to have BOTH instance members as well as dynamic(?) members under "Static members" -> "propertyBag" (seen in debugging mode).

The funny thing is when I change a property of one of those objects and serialize it the JSON will not have changed because it seems to retrieve the values from "propertyBag" which I can see hasn't changed.

It's extremely weird and I was working around it via some extra casting.
Now I noticed that when I changing my classes to use Resource instead of Document the "propertyBag" still existed but ONLY with fields like "_rid, etc. not my custom ones.

I'm not sure if all the DocumentDB operations have this issue but I noticed it on CreateDocumentQuery<T> with GetQueryResultAsync.

__
I'm writing here because this is the closest issue I was able to find online but maybe I'm just not searching for the right keywords? Would be nice to know if anyone here noticed the same behavior.