Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Duplicate entry or exception during serialization if JSON.NET has no way to set a property #2049

Open
JonPSmith opened this issue Apr 20, 2019 · 2 comments

Comments

Projects
None yet
1 participant
@JonPSmith
Copy link

commented Apr 20, 2019

Summary

I wanted to serialize classes that follow the Domain-Driven Design approach, i.e. properties have private setters. This didn't work and after a lot of digging I think there is a problem in JSON.NET (or maybe deeper - see orleans issue 5217.

Here is the summary of what I have found:

I have the following arrangement of classes (code shown later)

|--------| 0..1     1 |--------------| 1     0..1
| Book 1 | -----------| ManyToMany 1 |------------|
|--------|            |--------------|      |-----|-----|
                                            |  Author 1 |
|--------| 0..1    1  |--------------|      |-----|-----|
| Book 2 | -----------| ManyToMany 2 |------------|
|--------|            |------------- | 1      0..1

Summary of what I found:

  1. If the ManyToMany relationship properties have private setters then I get a "Self referencing loop detected" exception (details below).
  2. If the ManyToMany relationship properties have public setters then this work.
  3. If I set ReferenceLoopHandling = ReferenceLoopHandling.Ignore then it works with private setters in ManyToMany class, but duplicates Book 2.

UPDATE

I now know why this happens. Json.Net checks that a property can be set before it will serialize the data. This causes the two problems that are described below.

The way around this problem is to provide at serialization time a setting like ContractResolver = new ResolvePrivateSetters() or Json.NET attributes on constructors or properties to allow the property to be set.

Example 1 - fails with exception

The problem with this is the exception message gives no hint that the problem is because the properties cannot be set. Some more clearer message would help a lot.

// Put the types you are serializing or deserializing here
public class TestBook
{
    private TestBook() {}

    private readonly HashSet<ManyToMany> _many;

    public TestBook(ManyToMany many)
    {
        _many = new HashSet<ManyToMany>{many};
    }

    public int TestBookId { get; set; }
    public IEnumerable<ManyToMany> Many => _many.ToList();
}

public class TestAuthor
{
    public int TestAuthorId { get; set; }
    public ICollection<ManyToMany> Many { get; set; }
}

public class ManyToMany
{
    public int TestBookId { get; private set; }
    public int TestAuthorId { get; private set; }
    public TestBook BookLink { get; private set; }
    public TestAuthor AuthorLink { get; private set; }

    public void SetBookAuthor(TestBook book, TestAuthor author)
    {
        BookLink = book;
        AuthorLink = author;
    }
}

Now the unit test

[Fact]
public void TestJsonSerializeNotInDatabase()
{
    //SETUP
    var entities = GetLinkEntities();

    //ATTEMPT
    var json = JsonConvert.SerializeObject(entities, new JsonSerializerSettings()
    {
        PreserveReferencesHandling = PreserveReferencesHandling.Objects,
        Formatting = Formatting.Indented
    });

    //FAILS WITH EXCEPTION
}

private List<TestBook> GetLinkEntities()
{
    var many1 = new ManyToMany();
    var many2 = new ManyToMany();
    var book1 = new TestBook(many1);
    var book2 = new TestBook(many2);
    var author = new TestAuthor { TestAuthorId = 3 };
    many1.SetBookAuthor(book1, author);
    many2.SetBookAuthor(book2, author);
    author.Many = new List<ManyToMany> { many1, many2 };

    return new List<TestBook> { book1, book2 };
}

Exception

Newtonsoft.Json.JsonSerializationException : Self referencing loop detected for property 'AuthorLink' with type 'Test.UnitTests.TestDataResetter.TestDuplicateObjectInJsonSerialize+TestAuthor'. Path '[0].Many[0].AuthorLink.Many[1]'.
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CheckForCircularReference(JsonWriter writer, Object value, JsonProperty property, JsonContract contract, JsonContainerContract containerContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CalculatePropertyValues(JsonWriter writer, Object value, JsonContainerContract contract, JsonProperty member, JsonProperty property, JsonContract& memberContract, Object& memberValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(JsonWriter writer, IEnumerable values, JsonArrayContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(JsonWriter writer, IEnumerable values, JsonArrayContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(JsonWriter writer, IEnumerable values, JsonArrayContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.JsonConvert.SerializeObjectInternal(Object value, Type type, JsonSerializer jsonSerializer)
   at Test.UnitTests.TestDataResetter.TestDuplicateObjectInJsonSerialize.TestJsonSerializeNotInDatabase() in C:\Users\Jon\Source\Repos\EfCore.TestSupport\Test\UnitTests\TestDataResetter\TestDuplicateObjectInJsonSerialize.cs:line 32

Example 2 - runs correctly

All I have to do is change the two linking properties in ManyToMany class to have public setters, e.g.

public class ManyToMany
{
    public int TestBookId { get; private set; }
    public int TestAuthorId { get; private set; }
    public TestBook BookLink { get;  set; }
    public TestAuthor AuthorLink { get;  set; }

    public void SetBookAuthor(TestBook book, TestAuthor author)
    {
        BookLink = book;
        AuthorLink = author;
    }
}

All the rest of the code is the same as in example 1.

Here is the correct serialization (note that there are only two Book objects).

[
  {
    "$id": "1",
    "TestBookId": 0,
    "Many": [
      {
        "$id": "2",
        "TestBookId": 0,
        "TestAuthorId": 0,
        "BookLink": {
          "$ref": "1"
        },
        "AuthorLink": {
          "$id": "3",
          "TestAuthorId": 3,
          "Many": [
            {
              "$ref": "2"
            },
            {
              "$id": "4",
              "TestBookId": 0,
              "TestAuthorId": 0,
              "BookLink": {
                "$id": "5",
                "TestBookId": 0,
                "Many": [
                  {
                    "$ref": "4"
                  }
                ]
              },
              "AuthorLink": {
                "$ref": "3"
              }
            }
          ]
        }
      }
    ]
  },
  {
    "$ref": "5"
  }
]

Example 3 - ReferenceLoopHandling.Ignore causes duplicate

If I set ReferenceLoopHandling = ReferenceLoopHandling.Ignore then it doesn't throw an exception, but it duplicates Book 2. This happens because JSON.NET doesn't have a way to set the linking properties

Here is the unit test using the same data and classes shown in example 1, i.e. the ManyToMany's BookLink and AuthorLink have private setters.

[Fact]
public void TestJsonSerializeNotInDatabaseReferenceLoopHandlingIgnore()
{
    //SETUP
    var entities = GetLinkEntities();

    //ATTEMPT
    var json = JsonConvert.SerializeObject(entities, new JsonSerializerSettings()
    {
        PreserveReferencesHandling = PreserveReferencesHandling.Objects,
        ReferenceLoopHandling = ReferenceLoopHandling.Ignore,
        Formatting = Formatting.Indented
    });

    _output.WriteLine(json);
}

The JSON output looks like this - notes

  • There are three books in the JSON (the second one is duplicated)
  • If I change the ManyToMany's BookLink and AuthorLink public setters it works correctly, i.e. no duplicate Book 2
[
  {
    "$id": "1",
    "TestBookId": 0,
    "Many": [
      {
        "$id": "2",
        "TestBookId": 0,
        "TestAuthorId": 0,
        "BookLink": {
          "$ref": "1"
        },
        "AuthorLink": {
          "TestAuthorId": 1,
          "Many": [
            {
              "$ref": "2"
            },
            {
              "$id": "3",
              "TestBookId": 0,
              "TestAuthorId": 0,
              "BookLink": {
                "TestBookId": 0,
                "Many": [
                  {
                    "$ref": "3"
                  }
                ]
              }
            }
          ]
        }
      }
    ]
  },
  {
    "$id": "4",
    "TestBookId": 0,
    "Many": [
      {
        "$ref": "3"
      }
    ]
  }
]

Using JSON.NET 12.0.1 in NET Core 2.2.1

@JonPSmith

This comment has been minimized.

Copy link
Author

commented May 31, 2019

Hi,

I have tried to diagnose the problem in your package so that I could do a pull request. Unfortunately I couldn't follow the serialization handling of the circular references. However I did write a unit test in your format that I have provided below. You may find this useful in determining what the problem is.

#region License
// Copyright (c) 2007 James Newton-King
//
// Permission is hereby granted, free of charge, to any person
// obtaining a copy of this software and associated documentation
// files (the "Software"), to deal in the Software without
// restriction, including without limitation the rights to use,
// copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the
// Software is furnished to do so, subject to the following
// conditions:
//
// The above copyright notice and this permission notice shall be
// included in all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
// EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
// OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
// HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
// WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
// FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
// OTHER DEALINGS IN THE SOFTWARE.
#endregion

using Newtonsoft.Json.Linq;
using Newtonsoft.Json.Linq.JsonPath;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using Newtonsoft.Json.Serialization;
#if DNXCORE50
using Xunit;
using Test = Xunit.FactAttribute;
using Assert = Newtonsoft.Json.Tests.XUnitAssert;
#else
using NUnit.Framework;
#endif

namespace Newtonsoft.Json.Tests.Issues
{
    [TestFixture]
    public class Issue2049
    {
        [Test]
        public void TestJsonSerialize()
        {
            //SETUP
            var entities = GetLinkEntities();

            //ATTEMPT
            var json = JsonConvert.SerializeObject(entities, new JsonSerializerSettings()
            {
                PreserveReferencesHandling = PreserveReferencesHandling.Objects,
                ReferenceLoopHandling = ReferenceLoopHandling.Ignore,
                Formatting = Formatting.Indented
            });

            //VERIFY
            //This test finds Book2 by its title. There should be only one "Book2", but this shows there are 2 
            Assert.AreEqual(1, json.Split('\n').Count(x => x.Trim() == "\"Title\": \"Book2\","));
        }

        //-----------------------------------------------------------------
        //classes and setups for test

        private List<TestBook> GetLinkEntities()
        {
            var many1 = new ManyToMany();
            var many2 = new ManyToMany();
            var book1 = new TestBook("Book1", many1);
            var book2 = new TestBook("Book2", many2);
            var author1 = new TestAuthor("Author");
            many1.SetBookAuthor(book1, author1);
            many2.SetBookAuthor(book2, author1);

            author1.Many = new List<ManyToMany> { many1, many2 };

            return new List<TestBook> { book1, book2 };
        }

        public class TestBook
        {
            private TestBook() { }

            private readonly HashSet<ManyToMany> _many;

            public TestBook(string title, ManyToMany many)
            {
                Title = title;
                _many = new HashSet<ManyToMany> { many };
            }

            public int TestBookId { get; set; }
            public string Title { get; set; }
            public IEnumerable<ManyToMany> Many => _many.ToList();
        }

        public class TestAuthor
        {
            public TestAuthor(string name)
            {
                Name = name;
            }

            public int TestAuthorId { get; set; }

            public string Name { get; set; }
            public ICollection<ManyToMany> Many { get; set; }
        }

        public class ManyToMany
        {
            public int TestBookId { get; private set; }
            public int TestAuthorId { get; private set; }
            public TestBook BookLink { get; private set; }
            public TestAuthor AuthorLink { get; private set; }

            public void SetBookAuthor(TestBook book, TestAuthor author)
            {
                BookLink = book;
                AuthorLink = author;
            }
        }
    }
}

@JonPSmith JonPSmith changed the title "JsonSerializationException : Self referencing loop detected" caused by private setter (?!) Duplicate entry in serialization if JSON.NET has no way to set a property Jun 3, 2019

@JonPSmith JonPSmith changed the title Duplicate entry in serialization if JSON.NET has no way to set a property Duplicate entry or exception during serialization if JSON.NET has no way to set a property Jun 3, 2019

@JonPSmith

This comment has been minimized.

Copy link
Author

commented Jun 3, 2019

I have now diagnosed the problem and changed the issue's title to match. I have updated the initial statement of the problem to detail what fixes this problem, i.e. providing a method where Json.NET can set the property at serialization time.

However I still believe producing a duplicate at serialization is a bug (I also think the exception message could be better too).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.