OData Complex type in navigation property is null when expanded

[SiberaIndustries]

OData complex types in navigation properties are always serialized as null when the navigation property gets expanded. I am using Entity Framework Core in conjunction with ASP.NET Core + OData.

Assemblies affected

• Microsoft.AspNetCore.OData 7.2.1
• Microsoft.AspNetCore.OData 7.2.2
• Microsoft.AspNetCore.OData 7.3.0

Reproduce steps

I have exactly the same issue as described on Stackoverflow. So this example is translated from there.

// No Identifier = Owned Type in EF / Complex Type in OData
[Owned]
public class StreetAddress
{
    public string Street { get; set; }
    public string City { get; set; }
}

// Owns StreetAddress
public class Order
{
    public int Id { get; set; }
    public StreetAddress ShippingAddress { get; set; }
}

// Order collection as navigation property
public class Customer
{
    public int Id { get; set; }
    public string Name { get; set; }
    public Order Order { get; set; }
}

Expected result

The complex property ShippingAddress should NOT be null when the navigation property Order gets expanded with the request GET /Customers(1337)?$expand=Order:

{
    "Id": 1337,
    "Name": "Jon Snow",
    "Order": {
        "Id": 42,
        "ShippingAddress": {
          "Street": "Wall St",
          "City": "New York"
        }
    }
}

Actual result

The complex property ShippingAddress is ALWAYS null when the navigation property Order gets expanded with the request GET /Customers(1337)?$expand=Order:

{
    "Id": 1337,
    "Name": "Jon Snow",
    "Order": {
        "Id": 42,
        "ShippingAddress": null
    }
}

Additional detail

As a workaround it is possible to explicitly request every property with $select like GET /Customers(1337)?$expand=Order($select=Id,StreetAddress). Sadly $select=* is not working.

[KanishManuja-MS]

@Discjoggy I have gotten it added to the backlog since I was not able to spare time to figure out the issue. Someone will pick this up before the next release.

[SiberaIndustries]

That's okay. Nice to hear that this will be fixed in the next release somehow. If you need some more information or a sample implementation, just let me know, I thought the information here and on Stackoverflow would be enough.

[rvanmaanen]

A similar (but different) issue here:

Entity Employee

• has many Assignments

Entity Assignment

• has required Jobprofile (navigation property)
• has WorkingAmount (complex type)

Entity Jobprofile

• just has 2 properties

Strangely, in my case when selecting an Employee and expanding the Assignments the following happens:

• Assignment.Jobprofile = null
• Assignment.WorkingAmount = filled properly

So the navigation property is null (and removed from result through a custom ODataSerializerProvider) but the complex type is filled properly:

        {
            "employmentCode": 1,
            "startDate": "2009-10-22",
            "workingAmount": {
                "amountOfWork": 16,
                "unitOfWork": "Hours",
                "periodOfWork": "Week"
            }
        }
    ]

When I drop the Jobprofile controller and the corresponding configuration, effectively making Jobprofile just a complex type instead of an entity, it does show up in the result. Making it pretty clear that at least the rest of the implementation seems to be in order. Also interesting: Issue wasn't happening before we did the migration to .NET Core but almost all OData configuration has been rewritten so not sure if that is related. Models, repositories, mapping and other classes are pretty much identical though (Update 2: was a change in here actually, see bottom of post).

Also tried setting MaxExpansionDepth to 0 (and to 10) as someone suggested on SO, but that doesn't help.

Any suggestions on what to try?

Update: Ofcourse, just after posting this I realized that my query might be wrong. The query I was testing with:

/employees(1000)?$expand=assignments,employments

New query which does expand all entities, related entities and complex properties properly:

/employees(1000)?$expand=assignments($expand=Jobprofile,OrganizationUnit),employments($expand=Jobprofile,OrganizationUnit)

Guess I will look into auto expanding Jobprofiles and OrganizationUnits. This seems to be working as a final result for now. No other configuration in the models or wherever was needed to get everything working:

            var employee = builder.EntitySet<Employee>("Employees").EntityType;
            employee.HasKey(e => e.PersonCode);
            employee.HasMany(e => e.Employments);
            employee.HasMany(e => e.Assignments);
            employee.Select().Count().Expand().Page();

            var employment = builder.EntitySet<Employment>("Employments").EntityType;
            employment.HasKey(e => e.EmploymentCode);
            employment.HasRequired(e => e.JobProfile).AutoExpand = true;
            employment.HasRequired(e => e.OrganizationUnit).AutoExpand = true;
            employment.Select().Expand();

            [more of the same for Assignment, JobProfile and OrganizationUnit]

Sorry for the long post ;)

Also figured out what was happening in the old .NET Framework solution, where the expands appeared to be happening automatically: The team had 2 identical JobProfile classes and 2 identical OrganizationUnit classes in the codebase. One was acting as a complex type and the other was registered as an entity. Nasty.

[SiberaIndustries]

[..] Someone will pick this up before the next release.

Hm... 7.2.2 was released and this issue is still not fixed @KanishManuja-MS :/

[havotto]

I have also encountered this issue.
I tried with SQLite, and saw that the complex type's columns are missing from the SQL.

[havotto]

When using EF Core 3.1, this works as expected. Previously I tested it with 2.1

[DieDrachenfaenger]

We still see this issue with EF Core 3.1 and Microsoft.AspNet.OData.

[joelmeaders]

I am getting this same issue, but with complex properties & owned entities when I expand a navigation property and include a skip or top, even if skip is zero and top is not. When a skip and/or top is included EFC returns either a null on the complex/owned property, or a default if one is set in the model. This happens even if I don't include a nested expand.

EFCore 3.1.3
AspNetCore.OData 7.4.0-beta

Expected Result (I'm only including first record, not within the value array):
URL: https://localhost:5001/odata/qatemplatequestions?$filter=QaTemplateId%20eq%201c883fb0-7daa-4339-82ef-bd220fe65bfc&$expand=QaTemplateQuestionQaDocTags($expand=QaDocTag)&$orderby=Order%20asc&$top=30&$skip=0&$count=false&$format=application/json;odata.metadata=none

Click to expand

{
    "Id": "a4369749-89f6-46ec-beae-b1a3e88778a2",
    "QaTemplateId": "1c883fb0-7daa-4339-82ef-bd220fe65bfc",
    "Order": 1,
    "Text": "THis is a child question. It is a true/false with label overrides. It always moves forward. It has additional data at the initial stage",
    "CollectAdditionalDataAt": "Initial",
    "HelpText": "Here's some help text for the child question",
    "HelpLink": "https://google.com",
    "Active": true,
    "IsParent": false,
    "ParentId": "79663cff-8515-4685-884b-20066aa7f14f",
    "OptOpenText": {
      "Enabled": true,
      "Required": true,
      "Label": "Here's a required open text field."
    },
    "OptDateTime": {
      "Enabled": true,
      "Required": false,
      "Label": "Here's an optional date field"
    },
    "OptCurrency": {
      "Enabled": true,
      "Required": false,
      "Label": "Here's an optional currency field"
    },
    "TrueFalseConfig": {
      "Enabled": true,
      "MoveForwardOn": "Always",
      "OverrideTrueLabel": "Yes",
      "OverrideFalseLabel": "Nope!"
    },
    "QaTemplateQuestionQaDocTags": [
      {
        "Id": "a21f9cbb-77d2-43cd-8ea6-2d320a8753c4",
        "QaTemplateQuestionId": "a4369749-89f6-46ec-beae-b1a3e88778a2",
        "QaDocTagId": "320dcb36-601d-4fd3-74bf-08d762f6c0a0",
        "QaDocTag": {
          "Id": "320dcb36-601d-4fd3-74bf-08d762f6c0a0",
          "GroupName": "Tst1",
          "Text": "Doc1",
          "Active": true
        }
      },
      {
        "Id": "88d56142-b592-4301-b312-064086c063c9",
        "QaTemplateQuestionId": "a4369749-89f6-46ec-beae-b1a3e88778a2",
        "QaDocTagId": "9b085a36-4dc9-47af-74c0-08d762f6c0a0",
        "QaDocTag": {
          "Id": "9b085a36-4dc9-47af-74c0-08d762f6c0a0",
          "GroupName": "Tst1",
          "Text": "Doc2",
          "Active": true
        }
      }
    ]
  },

Actual result (I am only showing first record)
This returns the defaults on OptOpenText, OptDateTime, OptCurrency and TrueFalseConfig

Click to expand

{
      "Id": "a4369749-89f6-46ec-beae-b1a3e88778a2",
      "QaTemplateId": "1c883fb0-7daa-4339-82ef-bd220fe65bfc",
      "Order": 1,
      "Text": "THis is a child question. It is a true/false with label overrides. It always moves forward. It has additional data at the initial stage",
      "CollectAdditionalDataAt": "Initial",
      "HelpText": "Here's some help text for the child question",
      "HelpLink": "https://google.com",
      "Active": true,
      "IsParent": false,
      "ParentId": "79663cff-8515-4685-884b-20066aa7f14f",
      "OptOpenText": {
        "Enabled": false,
        "Required": false,
        "Label": null
      },
      "OptDateTime": {
        "Enabled": false,
        "Required": false,
        "Label": null
      },
      "OptCurrency": {
        "Enabled": false,
        "Required": false,
        "Label": null
      },
      "TrueFalseConfig": {
        "Enabled": false,
        "MoveForwardOn": "OnFalse",
        "OverrideTrueLabel": null,
        "OverrideFalseLabel": null
      },
      "QaTemplateQuestionQaDocTags": [
        {
          "Id": "a21f9cbb-77d2-43cd-8ea6-2d320a8753c4",
          "QaTemplateQuestionId": "a4369749-89f6-46ec-beae-b1a3e88778a2",
          "QaDocTagId": "320dcb36-601d-4fd3-74bf-08d762f6c0a0",
          "QaDocTag": {
            "Id": "320dcb36-601d-4fd3-74bf-08d762f6c0a0",
            "GroupName": "Tst1",
            "Text": "Doc1",
            "Active": true
          }
        },
        {
          "Id": "88d56142-b592-4301-b312-064086c063c9",
          "QaTemplateQuestionId": "a4369749-89f6-46ec-beae-b1a3e88778a2",
          "QaDocTagId": "9b085a36-4dc9-47af-74c0-08d762f6c0a0",
          "QaDocTag": {
            "Id": "9b085a36-4dc9-47af-74c0-08d762f6c0a0",
            "GroupName": "Tst1",
            "Text": "Doc2",
            "Active": true
          }
        }
      ]
    },

Removing both skip and top results in the correct response
URL https://localhost:5001/odata/qatemplatequestions?$filter=QaTemplateId%20eq%201c883fb0-7daa-4339-82ef-bd220fe65bfc&$expand=QaTemplateQuestionQaDocTags($expand=QaDocTag)&$orderby=Order%20asc&$count=false&$format=application/json;odata.metadata=none

Model (works the same regardless of if the [Owned] attribute is on the complex properties)

Click to expand

public partial class QaTemplateQuestion : BaseModel
	{
		[Required]
		public Guid QaTemplateId { get; set; }

		[Range(0, 99)]
		public short? Order { get; set; }

		[Required, MinLength(20), MaxLength(600)]
		public string Text { get; set; }

		[Column(TypeName = "tinyint")]
		[JsonConverter(typeof(StringEnumConverter))]
		public CollectAdditionalDataAt CollectAdditionalDataAt { get; set; } = CollectAdditionalDataAt.Never;

		public OptionalCustomField OptOpenText { get; set; } = new OptionalCustomField();

		public OptionalCustomField OptDateTime { get; set; } = new OptionalCustomField();

		public OptionalCustomField OptCurrency { get; set; } = new OptionalCustomField();

		public TrueFalseConfig TrueFalseConfig { get; set; } = new TrueFalseConfig();

		[MaxLength(512)]
		public string HelpText { get; set; }

		[MaxLength(256)]
		public string HelpLink { get; set; }

		public bool Active { get; set; }

		public bool IsParent { get; set; }

		public Guid? ParentId { get; set; }



		public virtual QaTemplateQuestion Parent { get; set; }

		public virtual QaTemplate QaTemplate { get; set; }

		public virtual ICollection<QaTemplateQuestion> InverseParent { get; set; }

		public virtual ICollection<InitialReview> InitialReviews { get; set; }

		public virtual ICollection<ResolutionReview> ResolutionReviews { get; set; }

		public virtual ICollection<VerificationReview> VerificationReviews { get; set; }

		public virtual ICollection<AdditionalResponseData> AdditionalResponseDatas { get; set; }

		public virtual ICollection<QaTemplateQuestionQaDocTag> QaTemplateQuestionQaDocTags { get; set; }

		public virtual ICollection<ReviewResponseComment> ReviewResponseComments { get; set; }



		public bool AdditionalResponseDataRequired()
		{
			return OpenTextRequired() || DateTimeRequired() || CurrencyRequired();
		}
		
		public bool OpenTextRequired()
		{
			return OptOpenText.Enabled && OptOpenText.Required;
		}

		public bool DateTimeRequired()
		{
			return OptDateTime.Enabled && OptDateTime.Required;
		}

		public bool CurrencyRequired()
		{
			return OptCurrency.Enabled && OptCurrency.Required;
		}
	}

	public interface IOptionalOwnedType
	{
		bool Enabled { get; set; }
	}


	[Owned]
	public class OptionalCustomField : IOptionalOwnedType
	{
		public OptionalCustomField() { }

		

		[Required]
		public bool Enabled { get; set; } = false;

		public bool Required { get; set; } = false;

		public string Label { get; set; }

	}

	[Owned]
	public class TrueFalseConfig : IOptionalOwnedType
	{
		public TrueFalseConfig() { }



		[Required]
		public bool Enabled { get; set; } = false;

		[Column(TypeName = "smallint")]
		[JsonConverter(typeof(StringEnumConverter))]
		public TrueFalseProceedOn MoveForwardOn { get; set; } = TrueFalseProceedOn.OnFalse;

		public string OverrideTrueLabel { get; set; }

		public string OverrideFalseLabel { get; set; }
	}

	public enum TrueFalseProceedOn
	{
		Never = -1,
		OnFalse = 0,
		OnTrue = 1,
		Always = 2
	}

	public enum CollectAdditionalDataAt
	{
		Never = 0,
		Initial = 1,
		Resolution = 2,
		Confirmation = 3,
		Verification = 4
	}

[smitpatel]

The original issue has been fixed in EF Core 3.1 release. When projecting a type which contains owned types they are automatically included now.

As for @joelmeaders repro code. I believe that the issue is similar to dotnet/efcore#18672 (which had quite a different variations of the same when projection is custom and then skip/take is applied). All of those have been fixed in EF Core 5.0 preview1

[MonkeyTennis]

I am still seeing the original issue when using EF Core 3.1.5 and ASP.NET Core OData 7.4.1.

It appears that owned entities, whilst clearly populated in the IQueryable as observed server-side, deserialize to null in the client.

The comment of 11th May suggests this is fixed in the releases I am using. Is this definitely correct?

[djrerun]

@MonkeyTennis I am also still seeing the original issue with EF Core 3.1.5 and OData 7.4.1. My entity relationship is a little different in that I do not have an explicit owned as defined in the example [Order]---<[Address]. I have a many to many defined as:

[Person]---<[PersonAchievement]>---[Achievement]

Would that affect the outcome?

[m4ss1m0g]

@xuzhg Do you have plans to solve this issue on some nightly build ?

I am still seeing the issue using EF Core 3.1.5 and ASP.NET Core OData 7.4.1.
If I use $skip $top and $expand complex properties are all null, in the example below
EMails and Address are complex properties

http://localhost:5000/api/customers?$skip=0&$top=1&$expand=customerType($select=title)

{
"value": [
{
"Id": 1,
"CustomerTypeId": null,
"Name": "Customer Name",
"Note": null,
"Vat": "0000000000",
"Address": null,
"EMails": null,
"CustomerType": null
}
]
}

[MonkeyTennis]

I haven't looked at this for a while but did attempt the upgrade again, and got the same issue.

The GET APIs that are experiencing the issue have a return type of this.Ok(SingleResult.Create(query)). I have not found a way of getting this to include owned types. However, I have also noticed that if I debug the method and force the Queryable within the SingleResult to be executed, the owned types ARE returned to the OData client.

I've worked around this by changing the response to be this.Ok(entity) where entity is the executed LINQ query, making sure to Include (and ThenInclude) where necessary.

Could the development team comment on this? Am I misusing SingleResult in some way or is there a bug here?

[MonkeyTennis]

Actually, this only fixes the issue where a single result is being returned. If an IQueryable is returned, the issue remains. I have isolated it to the following:

1. This query works and all data is populated:

/Odata/Users('username')/Possessions

Resulting in the following JSON:

{ "@odata.context": "http://localhost/odata/$metadata#Users('username')/Possessions", "value": [ { "Name": "5dd33eff-c068-49e8-81f2-24ad4ccc9706", "Value": "Android", **"CreatedBy": { "IdentityType": "User", "Name": "someuser" }** } ] }

2. This query does not work and no owned types within the object graph are populated:

/Odata/Users('username')/Possessions?$expand=PossessionDefinition

Resulting in the following JSON:

{ "@odata.context": "http://localhost/odata/$metadata#Users('username')/Possessions(PossessionDefinition())", "value": [ { "Name": "46509a55-52db-4a86-95c5-4f34bf0507ac", "Value": "Android", **"CreatedBy": null,** "PossessionDefinition": { "Name": "Phone", "CreatedBy": null } } ] }

I do not know why but the inclusion of $expand causes the issue to occur.

Does this ring any bells with the development team?

[orencomer]

I am getting this same issue. I explained the situation in issue number 2562.
https://github.com/OData/WebApi/issues/2562
Have you been able to solve this situation?