JSON deserialization fails in iOS release build if class implements an interface #75802

markuspalme · 2022-09-17T19:12:55Z

Description

A simple JSON deserialization using Newtonsoft.Json fails on iOS in a release build as soon as the class that is trying be be de-serialized implements an interface.

The same app works in Debug build in the iOS simulator as expected.

Reproduction Steps

Create a new iOS app with dotnet new ios
Set a valid bundle identifier that allows installing the app on a physical device

Add this code to the project:

public class ProductImage : IEntity
{
   public string ProductNumber { get; set; }
   public Guid Id { get; set; }
}
  
public interface IEntity
{
   Guid Id { get; set; }
}

During startup, run this code:

var x = @"{
   ""productNumber"": ""P1"",
   ""id"": ""46c67d7c-fd15-4d89-9fed-991c48c1bacf""
}";

var pi = JsonConvert.DeserializeObject<ProductImage>(x);

Build the app for a device: dotnet publish testapp.csproj -f:net6.0-ios -c:Release /p:CodesignKey="Apple Development: ******" /p:CodesignProvision="Test app" /p:ArchiveOnBuild=true -r ios-arm64 --self-contained
Deploy to device and run

Full project file:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net6.0-ios</TargetFramework>
    <OutputType>Exe</OutputType>
    <Nullable>enable</Nullable>
    <ImplicitUsings>true</ImplicitUsings>
    <SupportedOSPlatformVersion>13.0</SupportedOSPlatformVersion>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
  </ItemGroup>
</Project>

Self-contained example project:
https://github.com/markuspalme/dotnetruntime-75802

Expected behavior

The app starts successfully and manages to de-serialize the JSON regardless of whether the class implements an interface or not.

Note that removing the interface will make the de-serialization work as expected.

Actual behavior

JSON deserialization fails with this exception:

Error setting value to 'Id' on 'testapp.ProductImage'.

Regression?

The same code works in Xamarin.iOS.

Known Workarounds

No response

Configuration

.NET 6.0.401
iOS 16, same on 15.6.1

Other information

No response

The text was updated successfully, but these errors were encountered:

ghost · 2022-09-17T19:13:06Z

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

A simple JSON deserialization using Newtonsoft.Json fails on iOS in a release build as soon as the class that is trying be be de-serialized implements an interface.

The same app works in Debug build in the iOS simulator as expected.

Reproduction Steps

Create a new iOS app with dotnet new ios
Set a valid bundle identifier that allows installing the app on a physical device
Add this code to the project:


  public class ProductImage : IEntity
  {
	  public string ProductNumber { get; set; }
  
	  public Guid Id { get; set; }
  }
  
  public interface IEntity
  {
	  Guid Id { get; set; }
  }

During startup, run this code:

var x = @"{
      ""productNumber"": ""P1"",
      ""id"": ""46c67d7c-fd15-4d89-9fed-991c48c1bacf""
  }";

var pi = JsonConvert.DeserializeObject<ProductImage>(x);

Build the app for a device: dotnet publish testapp.csproj -f:net6.0-ios -c:Release /p:CodesignKey="Apple Development: ******" /p:CodesignProvision="Test app" /p:ArchiveOnBuild=true -r ios-arm64 --self-contained
Deploy to device and run

Full project file:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net6.0-ios</TargetFramework>
    <OutputType>Exe</OutputType>
    <Nullable>enable</Nullable>
    <ImplicitUsings>true</ImplicitUsings>
    <SupportedOSPlatformVersion>13.0</SupportedOSPlatformVersion>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
  </ItemGroup>
</Project>

Expected behavior

The app starts successfully and manages to de-serialize the JSON regardless of whether the class implements an interface or not.

Note that removing the interface

Actual behavior

JSON deserialization fails with this exception:

Error setting value to 'Id' on 'testapp.ProductImage'.

Regression?

The same code works in Xamarin.iOS.

Known Workarounds

No response

Configuration

.NET 6.0.4.1
iOS 16

Other information

No response

Author:	markuspalme
Assignees:	-
Labels:	`area-System.Text.Json`, `untriaged`
Milestone:	-

gregsdennis · 2022-09-17T19:49:43Z

Are you certain this is a bug in .Net? Seems like it's a Newtonsoft issue. If so, the bug should be filed in that repo.

markuspalme · 2022-09-17T19:53:44Z

@gregsdennis It works just fine in net6 on Windows, MacOS and also in the iOS simulator - all with the same Newtonsoft.Json package. So I think this is a runtime or maybe AOT issue (iOS requires AOT).

gregsdennis · 2022-09-17T19:55:46Z

Understood, but it might be that the package just needs handle a special case in this OS. There is the possibility that the issue is the package.

markuspalme · 2022-09-17T19:57:15Z

Unlikely, the same code works with Xamarin.iOS.

Cheesebaron · 2022-09-19T06:15:57Z

@markuspalme could it be the linker stripping out stuff?

Could you try add <PublishTrimmed>false</PublishTrimmed> in your Project Properties and see if that helps?

markuspalme · 2022-09-19T07:38:22Z

@Cheesebaron Thanks for the suggestion, on iOS that is not an option though:

ghost · 2022-09-19T07:45:29Z

Tagging subscribers to 'os-ios': @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

A simple JSON deserialization using Newtonsoft.Json fails on iOS in a release build as soon as the class that is trying be be de-serialized implements an interface.

The same app works in Debug build in the iOS simulator as expected.

Reproduction Steps

Create a new iOS app with dotnet new ios
Set a valid bundle identifier that allows installing the app on a physical device
Add this code to the project:


  public class ProductImage : IEntity
  {
	  public string ProductNumber { get; set; }
  
	  public Guid Id { get; set; }
  }
  
  public interface IEntity
  {
	  Guid Id { get; set; }
  }

During startup, run this code:

var x = @"{
      ""productNumber"": ""P1"",
      ""id"": ""46c67d7c-fd15-4d89-9fed-991c48c1bacf""
  }";

var pi = JsonConvert.DeserializeObject<ProductImage>(x);

Build the app for a device: dotnet publish testapp.csproj -f:net6.0-ios -c:Release /p:CodesignKey="Apple Development: ******" /p:CodesignProvision="Test app" /p:ArchiveOnBuild=true -r ios-arm64 --self-contained
Deploy to device and run

Full project file:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net6.0-ios</TargetFramework>
    <OutputType>Exe</OutputType>
    <Nullable>enable</Nullable>
    <ImplicitUsings>true</ImplicitUsings>
    <SupportedOSPlatformVersion>13.0</SupportedOSPlatformVersion>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
  </ItemGroup>
</Project>

Expected behavior

The app starts successfully and manages to de-serialize the JSON regardless of whether the class implements an interface or not.

Note that removing the interface will make the de-serialization work as expected.

Actual behavior

JSON deserialization fails with this exception:

Error setting value to 'Id' on 'testapp.ProductImage'.

Regression?

The same code works in Xamarin.iOS.

Known Workarounds

No response

Configuration

.NET 6.0.401
iOS 16, same on 15.6.1

Other information

No response

Author:	markuspalme
Assignees:	-
Labels:	`area-System.Text.Json`, `untriaged`, `os-ios`
Milestone:	-

markuspalme · 2022-09-19T07:47:54Z

I have tried System.Text.Json instead of Newtonsoft.Json. It does not throw an exception, but it does not set any property values.

var pi = System.Text.Json.JsonSerializer.Deserialize<ProductImage>(x);

markuspalme · 2022-09-19T07:54:58Z

Here's a small self-contained example:
https://github.com/markuspalme/dotnetruntime-75802

markuspalme · 2022-09-19T08:11:45Z

Adding <UseInterpreter>true</UseInterpreter> as suggested in this issue helps - but I don't think this should be neccesary:
xamarin/xamarin-macios#15961

Cheesebaron · 2022-09-19T10:14:28Z

@markuspalme UseInterpreter is not for Release builds, Apple won't allow that.

markuspalme · 2022-09-19T10:35:53Z

@Cheesebaron Yes, I figured but wanted to share that it helps here - maybe it helps pinpointing the root cause.

ghost · 2022-09-19T14:06:10Z

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

A simple JSON deserialization using Newtonsoft.Json fails on iOS in a release build as soon as the class that is trying be be de-serialized implements an interface.

The same app works in Debug build in the iOS simulator as expected.

Reproduction Steps

Create a new iOS app with dotnet new ios
Set a valid bundle identifier that allows installing the app on a physical device

Add this code to the project:

public class ProductImage : IEntity
{
   public string ProductNumber { get; set; }
   public Guid Id { get; set; }
}
  
public interface IEntity
{
   Guid Id { get; set; }
}

During startup, run this code:

var x = @"{
   ""productNumber"": ""P1"",
   ""id"": ""46c67d7c-fd15-4d89-9fed-991c48c1bacf""
}";

var pi = JsonConvert.DeserializeObject<ProductImage>(x);

Build the app for a device: dotnet publish testapp.csproj -f:net6.0-ios -c:Release /p:CodesignKey="Apple Development: ******" /p:CodesignProvision="Test app" /p:ArchiveOnBuild=true -r ios-arm64 --self-contained
Deploy to device and run

Full project file:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net6.0-ios</TargetFramework>
    <OutputType>Exe</OutputType>
    <Nullable>enable</Nullable>
    <ImplicitUsings>true</ImplicitUsings>
    <SupportedOSPlatformVersion>13.0</SupportedOSPlatformVersion>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
  </ItemGroup>
</Project>

Self-contained example project:
https://github.com/markuspalme/dotnetruntime-75802

Expected behavior

The app starts successfully and manages to de-serialize the JSON regardless of whether the class implements an interface or not.

Note that removing the interface will make the de-serialization work as expected.

Actual behavior

JSON deserialization fails with this exception:

Error setting value to 'Id' on 'testapp.ProductImage'.

Regression?

The same code works in Xamarin.iOS.

Known Workarounds

No response

Configuration

.NET 6.0.401
iOS 16, same on 15.6.1

Other information

No response

Author:	markuspalme
Assignees:	-
Labels:	`area-System.Text.Json`, `untriaged`, `os-ios`
Milestone:	-

eiriktsarpalis · 2022-09-19T14:41:46Z

@jeffschwMSFT I don't believe the issue is related to System.Text.Json. I removed the label but wasn't sure what the appropriate area label should be. Perhaps @steveisok or @akoeplinger know.

jkotas · 2022-09-19T14:59:10Z

System.Text.Json should be the right area label.

Source generators are the only 100% reliable way to perform Json serialization/deserialization in the presence of trimming or AOT compilation without fallbacks.

Unfortunately, trim warnings that would notify about this problem are disabled for iOS apps by default. You can set <SuppressTrimAnalysisWarnings>false</SuppressTrimAnalysisWarnings> in your .csproj to enable trim warnings to see all potential places in your app that can break due to trimming.

This is duplicate #74141 and number of other similar issues.

@eiriktsarpalis My suggestion would be:

Make sure that we have documentation that we can point to and resolve these issues against. Do we have a documentation that says "System.Text.Json source generators are the only 100% reliable way to perform Json serialization/deserialization in the presence of trimming"?
Note the feedback on issues that track enabling the link warning in app-model specific SDKs. It is Don't suppress trim analysis warnings when linking all assemblies xamarin/xamarin-macios#11246 for iOS.

eiriktsarpalis · 2022-09-19T15:20:02Z

System.Text.Json should be the right area label.

The OP concerns Json.NET (Newtonsoft) serialization issues in iOS. I agree that it falls under the same category as #74141, but I wonder what our approach should be when users report trimming-related failures when using third party reflection-based libraries. We might want to consider asking them to file an issue with said third-party libraries asking about AOT support.

Do we have a documentation that says "System.Text.Json source generators are the only 100% reliable way to perform Json serialization/deserialization in the presence of trimming"?

It is, although the wording is certainly more understated than that.

markuspalme · 2022-09-19T15:21:12Z

@jkotas I will try to enable the trim warnings.

JSON is at the heart of many apps and this feels like a regression coming from Xamarin.iOS where such scenarios worked out of the box. Migrating toSystem.Text.Json source generators is a possibility for user code, but many libraries depend on Newtonsoft.Json, consider a generic key-value store like Akavache (https://github.com/reactiveui/Akavache)

eiriktsarpalis · 2022-09-19T15:39:19Z

@markuspalme I'm not sure if and when Json.NET plans on bringing AOT support, but you might want to consider opening an issue in its own repo or those of the other libraries that depend on it. In the meantime, migrating to System.Text.Json source generation is probably the only viable option.

markuspalme · 2022-09-19T15:42:25Z

@eiriktsarpalis Thanks for your input. Given that earlier versions of Xamarin.iOS supported this scenario out of the box even in AOT mode, this seems like a bad surprise or even a blocker for anyone porting applications to net6-ios.

markuspalme · 2022-09-19T17:37:27Z

@eiriktsarpalis So my understanding is that this problem here is caused by trimming. To preserve all the types in my code, I have added this to the project file as described here (https://devblogs.microsoft.com/dotnet/customizing-trimming-in-net-core-5/):

<ItemGroup>
  <TrimmerRootDescriptor Include="TrimmerRoots.xml" />
</ItemGroup>

The TrimmerRoots.xml looks like this:

<?xml version="1.0" encoding="utf-8"?>

<linker>
    <assembly fullname="testapp" preserve="all" />
</linker>

The de-serialization does not work with that in place. Shouldn't this ensure that all my types are preserved and available through reflection?

The whole story around trimming and linking is rather intimidating:

there is the [Preserve] attribute
there is the before mentioned external XML configuration option
For net6-ios projects there is the MtouchLink property in the project file

Is there any guidance available on which one to use when and how they work together?

markuspalme · 2022-09-20T18:16:00Z

@sbomer here you go:
build.binlog.zip

Build command:
dotnet publish testapp.csproj -f:net6.0-ios -c:Release /p:CodesignKey="Apple Development: ******" /p:CodesignProvision="Test app" /p:ArchiveOnBuild=true -r ios-arm64 --self-contained -bl:build.binlog

markuspalme · 2022-09-29T17:55:56Z

@sbomer Did the binlog reveal anything interesting?

steveisok · 2022-09-29T18:00:57Z

@sbomer if you need to bounce ideas off of someone who has ios set up, please let me know.

sbomer · 2022-10-03T16:59:41Z

Sorry for the delay - I wasn't able to see anything wrong from the binlog, which indicates that the descriptor is getting passed along correctly. @vitek-karas would your repro tool help here to collect the input assemblies?

sbomer · 2022-10-04T19:07:25Z

@markuspalme would you be able to try this tool by @vitek-karas to see if you can obtain a repro? https://github.com/vitek-karas/illinkrepro

You'll need to clone it on the machine where you ran the publish command and obtained the binlog. Then inside of the cloned directory, do dotnet run -- create path/to/msbuild.binlog. It tries creates a repro directory containing the linker command-line and input assemblies. If you can share that with us, it would be very helpful.

markuspalme · 2022-10-04T19:21:07Z

@sbomer Here you go: https://github.com/markuspalme/dotnetruntime-75802/raw/master/repro.zip

sbomer · 2022-10-04T21:30:14Z

Thanks @markuspalme, that gets us a bit further. I was able to tweak the repro as I described here to get some linker output.

I was hoping that the linker output would show something obviously wrong, but I didn't notice anything. I see that ProductImage's interface implementation of IEntity is kept. It seems like it's actually kept with or without the descriptor, which is a bit surprising since the win-x64 repro I tried (without root descriptor) was failing with the interface and ctor being removed.

Would you also be able to share your publish output (ideally both with and without the descriptor) so I can compare? If that doesn't show what's wrong I will probably need to set up an iOS development environment to debug (or debug it with @steveisok).

markuspalme · 2022-10-05T08:20:11Z

@sbomer Here are the IPA files:

With descriptor:
testapp-descriptor.ipa.zip

Without descriptor:
testapp-without-descriptor.ipa.zip

There is is at least a difference in size:

sbomer · 2022-10-05T19:23:31Z

Thanks @markuspalme, in the output you shared I noticed that the ProductImage accessors don't look right:

I am not seeing the same thing in any of my repros so far. I also got your original repro to run on a simulator, but didn't see the issue there either. Now I'm trying to get set up with a provisioning profile with some help from @steveisok. Thanks for your patience.

markuspalme · 2022-10-05T19:31:51Z

@sbomer I saw the same but was not sure it's relevant because I don't know if the assembly is used for anything but metadata in the AOT build.

The error is not reproducible on the simulator. I do think it's an AOT issue and not a trimming issue for this reason - but I have no idea how to investigate any further by myself.

sbomer · 2022-10-05T19:54:17Z

So far I tend to agree that it doesn't look like a linker issue. I think we need someone more familiar with the AOT/Xamarin scenarios to investigate. /cc @steveisok

MichalStrehovsky · 2022-10-06T06:56:11Z

Looks like the assembly has been ran through ILStrip after trimming. ILStrip builds on top of some unserviced version of Cecil. Is there a way to disable it to get it out of the picture? I can't figure out which targets enable it, but the methods are marked noininling and the bodies are just a ret and that's ILStrip.

I saw some hints in the .targets I found that ILStrip might be disabled if interpreter is enabled and we've established that enabling interpreted fixes it here #75802 (comment) so this might be ILStrip.

MichalStrehovsky · 2022-10-06T06:57:22Z

Cc @akoeplinger for ILStrip

akoeplinger · 2022-10-06T08:46:52Z

@MichalStrehovsky @markuspalme you can try setting EnableAssemblyILStripping to false in your csproj

markuspalme · 2022-10-06T08:55:41Z

@MichalStrehovsky I did that. The assembly is now intact:

But the error still occurs on device. As far as I know, the IL is used for metadata only with AOT enabled which is probably why method bodies are stripped away to save space.

markuspalme · 2022-10-24T18:33:31Z

@MichalStrehovsky Is there anything else I can help with analyzing this issue?

markuspalme · 2022-12-12T11:56:01Z

@MichalStrehovsky @steveisok @sbomer Could this be related to #69410?

markuspalme · 2023-01-24T09:25:40Z

Can still be reproduced with .NET 7.

minglu · 2023-01-31T01:27:56Z

following

steveisok · 2023-01-31T01:33:04Z

@akoeplinger please investigate if this is indeed an ILStrip issue.

steveisok · 2023-08-08T04:18:35Z

@akoeplinger do you the recent ILStrip changes might have fixed this? Can you please validate?

markuspalme · 2023-08-09T07:15:59Z

This is fixed in 8.0.100-preview.7.23376.3, presumably by #69410.

ghost added the untriaged New issue has not been triaged by the area owner label Sep 17, 2022

dotnet-issue-labeler bot added the area-System.Text.Json label Sep 17, 2022

markuspalme mentioned this issue Sep 18, 2022

MvvmCross 9.0.1 iOS release build crashes on device MvvmCross/MvvmCross#4484

Closed

7 tasks

marek-safar added the os-ios Apple iOS label Sep 19, 2022

markuspalme mentioned this issue Sep 19, 2022

aot-compiled net6-ios app fails retrieving property xamarin/xamarin-macios#15961

Open

eiriktsarpalis removed the area-System.Text.Json label Sep 19, 2022

jeffschwMSFT added the area-System.Text.Json label Sep 19, 2022

eiriktsarpalis added question Answer questions and provide assistance, not an issue with source code or documentation. and removed untriaged New issue has not been triaged by the area owner labels Sep 19, 2022

eiriktsarpalis removed the area-System.Text.Json label Sep 22, 2022

jeffschwMSFT added the area-VM-meta-mono label Sep 23, 2022

steveisok added this to the 8.0.0 milestone Sep 28, 2022

sbomer mentioned this issue Oct 3, 2022

Add option to create self-contained trimming repro dotnet/linker#3056

Open

steveisok assigned akoeplinger Jan 31, 2023

minglu mentioned this issue Jan 31, 2023

iOS Release builds stopped working after last update dotnet/maui#12519

Closed

markuspalme closed this as completed Aug 9, 2023

ghost locked as resolved and limited conversation to collaborators Sep 8, 2023

JSON deserialization fails in iOS release build if class implements an interface #75802

JSON deserialization fails in iOS release build if class implements an interface #75802

Comments

markuspalme commented Sep 17, 2022 • edited Loading

Description

Reproduction Steps

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

Other information

ghost commented Sep 17, 2022

Description

Reproduction Steps

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

Other information

gregsdennis commented Sep 17, 2022

markuspalme commented Sep 17, 2022

gregsdennis commented Sep 17, 2022

markuspalme commented Sep 17, 2022

Cheesebaron commented Sep 19, 2022

markuspalme commented Sep 19, 2022 • edited Loading

ghost commented Sep 19, 2022

Description

Reproduction Steps

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

Other information

markuspalme commented Sep 19, 2022

markuspalme commented Sep 19, 2022

markuspalme commented Sep 19, 2022

Cheesebaron commented Sep 19, 2022

markuspalme commented Sep 19, 2022

ghost commented Sep 19, 2022

Description

Reproduction Steps

Expected behavior

Actual behavior

Regression?

Known Workarounds

Configuration

Other information

eiriktsarpalis commented Sep 19, 2022

jkotas commented Sep 19, 2022

eiriktsarpalis commented Sep 19, 2022

markuspalme commented Sep 19, 2022 • edited Loading

eiriktsarpalis commented Sep 19, 2022

markuspalme commented Sep 19, 2022

markuspalme commented Sep 19, 2022

markuspalme commented Sep 20, 2022 • edited Loading

markuspalme commented Sep 29, 2022

steveisok commented Sep 29, 2022

sbomer commented Oct 3, 2022

sbomer commented Oct 4, 2022

markuspalme commented Oct 4, 2022 • edited Loading

sbomer commented Oct 4, 2022

markuspalme commented Oct 5, 2022

sbomer commented Oct 5, 2022

markuspalme commented Oct 5, 2022

sbomer commented Oct 5, 2022

MichalStrehovsky commented Oct 6, 2022

MichalStrehovsky commented Oct 6, 2022

akoeplinger commented Oct 6, 2022

markuspalme commented Oct 6, 2022 • edited Loading

markuspalme commented Oct 24, 2022

markuspalme commented Dec 12, 2022

markuspalme commented Jan 24, 2023

minglu commented Jan 31, 2023

steveisok commented Jan 31, 2023

steveisok commented Aug 8, 2023

markuspalme commented Aug 9, 2023

markuspalme commented Sep 17, 2022 •

edited

Loading

markuspalme commented Sep 19, 2022 •

edited

Loading

markuspalme commented Sep 19, 2022 •

edited

Loading

markuspalme commented Sep 20, 2022 •

edited

Loading

markuspalme commented Oct 4, 2022 •

edited

Loading

markuspalme commented Oct 6, 2022 •

edited

Loading