Split TryRead implementation into TryPopulate and TryCreateObject #79659

krwq · 2022-12-14T17:03:45Z

Contributes to: #78556
Contibutes to: #77018

Cleanup converter internal implementation - splits TryRead into:

TryCreateObject - creates intermediate object (it can be same as final)
TryPopulate - populates intermediate object
TryConvert - converts intermediate object into final object (or just casts for most of the cases)

TODO:

ghost · 2022-12-14T17:03:56Z

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Issue Details

Contributes to: #78556

Cleanup converter internal implementation - splits TryRead into TryPopulate and TryCreateOrGetObject and unify some code.

Author:	krwq
Assignees:	-
Labels:	`area-System.Text.Json`
Milestone:	-

...tem.Text.Json/src/System/Text/Json/Serialization/Converters/Object/ObjectDefaultConverter.cs

...tem/Text/Json/Serialization/Converters/Object/ObjectWithParameterizedConstructorConverter.cs

krwq · 2022-12-15T17:30:15Z

[Triage] Offline review feedback:
I've cut content and manage TODO list in the description of the PR now

...m.Text.Json/src/System/Text/Json/Serialization/Converters/Collection/IDictionaryConverter.cs

.../System.Text.Json/src/System/Text/Json/Serialization/Converters/Collection/IListConverter.cs

ghost · 2023-01-16T05:02:04Z

Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it.

.../System.Text.Json/src/System/Text/Json/Serialization/Converters/Collection/ArrayConverter.cs

eiriktsarpalis · 2023-01-30T15:09:36Z

.../System.Text.Json/src/System/Text/Json/Serialization/Converters/Collection/ArrayConverter.cs

    {
        internal override bool CanHaveMetadata => false;

-        protected override void Add(in TElement value, ref ReadStack state)
+        private protected override void Add(ref List<TElement> collection, in TElement value, JsonTypeInfo collectionTypeInfo)


In any case, we don't stand to win much by making it strongly typed -- I don't think any of the intermediate collections we're using are structs so we aren't boxing anything redundant right now.

actually ReadOnlyArray is a struct and uses intermediate type - that's why I added that in the first place

ReadOnlyArray? Was this added in this PR?

no, there are existing tests though which implement read only collections as structs and those are failing unless this is ref previously we changed state.Current.ReturnValue here and reassigning if this was value type

eiriktsarpalis · 2023-01-30T15:10:52Z

.../System.Text.Json/src/System/Text/Json/Serialization/Converters/Collection/ArrayConverter.cs

        }

-        protected override void ConvertCollection(ref ReadStack state, JsonSerializerOptions options)
+        private protected override bool TryConvert(ref Utf8JsonReader reader, JsonTypeInfo jsonTypeInfo, scoped ref ReadStack state, List<TElement> obj, out TElement[] value)


What does the bool result in TryConvert signify?

honestly I added it because I think we can implement parametrized constructors in terms of these 3 and in that case it will need the bool result. In general it means same as in other places: "I did not finish reading, continuation will need to happen"

Isn't convert something meant to run after serialization has completed to materialize the final instance from the intermediate one? If so, there's not async reading necessary that might necessitate producing partial results.

that is certainly possible it's not needed and I added it initially because I wasn't initially sure if we do that or not. I will reiterate on that

eiriktsarpalis · 2023-01-30T15:11:59Z

...Json/src/System/Text/Json/Serialization/Converters/Collection/ConcurrentQueueOfTConverter.cs

        where TCollection : ConcurrentQueue<TElement>
    {
-        protected override void Add(in TElement value, ref ReadStack state)
+        private protected override void Add(ref TCollection collection, in TElement value, JsonTypeInfo collectionTypeInfo)


Why does it need to be private protected? The type itself is internal so it shouldn't matter much.

no preference

eiriktsarpalis · 2023-01-30T15:13:33Z

....Json/src/System/Text/Json/Serialization/Converters/Collection/DictionaryDefaultConverter.cs

 using System.Text.Json.Serialization.Metadata;

 namespace System.Text.Json.Serialization.Converters
 {
    /// <summary>
    /// Default base class implementation of <cref>JsonDictionaryConverter{TCollection}</cref> .
    /// </summary>
-    internal abstract class DictionaryDefaultConverter<TDictionary, TKey, TValue>
-        : JsonDictionaryConverter<TDictionary, TKey, TValue>
+    internal abstract class DictionaryDefaultConverter<TDictionary, TKey, TValue, IntermediateType>


See my earlier feedback. I think this could regress application sizes without much tangible impact.

eiriktsarpalis · 2023-01-30T15:18:08Z

...ries/System.Text.Json/src/System/Text/Json/Serialization/Converters/JsonAdvancedConverter.cs

+    /// <summary>
+    /// Base class for object, enumerable and dictionary converters
+    /// </summary>
+    internal abstract class JsonAdvancedConverter<T, IntermediateType> : JsonAdvancedConverter<T>


Nit: would avoid terms like "advanced" in type names since they don't necessarily convey what this layer of abstraction is meant to achieve.

didn't have a better name, I'm open for feedback

eiriktsarpalis · 2023-01-30T15:18:55Z

...ries/System.Text.Json/src/System/Text/Json/Serialization/Converters/JsonAdvancedConverter.cs

+
+            if (IsReadOnly(obj))
+            {
+                // TODO: not possible for objects, might be a good idea to still generalize exception


Object deserialization is more common than collection deserialization, wouldn't want to spend CPU cycles checking for something that is always true.

that's really a single check per object, I was thinking that this type could eventually be public and making this generalized (if we do delegates that will also be true)

eiriktsarpalis · 2023-01-30T15:58:45Z

...ext.Json/src/System/Text/Json/Serialization/Converters/Collection/JsonCollectionConverter.cs

-        }
-
-        protected virtual void ConvertCollection(ref ReadStack state, JsonSerializerOptions options) { }
+        private protected abstract void Add(ref IntermediateType collection, in TElement value, JsonTypeInfo collectionTypeInfo);


I think once we get to the point of bringing collection support to contract customization, I would imagine we would be exposing a Action<object, object>? Add { get; set; } to JsonTypeInfo. Perhaps we could abstract away the concept of intermediate accumulators by having all the collection lambdas accept an additional object? state parameter.

By pushing all these concerns to JsonTypeInfo, it might be that we could replace all enumerable converter types with just one that calls into delegates. I expect this might offer even more size reduction benefits.

ghost · 2023-03-02T17:02:10Z

Draft Pull Request was automatically closed for 30 days of inactivity. Please let us know if you'd like to reopen it.

krwq requested a review from eiriktsarpalis December 14, 2022 17:03

dotnet-issue-labeler bot added the area-System.Text.Json label Dec 14, 2022

ghost assigned krwq Dec 14, 2022

eiriktsarpalis requested a review from layomia December 14, 2022 17:04

krwq commented Dec 14, 2022

View reviewed changes

...tem.Text.Json/src/System/Text/Json/Serialization/Converters/Object/ObjectDefaultConverter.cs Outdated Show resolved Hide resolved

This was referenced Dec 14, 2022

SyndicationFeed_Write_RSS_Atom test failing with "IOException : The file '/tmp/' already exists." #78454

Closed

[wasm] Library tests failing during linking for AOT - SIGKILL #79569

Closed

krwq commented Dec 14, 2022

View reviewed changes

...tem/Text/Json/Serialization/Converters/Object/ObjectWithParameterizedConstructorConverter.cs Outdated Show resolved Hide resolved

krwq marked this pull request as ready for review December 15, 2022 11:16

build-analysis bot mentioned this pull request Dec 15, 2022

NuGet failing with Response status code does not indicate success: 503 (Service Unavailable) dotnet/arcade#11723

Open

5 tasks

krwq marked this pull request as draft December 15, 2022 17:30

krwq commented Dec 16, 2022

View reviewed changes

...m.Text.Json/src/System/Text/Json/Serialization/Converters/Collection/IDictionaryConverter.cs Outdated Show resolved Hide resolved

krwq commented Dec 16, 2022

View reviewed changes

.../System.Text.Json/src/System/Text/Json/Serialization/Converters/Collection/IListConverter.cs Outdated Show resolved Hide resolved

build-analysis bot mentioned this pull request Dec 16, 2022

Tracking issue for CI build timeouts #76454

Closed

ghost closed this Jan 16, 2023

krwq reopened this Jan 16, 2023

krwq force-pushed the json-converters-populate branch 2 times, most recently from a7e69af to 3e4786c Compare January 16, 2023 12:07

Split TryRead implementation into TryPopulate and TryCreateObject

c8981ac

krwq force-pushed the json-converters-populate branch from 5076bf1 to c8981ac Compare January 23, 2023 10:51

CreateObject for collections/dictionaries

9635dca

eiriktsarpalis reviewed Jan 30, 2023

View reviewed changes

.../System.Text.Json/src/System/Text/Json/Serialization/Converters/Collection/ArrayConverter.cs Show resolved Hide resolved

eiriktsarpalis reviewed Jan 30, 2023

View reviewed changes

ghost closed this Mar 2, 2023

krwq mentioned this pull request Mar 28, 2023

Support top-level populate methods for JSON (patch) #84018

Open

ghost locked as resolved and limited conversation to collaborators Apr 1, 2023

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Split TryRead implementation into TryPopulate and TryCreateObject #79659

Split TryRead implementation into TryPopulate and TryCreateObject #79659

krwq commented Dec 14, 2022 •

edited

ghost commented Dec 14, 2022

krwq commented Dec 15, 2022 •

edited

ghost commented Jan 16, 2023

eiriktsarpalis Jan 30, 2023

krwq Jan 30, 2023 •

edited

eiriktsarpalis Jan 30, 2023

krwq Jan 31, 2023

eiriktsarpalis Jan 30, 2023

krwq Jan 30, 2023

eiriktsarpalis Jan 30, 2023

krwq Jan 31, 2023

eiriktsarpalis Jan 30, 2023

krwq Jan 30, 2023

eiriktsarpalis Jan 30, 2023

eiriktsarpalis Jan 30, 2023

krwq Jan 30, 2023

eiriktsarpalis Jan 30, 2023 •

edited

krwq Jan 30, 2023

eiriktsarpalis Jan 30, 2023

eiriktsarpalis Jan 30, 2023

ghost commented Mar 2, 2023

Split TryRead implementation into TryPopulate and TryCreateObject #79659

Split TryRead implementation into TryPopulate and TryCreateObject #79659

Conversation

krwq commented Dec 14, 2022 • edited

ghost commented Dec 14, 2022

krwq commented Dec 15, 2022 • edited

ghost commented Jan 16, 2023

Choose a reason for hiding this comment

krwq Jan 30, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eiriktsarpalis Jan 30, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ghost commented Mar 2, 2023

krwq commented Dec 14, 2022 •

edited

krwq commented Dec 15, 2022 •

edited

krwq Jan 30, 2023 •

edited

eiriktsarpalis Jan 30, 2023 •

edited