-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix to #30604 - Implement JSON serialization/deserialization via Utf8JsonReader/Utf8JsonWriter #31160
Conversation
@roji @ajcvickers this is ready for review now, there are still some minor cleanups to be done but overall logic is complete. Good luck ;) |
test/EFCore.Relational.Specification.Tests/Query/JsonQueryAdHocTestBase.cs
Outdated
Show resolved
Hide resolved
...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs
Outdated
Show resolved
Hide resolved
...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs
Outdated
Show resolved
Hide resolved
4102a94
to
928ed3a
Compare
I'll be reviewing this in the coming days for sure.
These would actually be useful as doc comments in the source code too (where necessary/reasonable). |
I added fragments that are generated here and there - will add more. But I think the entire output is not feasible as comments in the code - it's 3+ pages of code for the simplest nested scenario - will add some samples for various scenarios on the pr tho |
most basic scenario, no tracking ss.Set<JsonEntityBasic>().Select(x => x.OwnedReferenceRoot.OwnedReferenceBranch.OwnedReferenceLeaf).AsNoTracking() (queryContext, dataReader, resultContext, resultCoordinator) =>
{
Stream namelessParameter{0};
JsonReaderData namelessParameter{1};
Utf8JsonReaderManager namelessParameter{2};
object[] namelessParameter{3};
JsonOwnedLeaf namelessParameter{4};
namelessParameter{0} = dataReader.IsDBNull(0) ? default(MemoryStream) : new MemoryStream(Encoding.UTF8.GetBytes((string)dataReader.GetFieldValue<object>(0)));
namelessParameter{1} = namelessParameter{0} == default(MemoryStream) ? default(JsonReaderData) : new JsonReaderData(namelessParameter{0});
namelessParameter{1} != default(JsonReaderData) ?
{
namelessParameter{2} = new Utf8JsonReaderManager(namelessParameter{1});
namelessParameter{2}.MoveNext();
namelessParameter{2}.CaptureState();
} : default(void);
namelessParameter{3} = new object[]{ (object)dataReader.GetInt32(1) };
namelessParameter{4} = ShaperProcessingExpressionVisitor.MaterializeJsonEntity<JsonOwnedLeaf>(
queryContext: queryContext,
keyPropertyValues: namelessParameter{3},
jsonReaderData: namelessParameter{1},
nullable: False,
shaper: (queryContext, namelessParameter{5}, namelessParameter{6}) =>
{
JsonOwnedLeaf namelessParameter{7};
return namelessParameter{7} =
{
MaterializationContext materializationContext1;
IEntityType entityType1;
JsonOwnedLeaf instance1;
materializationContext1 = new MaterializationContext(
ValueBuffer,
queryContext.Context
);
instance1 = null;
namelessParameter{5}[0] != null ?
{
ValueBuffer shadowValueBuffer1;
shadowValueBuffer1 = ValueBuffer;
entityType1 = EntityType: JsonEntityBasic.OwnedReferenceRoot#JsonOwnedRoot.OwnedReferenceBranch#JsonOwnedBranch.OwnedReferenceLeaf#JsonOwnedLeaf CLR Type: JsonOwnedLeaf Owned;
instance1 =
{
Utf8JsonReaderManager namelessParameter{8};
JsonTokenType tokenType;
JsonOwnedLeaf instance;
string namelessParameter{9};
namelessParameter{8} = new Utf8JsonReaderManager(namelessParameter{6});
tokenType = namelessParameter{8}.CurrentReader.TokenType;
Loop(Break: done Continue: )
{
{
tokenType = namelessParameter{8}.MoveNext();
tokenType != EndObject ? switch (tokenType)
{
case PropertyName:
namelessParameter{8}.CurrentReader.ValueTextEquals(SomethingSomething.EncodedUtf8Bytes) ?
{
namelessParameter{8}.MoveNext();
namelessParameter{9} = namelessParameter{8}.CurrentReader.TokenType == Null ? default(string) : (string)JsonStringReaderWriter.FromJson(namelessParameter{8});
} : default(void)
default:
{
namelessParameter{8}.CurrentReader.TrySkip();
}
}
: Goto(break done)
;
}}
namelessParameter{8}.CaptureState();
instance = new JsonOwnedLeaf();
instance.<SomethingSomething>k__BackingField = namelessParameter{9};
(instance is IInjectableService) ? ((IInjectableService)instance).Injected(
context: materializationContext1.Context,
entity: instance,
bindingInfo: ParameterBindingInfo) : default(void);
return instance;
};
return instance1;
} :
{
object[] keyValues1;
keyValues1 = new object[]{ namelessParameter{5}[0] };
return EntityMaterializerInjectingExpressionVisitor.CreateNullKeyValueInNoTrackingQuery(
entityType: EntityType: JsonEntityBasic.OwnedReferenceRoot#JsonOwnedRoot.OwnedReferenceBranch#JsonOwnedBranch.OwnedReferenceLeaf#JsonOwnedLeaf CLR Type: JsonOwnedLeaf Owned,
properties: List<RuntimeProperty> { RuntimeProperty },
keyValues: keyValues1);
};
return instance1;
};
});
return namelessParameter{4};
} |
nested scenario (still no-tracking) ss.Set<JsonEntityBasic>().Select(x => x.OwnedReferenceRoot.OwnedCollectionBranch).AsNoTracking() (queryContext, dataReader, resultContext, resultCoordinator) =>
{
Stream namelessParameter{0};
JsonReaderData namelessParameter{1};
Utf8JsonReaderManager namelessParameter{2};
object[] namelessParameter{3};
List<JsonOwnedBranch> namelessParameter{4};
namelessParameter{0} = dataReader.IsDBNull(0) ? default(MemoryStream) : new MemoryStream(Encoding.UTF8.GetBytes((string)dataReader.GetFieldValue<object>(0)));
namelessParameter{1} = namelessParameter{0} == default(MemoryStream) ? default(JsonReaderData) : new JsonReaderData(namelessParameter{0});
namelessParameter{1} != default(JsonReaderData) ?
{
namelessParameter{2} = new Utf8JsonReaderManager(namelessParameter{1});
namelessParameter{2}.MoveNext();
namelessParameter{2}.CaptureState();
} : default(void);
namelessParameter{3} = new object[]{ (object)dataReader.GetInt32(1) };
namelessParameter{4} = ShaperProcessingExpressionVisitor.MaterializeJsonEntityCollection<JsonOwnedBranch, List<JsonOwnedBranch>>(
queryContext: queryContext,
keyPropertyValues: namelessParameter{3},
jsonReaderData: namelessParameter{1},
navigation: Navigation: JsonEntityBasic.OwnedReferenceRoot#JsonOwnedRoot.OwnedCollectionBranch (List<JsonOwnedBranch>) Collection ToDependent JsonEntityBasic.OwnedReferenceRoot#JsonOwnedRoot.OwnedCollectionBranch#JsonOwnedBranch,
innerShaper: (queryContext, namelessParameter{5}, namelessParameter{6}) =>
{
JsonOwnedBranch namelessParameter{7};
return namelessParameter{7} =
{
MaterializationContext materializationContext1;
IEntityType entityType1;
JsonOwnedBranch instance1;
materializationContext1 = new MaterializationContext(
ValueBuffer,
queryContext.Context
);
instance1 = null;
namelessParameter{5}[0] != null && namelessParameter{5}[1] != null ?
{
ValueBuffer shadowValueBuffer1;
shadowValueBuffer1 = ValueBuffer;
entityType1 = EntityType: JsonEntityBasic.OwnedReferenceRoot#JsonOwnedRoot.OwnedCollectionBranch#JsonOwnedBranch CLR Type: JsonOwnedBranch Owned;
instance1 =
{
Utf8JsonReaderManager namelessParameter{8};
JsonTokenType tokenType;
JsonOwnedBranch instance;
DateTime namelessParameter{9};
JsonEnum namelessParameter{10};
decimal namelessParameter{11};
JsonEnum? namelessParameter{12};
List<JsonOwnedLeaf> namelessParameter{13};
JsonOwnedLeaf namelessParameter{14};
namelessParameter{8} = new Utf8JsonReaderManager(namelessParameter{6});
tokenType = namelessParameter{8}.CurrentReader.TokenType;
Loop(Break: done Continue: )
{
{
tokenType = namelessParameter{8}.MoveNext();
tokenType != EndObject ? switch (tokenType)
{
case PropertyName:
namelessParameter{8}.CurrentReader.ValueTextEquals(Date.EncodedUtf8Bytes) ?
{
namelessParameter{8}.MoveNext();
namelessParameter{9} = (DateTime)JsonDateTimeReaderWriter.FromJson(namelessParameter{8});
} : namelessParameter{8}.CurrentReader.ValueTextEquals(Enum.EncodedUtf8Bytes) ?
{
namelessParameter{8}.MoveNext();
namelessParameter{10} = StringEnumConverter<JsonEnum, string, JsonEnum>.ConvertToEnum((string)JsonStringReaderWriter.FromJson(namelessParameter{8}));
} : namelessParameter{8}.CurrentReader.ValueTextEquals(Fraction.EncodedUtf8Bytes) ?
{
namelessParameter{8}.MoveNext();
namelessParameter{11} = (decimal)JsonDecimalReaderWriter.FromJson(namelessParameter{8});
} : namelessParameter{8}.CurrentReader.ValueTextEquals(NullableEnum.EncodedUtf8Bytes) ?
{
namelessParameter{8}.MoveNext();
namelessParameter{12} = namelessParameter{8}.CurrentReader.TokenType == Null ? default(JsonEnum?) : (JsonEnum?)StringEnumConverter<JsonEnum, string, JsonEnum>.ConvertToEnum((string)JsonStringReaderWriter.FromJson(namelessParameter{8}));
} : namelessParameter{8}.CurrentReader.ValueTextEquals(OwnedCollectionLeaf.EncodedUtf8Bytes) ?
{
namelessParameter{8}.MoveNext();
namelessParameter{8}.CaptureState();
namelessParameter{13} = ShaperProcessingExpressionVisitor.MaterializeJsonEntityCollection<JsonOwnedLeaf, List<JsonOwnedLeaf>>(
queryContext: queryContext,
keyPropertyValues: namelessParameter{5},
jsonReaderData: namelessParameter{6},
navigation: Navigation: JsonEntityBasic.OwnedReferenceRoot#JsonOwnedRoot.OwnedCollectionBranch#JsonOwnedBranch.OwnedCollectionLeaf (List<JsonOwnedLeaf>) Collection ToDependent JsonEntityBasic.OwnedReferenceRoot#JsonOwnedRoot.OwnedCollectionBranch#JsonOwnedBranch.OwnedCollectionLeaf#JsonOwnedLeaf Inverse: Parent,
innerShaper: (queryContext, namelessParameter{15}, namelessParameter{16}) =>
{
JsonOwnedLeaf namelessParameter{17};
return namelessParameter{17} =
{
MaterializationContext materializationContext2;
IEntityType entityType2;
JsonOwnedLeaf instance2;
materializationContext2 = new MaterializationContext(
ValueBuffer,
queryContext.Context
);
instance2 = null;
namelessParameter{15}[0] != null && namelessParameter{15}[1] != null && namelessParameter{15}[2] != null ?
{
ValueBuffer shadowValueBuffer2;
shadowValueBuffer2 = ValueBuffer;
entityType2 = EntityType: JsonEntityBasic.OwnedReferenceRoot#JsonOwnedRoot.OwnedCollectionBranch#JsonOwnedBranch.OwnedCollectionLeaf#JsonOwnedLeaf CLR Type: JsonOwnedLeaf Owned;
instance2 =
{
Utf8JsonReaderManager namelessParameter{18};
JsonTokenType tokenType;
JsonOwnedLeaf instance;
string namelessParameter{19};
namelessParameter{18} = new Utf8JsonReaderManager(namelessParameter{16});
tokenType = namelessParameter{18}.CurrentReader.TokenType;
Loop(Break: done Continue: )
{
{
tokenType = namelessParameter{18}.MoveNext();
tokenType != EndObject ? switch (tokenType)
{
case PropertyName:
namelessParameter{18}.CurrentReader.ValueTextEquals(SomethingSomething.EncodedUtf8Bytes) ?
{
namelessParameter{18}.MoveNext();
namelessParameter{19} = namelessParameter{18}.CurrentReader.TokenType == Null ? default(string) : (string)JsonStringReaderWriter.FromJson(namelessParameter{18});
} : default(void)
default:
{
namelessParameter{18}.CurrentReader.TrySkip();
}
}
: Goto(break done)
;
}}
namelessParameter{18}.CaptureState();
instance = new JsonOwnedLeaf();
instance.<SomethingSomething>k__BackingField = namelessParameter{19};
(instance is IInjectableService) ? ((IInjectableService)instance).Injected(
context: materializationContext2.Context,
entity: instance,
bindingInfo: ParameterBindingInfo) : default(void);
return instance;
};
return instance2;
} : default(void);
return instance2;
};
});
namelessParameter{8} = new Utf8JsonReaderManager(namelessParameter{6});
} : namelessParameter{8}.CurrentReader.ValueTextEquals(OwnedReferenceLeaf.EncodedUtf8Bytes) ?
{
namelessParameter{8}.MoveNext();
namelessParameter{8}.CaptureState();
namelessParameter{14} = ShaperProcessingExpressionVisitor.MaterializeJsonEntity<JsonOwnedLeaf>(
queryContext: queryContext,
keyPropertyValues: namelessParameter{5},
jsonReaderData: namelessParameter{6},
nullable: True,
shaper: (queryContext, namelessParameter{20}, namelessParameter{21}) =>
{
JsonOwnedLeaf namelessParameter{22};
return namelessParameter{22} =
{
MaterializationContext materializationContext3;
IEntityType entityType3;
JsonOwnedLeaf instance3;
materializationContext3 = new MaterializationContext(
ValueBuffer,
queryContext.Context
);
instance3 = null;
namelessParameter{20}[0] != null && namelessParameter{20}[1] != null ?
{
ValueBuffer shadowValueBuffer3;
shadowValueBuffer3 = ValueBuffer;
entityType3 = EntityType: JsonEntityBasic.OwnedReferenceRoot#JsonOwnedRoot.OwnedCollectionBranch#JsonOwnedBranch.OwnedReferenceLeaf#JsonOwnedLeaf CLR Type: JsonOwnedLeaf Owned;
instance3 =
{
Utf8JsonReaderManager namelessParameter{23};
JsonTokenType tokenType;
JsonOwnedLeaf instance;
string namelessParameter{24};
namelessParameter{23} = new Utf8JsonReaderManager(namelessParameter{21});
tokenType = namelessParameter{23}.CurrentReader.TokenType;
Loop(Break: done Continue: )
{
{
tokenType = namelessParameter{23}.MoveNext();
tokenType != EndObject ? switch (tokenType)
{
case PropertyName:
namelessParameter{23}.CurrentReader.ValueTextEquals(SomethingSomething.EncodedUtf8Bytes) ?
{
namelessParameter{23}.MoveNext();
namelessParameter{24} = namelessParameter{23}.CurrentReader.TokenType == Null ? default(string) : (string)JsonStringReaderWriter.FromJson(namelessParameter{23});
} : default(void)
default:
{
namelessParameter{23}.CurrentReader.TrySkip();
}
}
: Goto(break done)
;
}}
namelessParameter{23}.CaptureState();
instance = new JsonOwnedLeaf();
instance.<SomethingSomething>k__BackingField = namelessParameter{24};
(instance is IInjectableService) ? ((IInjectableService)instance).Injected(
context: materializationContext3.Context,
entity: instance,
bindingInfo: ParameterBindingInfo) : default(void);
return instance;
};
return instance3;
} : default(void);
return instance3;
};
});
namelessParameter{8} = new Utf8JsonReaderManager(namelessParameter{6});
} : default(void)
default:
{
namelessParameter{8}.CurrentReader.TrySkip();
}
}
: Goto(break done)
;
}}
namelessParameter{8}.CaptureState();
instance = new JsonOwnedBranch();
instance.<Date>k__BackingField = namelessParameter{9};
instance.<Enum>k__BackingField = namelessParameter{10};
instance.<Fraction>k__BackingField = namelessParameter{11};
instance.<NullableEnum>k__BackingField = namelessParameter{12};
(instance is IInjectableService) ? ((IInjectableService)instance).Injected(
context: materializationContext1.Context,
entity: instance,
bindingInfo: ParameterBindingInfo) : default(void);
instance != null && namelessParameter{13} != null ? Invoke((namelessParameter{25}, namelessParameter{26}) =>
{
namelessParameter{25}.<OwnedCollectionLeaf>k__BackingField = namelessParameter{26};
ShaperProcessingExpressionVisitor.InverseCollectionFixup<JsonOwnedLeaf, JsonOwnedBranch>(
collection: namelessParameter{26},
entity: namelessParameter{25},
elementFixup: (namelessParameter{27}, namelessParameter{28}) =>
{
return namelessParameter{27}.<Parent>k__BackingField = namelessParameter{28};
});
}, instance, namelessParameter{13}) : default(void);
instance != null && namelessParameter{14} != null ? Invoke((namelessParameter{29}, namelessParameter{30}) =>
{
namelessParameter{29}.<OwnedReferenceLeaf>k__BackingField = namelessParameter{30};
return namelessParameter{30}.<Parent>k__BackingField = namelessParameter{29};
}, instance, namelessParameter{14}) : default(void);
return instance;
};
return instance1;
} : default(void);
return instance1;
};
});
return namelessParameter{4};
} |
tracking query with shadow properties, specifically note the differences between how we built materializer for tracked json entity vs non-json entity - there was a lot of rewrite there (queryContext, dataReader, resultContext, resultCoordinator) =>
{
MyEntityShadowProperties namelessParameter{0};
Stream namelessParameter{1};
JsonReaderData namelessParameter{2};
Utf8JsonReaderManager namelessParameter{3};
object[] namelessParameter{4};
Stream namelessParameter{5};
JsonReaderData namelessParameter{6};
Utf8JsonReaderManager namelessParameter{7};
object[] namelessParameter{8};
Stream namelessParameter{9};
JsonReaderData namelessParameter{10};
Utf8JsonReaderManager namelessParameter{11};
object[] namelessParameter{12};
Stream namelessParameter{13};
JsonReaderData namelessParameter{14};
Utf8JsonReaderManager namelessParameter{15};
object[] namelessParameter{16};
namelessParameter{0} =
{
MaterializationContext materializationContext1;
IEntityType entityType1;
MyEntityShadowProperties instance1;
InternalEntityEntry entry1;
bool hasNullKey1;
materializationContext1 = new MaterializationContext(
ValueBuffer,
queryContext.Context
);
instance1 = null;
entry1 = queryContext.TryGetEntry(
key: Key: MyEntityShadowProperties.Id PK,
keyValues: new object[]{ (object)dataReader.GetInt32(0) },
throwOnNullKey: True,
hasNullKey: hasNullKey1);
!(hasNullKey1) ? entry1 != default(InternalEntityEntry) ?
{
entityType1 = entry1.EntityType;
return instance1 = (MyEntityShadowProperties)entry1.Entity;
} :
{
ValueBuffer shadowValueBuffer1;
shadowValueBuffer1 = ValueBuffer;
entityType1 = EntityType: MyEntityShadowProperties;
instance1 = switch (entityType1)
{
case EntityType: MyEntityShadowProperties:
{
return
{
MyEntityShadowProperties instance;
instance = new MyEntityShadowProperties();
instance.<Id>k__BackingField = dataReader.GetInt32(0);
instance.<Name>k__BackingField = dataReader.IsDBNull(1) ? default(string) : (string)dataReader.GetFieldValue<object>(1);
(instance is IInjectableService) ? ((IInjectableService)instance).Injected(
context: materializationContext1.Context,
entity: instance,
bindingInfo: ParameterBindingInfo) : default(void);
return instance;
}}
default:
null
}
;
entry1 = entityType1 == default(IEntityType) ? default(InternalEntityEntry) : queryContext.StartTracking(
entityType: entityType1,
entity: instance1,
valueBuffer: shadowValueBuffer1);
return instance1;
} : default(void);
return instance1;
};
namelessParameter{1} = dataReader.IsDBNull(2) ? default(MemoryStream) : new MemoryStream(Encoding.UTF8.GetBytes((string)dataReader.GetFieldValue<object>(2)));
namelessParameter{2} = namelessParameter{1} == default(MemoryStream) ? default(JsonReaderData) : new JsonReaderData(namelessParameter{1});
namelessParameter{2} != default(JsonReaderData) ?
{
namelessParameter{3} = new Utf8JsonReaderManager(namelessParameter{2});
namelessParameter{3}.MoveNext();
namelessParameter{3}.CaptureState();
} : default(void);
namelessParameter{4} = new object[]{ (object)dataReader.GetInt32(0) };
namelessParameter{5} = dataReader.IsDBNull(3) ? default(MemoryStream) : new MemoryStream(Encoding.UTF8.GetBytes((string)dataReader.GetFieldValue<object>(3)));
namelessParameter{6} = namelessParameter{5} == default(MemoryStream) ? default(JsonReaderData) : new JsonReaderData(namelessParameter{5});
namelessParameter{6} != default(JsonReaderData) ?
{
namelessParameter{7} = new Utf8JsonReaderManager(namelessParameter{6});
namelessParameter{7}.MoveNext();
namelessParameter{7}.CaptureState();
} : default(void);
namelessParameter{8} = new object[]{ (object)dataReader.GetInt32(0) };
namelessParameter{9} = dataReader.IsDBNull(4) ? default(MemoryStream) : new MemoryStream(Encoding.UTF8.GetBytes((string)dataReader.GetFieldValue<object>(4)));
namelessParameter{10} = namelessParameter{9} == default(MemoryStream) ? default(JsonReaderData) : new JsonReaderData(namelessParameter{9});
namelessParameter{10} != default(JsonReaderData) ?
{
namelessParameter{11} = new Utf8JsonReaderManager(namelessParameter{10});
namelessParameter{11}.MoveNext();
namelessParameter{11}.CaptureState();
} : default(void);
namelessParameter{12} = new object[]{ (object)dataReader.GetInt32(0) };
namelessParameter{13} = dataReader.IsDBNull(5) ? default(MemoryStream) : new MemoryStream(Encoding.UTF8.GetBytes((string)dataReader.GetFieldValue<object>(5)));
namelessParameter{14} = namelessParameter{13} == default(MemoryStream) ? default(JsonReaderData) : new JsonReaderData(namelessParameter{13});
namelessParameter{14} != default(JsonReaderData) ?
{
namelessParameter{15} = new Utf8JsonReaderManager(namelessParameter{14});
namelessParameter{15}.MoveNext();
namelessParameter{15}.CaptureState();
} : default(void);
namelessParameter{16} = new object[]{ (object)dataReader.GetInt32(0) };
ShaperProcessingExpressionVisitor.IncludeJsonEntityCollection<MyEntityShadowProperties, MyJsonEntityShadowProperties>(
queryContext: queryContext,
keyPropertyValues: namelessParameter{4},
jsonReaderData: namelessParameter{2},
entity: namelessParameter{0},
innerShaper: (queryContext, namelessParameter{17}, namelessParameter{18}) =>
{
MyJsonEntityShadowProperties namelessParameter{19};
return namelessParameter{19} =
{
MaterializationContext materializationContext2;
IEntityType entityType2;
MyJsonEntityShadowProperties instance2;
InternalEntityEntry entry2;
bool hasNullKey2;
materializationContext2 = new MaterializationContext(
ValueBuffer,
queryContext.Context
);
instance2 = null;
entry2 = queryContext.TryGetEntry(
key: Key: MyEntityShadowProperties.Collection#MyJsonEntityShadowProperties.MyEntityShadowPropertiesId, MyEntityShadowProperties.Collection#MyJsonEntityShadowProperties.Id PK,
keyValues: new object[]
{
namelessParameter{17}[0],
namelessParameter{17}[1]
},
throwOnNullKey: False,
hasNullKey: hasNullKey2);
!(hasNullKey2) ?
{
bool namelessParameter{20};
ValueBuffer shadowValueBuffer2;
namelessParameter{20} = False;
shadowValueBuffer2 = ValueBuffer;
entityType2 = EntityType: MyEntityShadowProperties.Collection#MyJsonEntityShadowProperties CLR Type: MyJsonEntityShadowProperties Owned;
entry2 != default(InternalEntityEntry) ?
{
entityType2 = entry2.EntityType;
instance2 = (MyJsonEntityShadowProperties)entry2.Entity;
namelessParameter{20} = True;
} : default(void);
instance2 =
{
Utf8JsonReaderManager namelessParameter{21};
JsonTokenType tokenType;
MyJsonEntityShadowProperties instance;
object namelessParameter{22};
string namelessParameter{23};
namelessParameter{21} = new Utf8JsonReaderManager(namelessParameter{18});
tokenType = namelessParameter{21}.CurrentReader.TokenType;
Loop(Break: done Continue: )
{
{
tokenType = namelessParameter{21}.MoveNext();
tokenType != EndObject ? switch (tokenType)
{
case PropertyName:
namelessParameter{21}.CurrentReader.ValueTextEquals(ShadowDouble.EncodedUtf8Bytes) ?
{
namelessParameter{21}.MoveNext();
namelessParameter{22} = (object)(double)JsonDoubleReaderWriter.FromJson(namelessParameter{21});
} : namelessParameter{21}.CurrentReader.ValueTextEquals(Name.EncodedUtf8Bytes) ?
{
namelessParameter{21}.MoveNext();
namelessParameter{23} = namelessParameter{21}.CurrentReader.TokenType == Null ? default(string) : (string)JsonStringReaderWriter.FromJson(namelessParameter{21});
} : default(void)
default:
{
namelessParameter{21}.CurrentReader.TrySkip();
}
}
: Goto(break done)
;
}}
namelessParameter{21}.CaptureState();
!(namelessParameter{20}) ? shadowValueBuffer2 = new ValueBuffer(new object[]
{
namelessParameter{17}[0],
namelessParameter{17}[1],
namelessParameter{22}
}) : default(void);
namelessParameter{20} ?
{
instance = instance2;
} :
{
instance = new MyJsonEntityShadowProperties();
instance.<Name>k__BackingField = namelessParameter{23};
(instance is IInjectableService) ? ((IInjectableService)instance).Injected(
context: materializationContext2.Context,
entity: instance,
bindingInfo: ParameterBindingInfo) : default(void);
};
return instance;
};
!(namelessParameter{20} || entityType2 == default(IEntityType)) ?
{
queryContext.StartTracking(
entityType: entityType2,
entity: instance2,
valueBuffer: shadowValueBuffer2);
} : default(void);
} : default(void);
return instance2;
};
},
fixup: (namelessParameter{24}, namelessParameter{25}) =>
{
return ClrICollectionAccessor<MyEntityShadowProperties, List<MyJsonEntityShadowProperties>, MyJsonEntityShadowProperties>.Add(
entity: namelessParameter{24},
value: namelessParameter{25},
forMaterialization: True);
});
ShaperProcessingExpressionVisitor.IncludeJsonEntityCollection<MyEntityShadowProperties, MyJsonEntityShadowPropertiesWithCtor>(
queryContext: queryContext,
keyPropertyValues: namelessParameter{8},
jsonReaderData: namelessParameter{6},
entity: namelessParameter{0},
innerShaper: (queryContext, namelessParameter{26}, namelessParameter{27}) =>
{
MyJsonEntityShadowPropertiesWithCtor namelessParameter{28};
return namelessParameter{28} =
{
MaterializationContext materializationContext3;
IEntityType entityType3;
MyJsonEntityShadowPropertiesWithCtor instance3;
InternalEntityEntry entry3;
bool hasNullKey3;
materializationContext3 = new MaterializationContext(
ValueBuffer,
queryContext.Context
);
instance3 = null;
entry3 = queryContext.TryGetEntry(
key: Key: MyEntityShadowProperties.CollectionWithCtor#MyJsonEntityShadowPropertiesWithCtor.MyEntityShadowPropertiesId, MyEntityShadowProperties.CollectionWithCtor#MyJsonEntityShadowPropertiesWithCtor.Id PK,
keyValues: new object[]
{
namelessParameter{26}[0],
namelessParameter{26}[1]
},
throwOnNullKey: False,
hasNullKey: hasNullKey3);
!(hasNullKey3) ?
{
bool namelessParameter{29};
ValueBuffer shadowValueBuffer3;
namelessParameter{29} = False;
shadowValueBuffer3 = ValueBuffer;
entityType3 = EntityType: MyEntityShadowProperties.CollectionWithCtor#MyJsonEntityShadowPropertiesWithCtor CLR Type: MyJsonEntityShadowPropertiesWithCtor Owned;
entry3 != default(InternalEntityEntry) ?
{
entityType3 = entry3.EntityType;
instance3 = (MyJsonEntityShadowPropertiesWithCtor)entry3.Entity;
namelessParameter{29} = True;
} : default(void);
instance3 =
{
Utf8JsonReaderManager namelessParameter{30};
JsonTokenType tokenType;
MyJsonEntityShadowPropertiesWithCtor instance;
object namelessParameter{31};
string namelessParameter{32};
namelessParameter{30} = new Utf8JsonReaderManager(namelessParameter{27});
tokenType = namelessParameter{30}.CurrentReader.TokenType;
Loop(Break: done Continue: )
{
{
tokenType = namelessParameter{30}.MoveNext();
tokenType != EndObject ? switch (tokenType)
{
case PropertyName:
namelessParameter{30}.CurrentReader.ValueTextEquals(ShadowNullableByte.EncodedUtf8Bytes) ?
{
namelessParameter{30}.MoveNext();
namelessParameter{31} = (object)namelessParameter{30}.CurrentReader.TokenType == Null ? default(byte?) : (byte?)(byte)JsonByteReaderWriter.FromJson(namelessParameter{30});
} : namelessParameter{30}.CurrentReader.ValueTextEquals(Name.EncodedUtf8Bytes) ?
{
namelessParameter{30}.MoveNext();
namelessParameter{32} = namelessParameter{30}.CurrentReader.TokenType == Null ? default(string) : (string)JsonStringReaderWriter.FromJson(namelessParameter{30});
} : default(void)
default:
{
namelessParameter{30}.CurrentReader.TrySkip();
}
}
: Goto(break done)
;
}}
namelessParameter{30}.CaptureState();
!(namelessParameter{29}) ? shadowValueBuffer3 = new ValueBuffer(new object[]
{
namelessParameter{26}[0],
namelessParameter{26}[1],
namelessParameter{31}
}) : default(void);
namelessParameter{29} ?
{
instance = instance3;
} :
{
instance = new MyJsonEntityShadowPropertiesWithCtor(namelessParameter{32});
};
return instance;
};
!(namelessParameter{29} || entityType3 == default(IEntityType)) ?
{
queryContext.StartTracking(
entityType: entityType3,
entity: instance3,
valueBuffer: shadowValueBuffer3);
} : default(void);
} : default(void);
return instance3;
};
},
fixup: (namelessParameter{33}, namelessParameter{34}) =>
{
return ClrICollectionAccessor<MyEntityShadowProperties, List<MyJsonEntityShadowPropertiesWithCtor>, MyJsonEntityShadowPropertiesWithCtor>.Add(
entity: namelessParameter{33},
value: namelessParameter{34},
forMaterialization: True);
});
ShaperProcessingExpressionVisitor.IncludeJsonEntityReference<MyEntityShadowProperties, MyJsonEntityShadowProperties>(
queryContext: queryContext,
keyPropertyValues: namelessParameter{12},
jsonReaderData: namelessParameter{10},
entity: namelessParameter{0},
innerShaper: (queryContext, namelessParameter{35}, namelessParameter{36}) =>
{
MyJsonEntityShadowProperties namelessParameter{37};
return namelessParameter{37} =
{
MaterializationContext materializationContext4;
IEntityType entityType4;
MyJsonEntityShadowProperties instance4;
InternalEntityEntry entry4;
bool hasNullKey4;
materializationContext4 = new MaterializationContext(
ValueBuffer,
queryContext.Context
);
instance4 = null;
entry4 = queryContext.TryGetEntry(
key: Key: MyEntityShadowProperties.Reference#MyJsonEntityShadowProperties.MyEntityShadowPropertiesId PK,
keyValues: new object[]{ namelessParameter{35}[0] },
throwOnNullKey: False,
hasNullKey: hasNullKey4);
!(hasNullKey4) ?
{
bool namelessParameter{38};
ValueBuffer shadowValueBuffer4;
namelessParameter{38} = False;
shadowValueBuffer4 = ValueBuffer;
entityType4 = EntityType: MyEntityShadowProperties.Reference#MyJsonEntityShadowProperties CLR Type: MyJsonEntityShadowProperties Owned;
entry4 != default(InternalEntityEntry) ?
{
entityType4 = entry4.EntityType;
instance4 = (MyJsonEntityShadowProperties)entry4.Entity;
namelessParameter{38} = True;
} : default(void);
instance4 =
{
Utf8JsonReaderManager namelessParameter{39};
JsonTokenType tokenType;
MyJsonEntityShadowProperties instance;
object namelessParameter{40};
string namelessParameter{41};
namelessParameter{39} = new Utf8JsonReaderManager(namelessParameter{36});
tokenType = namelessParameter{39}.CurrentReader.TokenType;
Loop(Break: done Continue: )
{
{
tokenType = namelessParameter{39}.MoveNext();
tokenType != EndObject ? switch (tokenType)
{
case PropertyName:
namelessParameter{39}.CurrentReader.ValueTextEquals(ShadowString.EncodedUtf8Bytes) ?
{
namelessParameter{39}.MoveNext();
namelessParameter{40} = (object)namelessParameter{39}.CurrentReader.TokenType == Null ? default(string) : (string)JsonStringReaderWriter.FromJson(namelessParameter{39});
} : namelessParameter{39}.CurrentReader.ValueTextEquals(Name.EncodedUtf8Bytes) ?
{
namelessParameter{39}.MoveNext();
namelessParameter{41} = namelessParameter{39}.CurrentReader.TokenType == Null ? default(string) : (string)JsonStringReaderWriter.FromJson(namelessParameter{39});
} : default(void)
default:
{
namelessParameter{39}.CurrentReader.TrySkip();
}
}
: Goto(break done)
;
}}
namelessParameter{39}.CaptureState();
!(namelessParameter{38}) ? shadowValueBuffer4 = new ValueBuffer(new object[]
{
namelessParameter{35}[0],
namelessParameter{40}
}) : default(void);
namelessParameter{38} ?
{
instance = instance4;
} :
{
instance = new MyJsonEntityShadowProperties();
instance.<Name>k__BackingField = namelessParameter{41};
(instance is IInjectableService) ? ((IInjectableService)instance).Injected(
context: materializationContext4.Context,
entity: instance,
bindingInfo: ParameterBindingInfo) : default(void);
};
return instance;
};
!(namelessParameter{38} || entityType4 == default(IEntityType)) ?
{
queryContext.StartTracking(
entityType: entityType4,
entity: instance4,
valueBuffer: shadowValueBuffer4);
} : default(void);
} : default(void);
return instance4;
};
},
fixup: (namelessParameter{42}, namelessParameter{43}) =>
{
return namelessParameter{42}.<Reference>k__BackingField = namelessParameter{43};
});
ShaperProcessingExpressionVisitor.IncludeJsonEntityReference<MyEntityShadowProperties, MyJsonEntityShadowPropertiesWithCtor>(
queryContext: queryContext,
keyPropertyValues: namelessParameter{16},
jsonReaderData: namelessParameter{14},
entity: namelessParameter{0},
innerShaper: (queryContext, namelessParameter{44}, namelessParameter{45}) =>
{
MyJsonEntityShadowPropertiesWithCtor namelessParameter{46};
return namelessParameter{46} =
{
MaterializationContext materializationContext5;
IEntityType entityType5;
MyJsonEntityShadowPropertiesWithCtor instance5;
InternalEntityEntry entry5;
bool hasNullKey5;
materializationContext5 = new MaterializationContext(
ValueBuffer,
queryContext.Context
);
instance5 = null;
entry5 = queryContext.TryGetEntry(
key: Key: MyEntityShadowProperties.ReferenceWithCtor#MyJsonEntityShadowPropertiesWithCtor.MyEntityShadowPropertiesId PK,
keyValues: new object[]{ namelessParameter{44}[0] },
throwOnNullKey: False,
hasNullKey: hasNullKey5);
!(hasNullKey5) ?
{
bool namelessParameter{47};
ValueBuffer shadowValueBuffer5;
namelessParameter{47} = False;
shadowValueBuffer5 = ValueBuffer;
entityType5 = EntityType: MyEntityShadowProperties.ReferenceWithCtor#MyJsonEntityShadowPropertiesWithCtor CLR Type: MyJsonEntityShadowPropertiesWithCtor Owned;
entry5 != default(InternalEntityEntry) ?
{
entityType5 = entry5.EntityType;
instance5 = (MyJsonEntityShadowPropertiesWithCtor)entry5.Entity;
namelessParameter{47} = True;
} : default(void);
instance5 =
{
Utf8JsonReaderManager namelessParameter{48};
JsonTokenType tokenType;
MyJsonEntityShadowPropertiesWithCtor instance;
object namelessParameter{49};
string namelessParameter{50};
namelessParameter{48} = new Utf8JsonReaderManager(namelessParameter{45});
tokenType = namelessParameter{48}.CurrentReader.TokenType;
Loop(Break: done Continue: )
{
{
tokenType = namelessParameter{48}.MoveNext();
tokenType != EndObject ? switch (tokenType)
{
case PropertyName:
namelessParameter{48}.CurrentReader.ValueTextEquals(ShadowInt.EncodedUtf8Bytes) ?
{
namelessParameter{48}.MoveNext();
namelessParameter{49} = (object)(int)JsonInt32ReaderWriter.FromJson(namelessParameter{48});
} : namelessParameter{48}.CurrentReader.ValueTextEquals(Name.EncodedUtf8Bytes) ?
{
namelessParameter{48}.MoveNext();
namelessParameter{50} = namelessParameter{48}.CurrentReader.TokenType == Null ? default(string) : (string)JsonStringReaderWriter.FromJson(namelessParameter{48});
} : default(void)
default:
{
namelessParameter{48}.CurrentReader.TrySkip();
}
}
: Goto(break done)
;
}}
namelessParameter{48}.CaptureState();
!(namelessParameter{47}) ? shadowValueBuffer5 = new ValueBuffer(new object[]
{
namelessParameter{44}[0],
namelessParameter{49}
}) : default(void);
namelessParameter{47} ?
{
instance = instance5;
} :
{
instance = new MyJsonEntityShadowPropertiesWithCtor(namelessParameter{50});
};
return instance;
};
!(namelessParameter{47} || entityType5 == default(IEntityType)) ?
{
queryContext.StartTracking(
entityType: entityType5,
entity: instance5,
valueBuffer: shadowValueBuffer5);
} : default(void);
} : default(void);
return instance5;
};
},
fixup: (namelessParameter{51}, namelessParameter{52}) =>
{
return namelessParameter{51}.<ReferenceWithCtor>k__BackingField = namelessParameter{52};
});
return namelessParameter{0};
} |
f23c8cd
to
1fc77a3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Spent some time reviewing this; I'm definitely far from grokking everything that goes on :) But overall it looks good, a a good basis to build on for better perf in the future.
See comments which are almost only nits. I do think we should start systematically assigning names to ParameterExpressions we create, Having to figure out what namelessParameter{7}
means in the generated code isn't easy...
...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs
Show resolved
Hide resolved
...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs
Outdated
Show resolved
Hide resolved
...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs
Outdated
Show resolved
Hide resolved
...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs
Outdated
Show resolved
Hide resolved
...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs
Show resolved
Hide resolved
|
||
if (value is not null) | ||
{ | ||
property.GetJsonValueReaderWriter()!.ToJson(writer, value); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ajcvickers naming nit: should this simply be called a converter?
...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs
Show resolved
Hide resolved
…JsonReader/Utf8JsonWriter Using Utf8JsonReader to read JSON data rather than caching it using DOM. This should reduce allocations significantly. Tricky part is that entity materializers are build in a way that assumes we have random access to all the data we need. This is not the case here. We read JSON data sequentially and can only do it once, and we don't know the order in which we get the data. This is somewhat problematic in case where entity takes argument in the constructor. Those could be at the very end of the JSON string, so we must read all the data before we can instantiate the object, and populate it's properties and do navigation fixup. This requires us reading all the JSON data, store them in local variables, and only when we are done reading we instantiate the entity and populate all the properties with data stored in those variables. This adds some allocations (specifically navigations). We also have to disable de-duplication logic - we can't always safely re-read the JSON string, and definitely can't start reading it from arbitrary position, so now we have to add JSON string for every aggregate projected, even if we already project it's parent. Serialization implementation (i.e. Utf8JsonWriter) is pretty straighforward. Also fix to #30993 - Query/Json: data corruption for tracking queries with nested json entities, then updating nested entities outside EF and re-querying Fix is to recognize and modify shaper in case of tracking query, so that nav expansions are not skipped when parent entity is found in Change Tracker. This is necessary to fix alongside streaming, because now we throw exception from reader (unexpected token) if we don't process the entire stream correctly. Before it would be silently ignored apart from the edge case described in the bug. Fixes #30604 Fixes #30993
@@ -654,7 +649,6 @@ private ProjectionBindingExpression AddClientProjection(Expression expression, T | |||
return new ProjectionBindingExpression(_selectExpression, existingIndex, type); | |||
} | |||
|
|||
#pragma warning disable IDE0052 // Remove unread private members | |||
private static T GetParameterValue<T>(QueryContext queryContext, string parameterName) | |||
#pragma warning restore IDE0052 // Remove unread private members |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this line still needed?
} | ||
|
||
private static TResult? MaterializeJsonEntityCollection<TEntity, TResult>( | ||
private static void IncludeJsonEntityCollection<TIncludingEntity, TIncludedCollectionElement>( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can be refactored to share implementation with MaterializeJsonEntityCollection
Using Utf8JsonReader to read JSON data rather than caching it using DOM. This should reduce allocations significantly. Tricky part is that entity materializers are build in a way that assumes we have random access to all the data we need. This is not the case here.
We read JSON data sequentially and can only do it once, and we don't know the order in which we get the data. This is somewhat problematic in case where entity takes argument in the constructor. Those could be at the very end of the JSON string, so we must read all the data before we can instantiate the object, and populate it's properties and do navigation fixup.
This requires us reading all the JSON data, store them in local variables, and only when we are done reading we instantiate the entity and populate all the properties with data stored in those variables. This adds some allocations (specifically navigations).
We also have to disable de-duplication logic - we can't always safely re-read the JSON string, and definitely can't start reading it from arbitrary position, so now we have to add JSON string for every aggregate projected, even if we already project it's parent.
Serialization implementation (i.e. Utf8JsonWriter) is pretty straighforward.
Also fix to #30993 - Query/Json: data corruption for tracking queries with nested json entities, then updating nested entities outside EF and re-querying
Fix is to recognize and modify shaper in case of tracking query, so that nav expansions are not skipped when parent entity is found in Change Tracker. This is necessary to fix alongside streaming, because now we throw exception from reader (unexpected token) if we don't process the entire stream correctly. Before it would be silently ignored apart from the edge case described in the bug.
Fixes #30604
Fixes #30993