Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test failure: FindServicePoint_ReturnedServicePointMatchesExpectedValues #56689

Closed
karelz opened this issue Aug 1, 2021 · 13 comments · Fixed by #57016
Closed

Test failure: FindServicePoint_ReturnedServicePointMatchesExpectedValues #56689

karelz opened this issue Aug 1, 2021 · 13 comments · Fixed by #57016
Assignees
Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI test-run-core Test failures in .NET Core test runs
Milestone

Comments

@karelz
Copy link
Member

karelz commented Aug 1, 2021

Test: System.Net.Tests.ServicePointManagerTest.FindServicePoint_ReturnedServicePointMatchesExpectedValues

Failures 6/1-8/1 (incl. PRs):

Day Run OS
7/25 fullpgo Windows.10.Arm64v8.Open
7/31 PR #56651 - fullpgo Ubuntu.1804.Arm64.Open
7/31 fullpgo Ubuntu.1804.Arm64.Open

Failure:

System.MissingMethodException : .ctor

   at System.RuntimeMethodHandle.IsCAVisibleFromDecoratedType(QCallTypeHandle attrTypeHandle, RuntimeMethodHandleInternal attrCtor, QCallTypeHandle sourceTypeHandle, QCallModule sourceModule)
   at System.Reflection.CustomAttribute.AddCustomAttributes(ListBuilder`1& attributes, RuntimeModule decoratedModule, Int32 decoratedMetadataToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1 derivedAttributes) in /_/src/coreclr/System.Private.CoreLib/src/System/Reflection/RuntimeCustomAttributeData.cs:line 1263
   at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType) in /_/src/coreclr/System.Private.CoreLib/src/System/Reflection/RuntimeCustomAttributeData.cs:line 1132
   at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeAssembly assembly, RuntimeType caType) in /_/src/coreclr/System.Private.CoreLib/src/System/Reflection/RuntimeCustomAttributeData.cs:line 1051
   at System.Reflection.RuntimeAssembly.GetCustomAttributes(Type attributeType, Boolean inherit) in /_/src/coreclr/System.Private.CoreLib/src/System/Reflection/RuntimeAssembly.cs:line 307
   at System.Attribute.GetCustomAttributes(Assembly element, Type attributeType, Boolean inherit) in /_/src/coreclr/System.Private.CoreLib/src/System/Attribute.CoreCLR.cs:line 737
   at System.Attribute.GetCustomAttribute(Assembly element, Type attributeType, Boolean inherit) in /_/src/coreclr/System.Private.CoreLib/src/System/Attribute.CoreCLR.cs:line 782
   at System.Attribute.GetCustomAttribute(Assembly element, Type attributeType) in /_/src/coreclr/System.Private.CoreLib/src/System/Attribute.CoreCLR.cs:line 775
   at System.Reflection.CustomAttributeExtensions.GetCustomAttribute(Assembly element, Type attributeType) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/CustomAttributeExtensions.cs:line 13
   at System.Reflection.CustomAttributeExtensions.GetCustomAttribute[T](Assembly element) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/CustomAttributeExtensions.cs:line 30
   at System.Environment.get_Version() in /_/src/libraries/System.Private.CoreLib/src/System/Environment.cs:line 193
   at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.IsNetCore() in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteExecutor.cs:line 96
   at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.GetConsoleAppArgs(RemoteInvokeOptions options, IEnumerable`1& toDispose) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteExecutor.cs:line 421
   at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.Invoke(MethodInfo method, String[] args, RemoteInvokeOptions options, Boolean pasteArguments) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteExecutor.cs:line 397
   at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.Invoke(Action method, RemoteInvokeOptions options) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteExecutor.cs:line 118
   at System.Net.Tests.ServicePointManagerTest.FindServicePoint_ReturnedServicePointMatchesExpectedValues() in /_/src/libraries/System.Net.ServicePoint/tests/ServicePointManagerTest.cs:line 307
@karelz karelz added area-System.Net test-run-core Test failures in .NET Core test runs labels Aug 1, 2021
@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Aug 1, 2021
@ghost
Copy link

ghost commented Aug 1, 2021

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

Test: System.Net.Tests.ServicePointManagerTest.FindServicePoint_ReturnedServicePointMatchesExpectedValues

Failures 6/1-8/1 (incl. PRs):

Day Run OS
7/25 fullpgo Windows.10.Arm64v8.Open
7/31 PR #56651 - fullpgo Ubuntu.1804.Arm64.Open
7/31 fullpgo Ubuntu.1804.Arm64.Open

Failure:

System.MissingMethodException : .ctor

   at System.RuntimeMethodHandle.IsCAVisibleFromDecoratedType(QCallTypeHandle attrTypeHandle, RuntimeMethodHandleInternal attrCtor, QCallTypeHandle sourceTypeHandle, QCallModule sourceModule)
   at System.Reflection.CustomAttribute.AddCustomAttributes(ListBuilder`1& attributes, RuntimeModule decoratedModule, Int32 decoratedMetadataToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1 derivedAttributes) in /_/src/coreclr/System.Private.CoreLib/src/System/Reflection/RuntimeCustomAttributeData.cs:line 1263
   at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType) in /_/src/coreclr/System.Private.CoreLib/src/System/Reflection/RuntimeCustomAttributeData.cs:line 1132
   at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeAssembly assembly, RuntimeType caType) in /_/src/coreclr/System.Private.CoreLib/src/System/Reflection/RuntimeCustomAttributeData.cs:line 1051
   at System.Reflection.RuntimeAssembly.GetCustomAttributes(Type attributeType, Boolean inherit) in /_/src/coreclr/System.Private.CoreLib/src/System/Reflection/RuntimeAssembly.cs:line 307
   at System.Attribute.GetCustomAttributes(Assembly element, Type attributeType, Boolean inherit) in /_/src/coreclr/System.Private.CoreLib/src/System/Attribute.CoreCLR.cs:line 737
   at System.Attribute.GetCustomAttribute(Assembly element, Type attributeType, Boolean inherit) in /_/src/coreclr/System.Private.CoreLib/src/System/Attribute.CoreCLR.cs:line 782
   at System.Attribute.GetCustomAttribute(Assembly element, Type attributeType) in /_/src/coreclr/System.Private.CoreLib/src/System/Attribute.CoreCLR.cs:line 775
   at System.Reflection.CustomAttributeExtensions.GetCustomAttribute(Assembly element, Type attributeType) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/CustomAttributeExtensions.cs:line 13
   at System.Reflection.CustomAttributeExtensions.GetCustomAttribute[T](Assembly element) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/CustomAttributeExtensions.cs:line 30
   at System.Environment.get_Version() in /_/src/libraries/System.Private.CoreLib/src/System/Environment.cs:line 193
   at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.IsNetCore() in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteExecutor.cs:line 96
   at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.GetConsoleAppArgs(RemoteInvokeOptions options, IEnumerable`1& toDispose) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteExecutor.cs:line 421
   at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.Invoke(MethodInfo method, String[] args, RemoteInvokeOptions options, Boolean pasteArguments) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteExecutor.cs:line 397
   at Microsoft.DotNet.RemoteExecutor.RemoteExecutor.Invoke(Action method, RemoteInvokeOptions options) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteExecutor.cs:line 118
   at System.Net.Tests.ServicePointManagerTest.FindServicePoint_ReturnedServicePointMatchesExpectedValues() in /_/src/libraries/System.Net.ServicePoint/tests/ServicePointManagerTest.cs:line 307
Author: karelz
Assignees: -
Labels:

area-System.Net, test-run-core

Milestone: -

@karelz
Copy link
Member Author

karelz commented Aug 1, 2021

@dotnet/jit-contrib this looks like PGO run failure only + missing method ... is it a known problem?

@JulieLeeMSFT JulieLeeMSFT removed the untriaged New issue has not been triaged by the area owner label Aug 2, 2021
@JulieLeeMSFT
Copy link
Member

@jakobbotsch PTAL.

@AndyAyersMS
Copy link
Member

@jakobbotsch I'll probably be looking at this some too, since this is the last major failure symptom in the libraries-pgo runs, so let's share notes/progress.

@AndyAyersMS AndyAyersMS added area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI and removed area-System.Net labels Aug 3, 2021
@AndyAyersMS
Copy link
Member

Assuming this is codegen, so updated area label.

@AndyAyersMS
Copy link
Member

I've been trying to repro this by running the arm64 System.Console.Tests with checked jit/runtime/spc and with the right COMPlus settings on my RPI4; no luck so far. May try switching to a different host machine.

@AndyAyersMS
Copy link
Member

Also note this test DLL seems unable to be run via corerun, you need to swap parts into the test host and run that way, as some tests try to re-exec dotnet.

@AndyAyersMS
Copy link
Member

Ok, I can repro if I bog the machine down with stress -c 4 ...

@AndyAyersMS
Copy link
Member

I've isolated this down to something PGO does to System.Reflection.CustomAttribute.AddCustomAttributes(ListBuilder1& attributes, RuntimeModule decoratedModule, Int32 decoratedMetadataToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder1 derivedAttributes).

Suspect it involves inlining so will try and pin that down next.

@AndyAyersMS
Copy link
Member

Inline analysis indicates it's likely this chain of inlines (plus maybe some others)

Inlines into 06005215 [via ReplayPolicy] System.Reflection.CustomAttribute:AddCustomAttributes....
  [11 IL=0103 TR=000076 06005216] [accepted by log replay] System.Reflection.CustomAttribute:FilterCustomAttributeRecord(System.Reflection.MetadataToken,byref,System.Reflection.RuntimeModule,System.Reflection.MetadataToken,System.RuntimeType,bool,byref,byref,byref,byref):bool
    [103 IL=0510 TR=000702 06004F1B] [accepted by log replay] System.Runtime.CompilerServices.QCallTypeHandle:.ctor(byref):this
      [105 IL=0014 TR=002046 06000506] [accepted by log replay] System.RuntimeTypeHandle:get_Value():long:this

@AndyAyersMS
Copy link
Member

I don't have a general-purpose inline XML reducer (yet). The method normally does 136 inlines, and if I cut off inlining at 104, the tests pass, and at 105 they fail.

Inline 105 is RuntimeTypeHandle:get_Value noted above. While it is a small method, it has control flow, and biased PGO data, and inlining it leads to wide scale block reordering, so it's not that easy to diff between the 104 and 105 codegen to see if there are any unexpected changes. Disabling optOptimizeLayout in an attempt to minimize the ordering diffs makes the 105 case pass, so layout plays some kind of role in the bug.

So I may work on reducing the inline set further; I suspect there's a version that does a much smaller number of inlines that perhaps will have more tractable diffs.

Other options are to try and debug the failure, try disabling other parts of the jit, find a simpler test case that also fails, or rely on some psychic debugging skills. I've tried the latter two and so far they haven't panned out.

@AndyAyersMS
Copy link
Member

I managed to narrow it down by first noting that the failure required assertion prop and then by limiting the number of allowed assertions. This lead to a one-instruction diff in the generated code:

;; good (7 assertions)
IN0146: 000568                    mov     x0, #0
IN0147: 00056C                    mov     x2, x1
IN0148: 000570                    str     x20, [fp,#232]	// [V01 arg1]
IN0149: 000574                    str     x2, [fp,#200]	// [V13 loc7]

;; bad (8 assertions)
IN0146: 000568                    mov     x1, #0
IN0147: 00056C                    str     x20, [fp,#232]	// [V01 arg1]
IN0148: 000570                    str     x1, [fp,#200]	// [V13 loc7]

and the upshot it looks like there's another invalid peephole:

Generating: N1107 (  1,  2) [002721] ------------      t2721 =    CNS_INT   ref    null REG x1 $VN.Null
IN0146:                           mov     x1, #0
							GC regs: 100001 {x0 x20} => 100003 {x0 x1 x20}
                                                              /--*  t2721  ref    
Generating: N1109 (  2,  3) [001470] ----G-------              *  KEEPALIVE void   REG NA $2c5
							GC regs: 100003 {x0 x1 x20} => 100001 {x0 x20}
Generating: N1111 (  3,  2) [001471] -c----------      t1471 =    LCL_VAR   ref    V111 tmp83       u:2 NA (last use) REG NA $1ca
                                                              /--*  t1471  ref    
Generating: N1113 (  4,  3) [001472] ----G-------              *  KEEPALIVE void   REG NA $2c6
							Live vars: {V00 V01 V02 V03 V04 V06 V08 V11 V12 V14 V111 V119 V153 V155 V156 V157 V160 V170 V217} => {V00 V01 V02 V03 V04 V06 V08 V11 V12 V14 V119 V153 V155 V156 V157 V160 V170 V217}
							GCvars: {V03 V06 V12 V111 V155 V156} => {V03 V06 V12 V155 V156}
Generating: N1115 (  3,  2) [001543] ------------      t1543 =    LCL_VAR   ref    V119 tmp91       u:1 x0 (last use) REG x0 $5f6
                                                              /--*  t1543  ref    
Generating: N1117 (  3,  3) [000875] DA--G-------              *  STORE_LCL_VAR ref    V13 loc7         d:5 x1 REG x1
							V119 in reg x0 is becoming dead  [001543]
							Live regs: 3F00001 {x0 x20 x21 x22 x23 x24 x25} => 3F00000 {x20 x21 x22 x23 x24 x25}
							Live vars: {V00 V01 V02 V03 V04 V06 V08 V11 V12 V14 V119 V153 V155 V156 V157 V160 V170 V217} => {V00 V01 V02 V03 V04 V06 V08 V11 V12 V14 V153 V155 V156 V157 V160 V170 V217}
							GC regs: 100001 {x0 x20} => 100000 {x20}

 -- suppressing mov because previous instruction already moved from src to dst register.

wondering why arm64's emitter::IsRedundantMov does not check isValidLastInsFormats for same src/same dest movs.

@AndyAyersMS
Copy link
Member

This seems to fix things, as best I can tell:

diff --git a/src/coreclr/jit/emitarm64.cpp b/src/coreclr/jit/emitarm64.cpp
index 771ce88ec84..11263bd361b 100644
--- a/src/coreclr/jit/emitarm64.cpp
+++ b/src/coreclr/jit/emitarm64.cpp
@@ -15581,17 +15581,19 @@ bool emitter::IsRedundantMov(instruction ins, emitAttr size, regNumber dst, regN
         regNumber prevSrc    = emitLastIns->idReg2();
         insFormat lastInsfmt = emitLastIns->idInsFmt();
 
-        if ((prevDst == dst) && (prevSrc == src))
+
+
+        // Sometimes emitLastIns can be a mov with single register e.g. "mov reg, #imm". So ensure to
+        // optimize formats that does vector-to-vector or scalar-to-scalar register movs.
+        bool isValidLastInsFormats = ((lastInsfmt == IF_DV_3C) || (lastInsfmt == IF_DR_2G) || (lastInsfmt == IF_DR_2E));
+
+        if (isValidLastInsFormats && (prevDst == dst) && (prevSrc == src))
         {
             assert(emitLastIns->idOpSize() == size);
             JITDUMP("\n -- suppressing mov because previous instruction already moved from src to dst register.\n");
             return true;
         }
 
-        // Sometimes emitLastIns can be a mov with single register e.g. "mov reg, #imm". So ensure to
-        // optimize formats that does vector-to-vector or scalar-to-scalar register movs.
-        bool isValidLastInsFormats = ((lastInsfmt == IF_DV_3C) || (lastInsfmt == IF_DR_2G) || (lastInsfmt == IF_DR_2E));
-
         if ((prevDst == src) && (prevSrc == dst) && isValidLastInsFormats)
         {
             // For mov with EA_8BYTE, ensure src/dst are both scalar or both vector.

Issue was seemingly there from the start, eg #38179.

AndyAyersMS added a commit to AndyAyersMS/runtime that referenced this issue Aug 7, 2021
Otherwise we may mistake an immediate for a register and do an incorrect
optimization.

Fixes dotnet#56689.
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Aug 7, 2021
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Aug 8, 2021
AndyAyersMS added a commit that referenced this issue Aug 8, 2021
)

Otherwise we may mistake an immediate for a register and do an incorrect
optimization.

Fixes #56689.
@ghost ghost locked as resolved and limited conversation to collaborators Sep 7, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI test-run-core Test failures in .NET Core test runs
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants