Skip to content
This repository has been archived by the owner on Jan 23, 2023. It is now read-only.

Revert removal of SuppressGCTransition from SystemNative_GetTimestamp() #27473

Merged
merged 6 commits into from
Oct 29, 2019

Conversation

AaronRobinsonMSFT
Copy link
Member

@AaronRobinsonMSFT AaronRobinsonMSFT commented Oct 27, 2019

Instead of relying on the existing logic in fgCreateGCPoll() that inserts a GC_POLL at the end of a BasicBlock, supply a statement contained in the block and insert a new statement with the GC_POLL after the supplied statement.

Fixes https://github.com/dotnet/coreclr/issues/27465

@AaronRobinsonMSFT
Copy link
Member Author

/cc @jkotas @dotnet/jit-contrib

@AaronRobinsonMSFT AaronRobinsonMSFT marked this pull request as ready for review October 27, 2019 21:52
Copy link

@sandreenko sandreenko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks reasonable.

How urgent is it? Do we need to merge it ASAP?

Could you please show the jitdump before and after your change? As I understand the additional statement is inserted earlier in the block now. We will need asm diffs for that because it could change later optimizations behavior.

It would be good to have your windows repro in coreclr\tests\src\JIT\Regression\JitBlue\GitHub_27465.

src/jit/flowgraph.cpp Outdated Show resolved Hide resolved
src/jit/morph.cpp Show resolved Hide resolved
src/jit/flowgraph.cpp Outdated Show resolved Hide resolved
@AaronRobinsonMSFT
Copy link
Member Author

How urgent is it? Do we need to merge it ASAP?

No rush, can wait until next week.

Could you please show the jitdump before and after your change? As I understand the additional statement is inserted earlier in the block now. We will need asm diffs for that because it could change later optimizations behavior.

I can, but note this is only updating function calls that I previously changed in in #26458. Also, what is the metric for jit-diff? I don't always see it provided with codegen changes and am having issue trying to understand when it is required and when it isn't.

It would be good to have your windows repro in coreclr\tests\src\JIT\Regression\JitBlue\GitHub_27465.

I don't agree. This was discovered via an assert and fired because of usage of bad legacy code, not a regression from a known good state. Further the example doesn't capture the actual issue which is related to an assert related to tail call validation and insertion of calls at the end of a basic block. Neither of which have anything to do with the usage of the SuppressGCTransitionAttribute. Basically this test won't demonstrate an obvious invariant merely a manifestation of the issue through legacy code, which is slated to be removed/rewritten per https://github.com/dotnet/coreclr/issues/27186#issuecomment-542386711.

@sandreenko
Copy link

I can, but note this is only updating function calls that I previously changed in in #26458. Also, what is the metric for jit-diff? I don't always see it provided with codegen changes and am having issue trying to understand when it is required and when it isn't.

Hm, that is not easy to express. It is a good habbit to have them always collected, sometimes we do, but don't mention it in simple PRs (if the diffs are zero).

In this particular case I think it will be useful to see:

  1. if any places in FX libs are using this code path;
  2. will this statements movement affect other optimizations;

I expect to see diffs but the total size should be the same (so only code movements). If we see a size regression then we will need to check

@AaronRobinsonMSFT
Copy link
Member Author

Hitting an issue with my change to inserting a new statement:

noway_assert(fgRemoveRestOfBlock);

if (lastStmt->GetRootNode()->gtOper == GT_CALL)
{
    noway_assert(fgRemoveRestOfBlock);
    ...

The code to reproduce this is below. It looks like the new statement is causing issues during condition folding - see stack below example.

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
using System.Runtime.InteropServices;
namespace System.Runtime.InteropServices
{
    public class SuppressGCTransitionAttribute : Attribute
    {
        public SuppressGCTransitionAttribute() { }
    }
}
static class NativeLib
{
    public const string Path = @"NativeLib.dll";
    [DllImport(Path, EntryPoint = "SystemNative_GetTimestamp")]
    [SuppressGCTransition]
    internal static extern ulong GetTimestamp();
}
public unsafe class Program
{
    private static long TS => (long)NativeLib.GetTimestamp();

    static void Main(string[] args)
    {
        if (RunTest(TS))
            throw new Exception();
    }
    static bool RunTest(long tt)
    {
        long ts1 = TS;
        if (ts1 == tt)
        {
            return true;
        }
        return false;
    }
}
 	KernelBase.dll!00007ffe1d710192()	Unknown
 	clrjit.dll!assertAbort(const char * why, const char * file, unsigned int line) Line 310	C++
 	clrjit.dll!noWayAssertAbortHelper(const char * cond, const char * file, unsigned int line) Line 499	C++
 	clrjit.dll!noWayAssertBody(const char * cond, const char * file, unsigned int line) Line 525	C++
 	clrjit.dll!noWayAssertBodyConditional(const char * cond, const char * file, unsigned int line) Line 510	C++
>	clrjit.dll!Compiler::fgFoldConditional(BasicBlock * block) Line 14859	C++
 	clrjit.dll!Compiler::fgMorphStmts(BasicBlock * block, bool * lnot, bool * loadw) Line 15479	C++
 	clrjit.dll!Compiler::fgMorphBlocks() Line 15620	C++
 	clrjit.dll!Compiler::fgMorph() Line 16648	C++
 	clrjit.dll!Compiler::compCompile(void * * methodCodePtr, unsigned long * methodCodeSize, JitFlags * compileFlags) Line 4597	C++
 	clrjit.dll!Compiler::compCompileHelper(CORINFO_MODULE_STRUCT_ * classPtr, ICorJitInfo * compHnd, CORINFO_METHOD_INFO * methodInfo, void * * methodCodePtr, unsigned long * methodCodeSize, JitFlags * compileFlags, CorInfoInstantiationVerification instVerInfo) Line 6128	C++
 	clrjit.dll!`Compiler::compCompile'::`50'::__Body::Run(Compiler::compCompile::__l2::__JITParam * __JITpParam) Line 5464	C++
 	clrjit.dll!Compiler::compCompile(CORINFO_METHOD_STRUCT_ * methodHnd, CORINFO_MODULE_STRUCT_ * classPtr, ICorJitInfo * compHnd, CORINFO_METHOD_INFO * methodInfo, void * * methodCodePtr, unsigned long * methodCodeSize, JitFlags * compileFlags) Line 5468	C++
 	clrjit.dll!``jitNativeCode'::`8'::__Body::Run'::`6'::__Body::Run(jitNativeCode::__l8::__Body::Run::__l5::__JITParam * __JITpParam) Line 6766	C++
 	clrjit.dll!`jitNativeCode'::`8'::__Body::Run(jitNativeCode::__l2::__JITParam * __JITpParam) Line 6770	C++
 	clrjit.dll!jitNativeCode(CORINFO_METHOD_STRUCT_ * methodHnd, CORINFO_MODULE_STRUCT_ * classPtr, ICorJitInfo * compHnd, CORINFO_METHOD_INFO * methodInfo, void * * methodCodePtr, unsigned long * methodCodeSize, JitFlags * compileFlags, void * inlineInfoPtr) Line 6794	C++

@jkotas
Copy link
Member

jkotas commented Oct 28, 2019

Basically this test won't demonstrate an obvious invariant

Codegen regression tests are "interesting sequences" that happened to trigger a bug. They rarely target specific obvious invariants.

@AaronRobinsonMSFT
Copy link
Member Author

It would be good to have your windows repro in >>coreclr\tests\src\JIT\Regression\JitBlue\GitHub_27465.

I don't agree. This was discovered via an assert and fired because of usage of bad legacy code, not a regression from a known good state.

@sandreenko I am rethinking my statement here. I am discovering a lot of nuance with this work and am leaning more toward it being a good idea. Instead of putting this under the regression folder is there a better place for targeted scenario tests? I am thinking of one related to insertion of GC_POLLs. I have a few scenarios that we should validate.

@AaronRobinsonMSFT
Copy link
Member Author

@jkotas Agreed. This feature is becoming far more complex and I am writing a few examples that I have discovered that will need to be handled by any work improving the insert GC_POLL logic.

@jkotas
Copy link
Member

jkotas commented Oct 28, 2019

Insert GC_POLL before statement with unmanaged call.

The GC_POLL is more effective after the the unmanaged call. (It covers more time where the GC could not have been suspended after the call.) It would be nice to keep it inserting after the call.

@AaronRobinsonMSFT
Copy link
Member Author

AaronRobinsonMSFT commented Oct 28, 2019

The GC_POLL is more effective after the the unmanaged call.

@jkotas I don't understand why that would be the case. In tight loops regardless of which side of the call it will be hit and if unmanaged calls are stacked then we cascade the polling. If we really want this after the call in a robust manner, then I am going to have to defer this to the JIT team since there are significant issues in this area that I don't quite know how to address.

To unpack this a bit. It is going to require reordering statements and creating additional basic blocks. For example, GT_JTRUE statements at the end of a basic block. This is addressed in fgNewStmtNearEnd() but that isn't what we want since we could have multiple unmanaged calls in the block. Ideally if a basic block could only contain a single unmanaged call, this would be significantly easier. However, I assume that would be much more work.

@sandreenko
Copy link

@sandreenko I am rethinking my statement here. I am discovering a lot of nuance with this work and am leaning more toward it being a good idea. Instead of putting this under the regression folder is there a better place for targeted scenario tests? I am thinking of one related to insertion of GC_POLLs. I have a few scenarios that we should validate.

I think https://github.com/dotnet/coreclr/tree/master/tests/src/JIT/Methodical is the best match for it.

I am going to have to defer this to the JIT team since there are significant issues in this area that I don't quite know how to address.

I can take a look at the assert in morph, but only tomorrow.

@AaronRobinsonMSFT
Copy link
Member Author

I believe the solution here is to move GC_POLL insertion for P/Invokes into the JIT Rationalization phase. At the current phase the issue is updating the statement expression tree to insert a GC_POLL immediately after an unmanaged call. I don't believe this makes sense nor is really possible because we don't want the GC_POLL position optimized in any way, we want it at a specific location. From what I am reading the Rationalization phase is about converting from HIR to LIR (i.e. linear). If this is indeed the linear portion of the JIT I should be able to insert the appropriate call where it is desired without fear of impact to any other "optimization" phases which is what seems to be causing most of the current issues.

Welcome to other suggestions if the above doesn't make sense.

@AaronRobinsonMSFT
Copy link
Member Author

After speaking with various stakeholders, I am going to keep the changes I have and provide jit-diff and a new test for the GC_POLL scenarios.

@AaronRobinsonMSFT
Copy link
Member Author

@dotnet/jit-contrib jit-diff results are below.

$ jit-diff diff --pmi --base [ROOT]\master\Product\Windows_NT.x64.Checked --diff [ROOT]\Product\Windows_NT.x64.Checked --core_root [ROOT]\master\tests\Windows_NT.x64.Release\Tests\Core_Root --frameworks
Using --output [ROOT]\bin\diffs
Beginning PMI Diffs for System.Private.CoreLib.dll, framework assemblies
/ Finished 129/129 Base 129/129 Diff [259.3 sec]
Completed PMI Diffs for System.Private.CoreLib.dll, framework assemblies in 259.36s
Diffs (if any) can be viewed by comparing: [ROOT]\diffs\dasmset_1\base [ROOT]\diffs\dasmset_1\diff
Analyzing diffs...
Found 9 files with textual diffs.
PMI Diffs for System.Private.CoreLib.dll, framework assemblies for  default jit
Summary:
(Lower is better)
Total bytes of diff: -64 (-0.00% of base)
    diff is an improvement.
Top file improvements by size (bytes):
         -48 : System.Console.dasm (-0.09% of base)
          -5 : System.Private.CoreLib.dasm (-0.00% of base)
          -4 : Microsoft.Diagnostics.Tracing.TraceEvent.dasm (-0.00% of base)
          -4 : System.IO.FileSystem.dasm (-0.00% of base)
          -3 : System.Net.HttpListener.dasm (-0.00% of base)
5 total files with size differences (5 improved, 0 regressed), 124 unchanged.
Top method regressions by size (bytes):
          16 ( 2.31% of base) : System.Console.dasm - ConsolePal:GetStandardFile(int,int,bool):ref (4 methods)
           5 ( 0.92% of base) : System.Private.CoreLib.dasm - ILStubClass:IL_STUB_PInvoke(long):int (4 methods)
Top method improvements by size (bytes):
         -41 (-1.89% of base) : System.Console.dasm - ConsolePal:MoveBufferArea(int,int,int,int,int,int,ushort,int,int)
         -11 (-2.72% of base) : System.Console.dasm - ConsolePal:SetCursorPosition(int,int)
          -4 (-2.61% of base) : Microsoft.Diagnostics.Tracing.TraceEvent.dasm - QPCTime:.ctor():this
          -4 (-0.85% of base) : System.Console.dasm - ConsolePal:get_KeyAvailable():bool
          -4 (-2.30% of base) : System.IO.FileSystem.dasm - ILStubClass:IL_STUB_PInvoke(int,byref):bool (2 methods)
Top method regressions by size (percentage):
          16 ( 2.31% of base) : System.Console.dasm - ConsolePal:GetStandardFile(int,int,bool):ref (4 methods)
           5 ( 0.92% of base) : System.Private.CoreLib.dasm - ILStubClass:IL_STUB_PInvoke(long):int (4 methods)
Top method improvements by size (percentage):
         -11 (-2.72% of base) : System.Console.dasm - ConsolePal:SetCursorPosition(int,int)
          -4 (-2.61% of base) : Microsoft.Diagnostics.Tracing.TraceEvent.dasm - QPCTime:.ctor():this
          -2 (-2.53% of base) : System.Private.CoreLib.dasm - Stopwatch:Stop():this
          -4 (-2.30% of base) : System.IO.FileSystem.dasm - ILStubClass:IL_STUB_PInvoke(int,byref):bool (2 methods)
          -2 (-2.30% of base) : System.Private.CoreLib.dasm - ILStubClass:IL_STUB_PInvoke(int,byref):bool
18 total methods with size differences (16 improved, 2 regressed), 203690 unchanged.
4 files had text diffs but not size diffs.
Microsoft.DotNet.Cli.Utils.dasm had 24 diffs
xunit.execution.dotnet.dasm had 24 diffs
xunit.performance.execution.dasm had 10 diffs
System.Diagnostics.TraceSource.dasm had 4 diffs
Completed analysis in 20.69s

@CarolEidt
Copy link

This change seems reasonable to me, but wrt the comment above about possibly moving the insertion to the Rationalizer phase, note that it is a phase that modifies the IR in ways that we may expect to migrate earlier in the JIT phase ordering (see https://github.com/dotnet/coreclr/blob/master/Documentation/botr/ryujit-overview.md#rationalization) so it wouldn't be a good place for phase-order-specific transformations. Lowering might be a more logical place for something that you expect to happen "just before" the backend.

@AaronRobinsonMSFT
Copy link
Member Author

@CarolEidt Thanks for the additional context. I am going to defer that potential option to the future, possibly with work done to address https://github.com/dotnet/coreclr/issues/27186. Speaking to @briansull, he was also unconvinced it should be done in the Rationalizer phase.

@AaronRobinsonMSFT
Copy link
Member Author

@CarolEidt @sandreenko @jkotas Any additional feedback?

Copy link

@CarolEidt CarolEidt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link

@sandreenko sandreenko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the tests.

asm diffs are from different register allocations:
prolog before:

G_M40076_IG01:        ; func=00, offs=000000H, size=000DH, bbWeight=1   , gcrefRegs=00000000 {}, byrefRegs=00000000 {}, byref, nogc <-- Prolog IG
IN0012: 000000 push     rbp
IN0013: 000001 push     rdi
IN0014: 000002 push     rsi
IN0015: 000003 push     rbx <- used to save rdi GC pointer across the call to CORINFO_HELP_POLL_GC
IN0016: 000004 sub      rsp, 40
IN0017: 000008 lea      rbp, [rsp+40H]

prolog after (`rdi comes alive after the call now, so don't need to save it):

IN0011: 000000 push     rbp
IN0012: 000001 push     rdi
IN0013: 000002 push     rsi
IN0014: 000003 sub      rsp, 48

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Assertion failed 'retExpr->gtOper == GT_RETURN' caused by SuppressGCTransition
4 participants