Add IDisposable interface to GCHandle #55596

Sergio0694 · 2021-07-13T20:01:26Z

Closes #54792

namespace System.Runtime.InteropServices
{
      public struct GCHandle
+        : IDisposable
      {
+        public void Dispose();
      }
}

dotnet-issue-labeler · 2021-07-13T20:01:30Z

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, to please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/GCHandle.cs

src/libraries/System.Runtime/ref/System.Runtime.cs

sharwell · 2021-07-14T00:10:38Z

src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/GCHandle.cs

+        /// <inheritdoc/>
+        public void Dispose()
+        {
+            IntPtr handle = _handle;


❔ Can we not Interlocked.Exchange here?

Suggested change

IntPtr handle = _handle;

IntPtr handle = Interlocked.Exchange(ref _handle, IntPtr.Zero);

#54792 (comment)

stephentoub · 2021-07-14T19:04:08Z

The test failures here are real, e.g.

Assert failure(PID 20237 [0x00004f0d], Thread: 20252 [0x4f1c]): *(_UNCHECKED_OBJECTREF *)handle == NULL
    File: /__w/1/s/src/coreclr/gc/handletable.cpp Line: 298
    Image: /datadisks/disk1/work/B65009B3/p/dotnet

Sergio0694 · 2021-07-14T19:49:45Z

That's unexpected 🤔
I've double checked all the changes here twice (and also asked @tannergooding to have a look for good measure) and it doesn't seem like any of the changes here are invalid on their own or should possibly break anything, unless some of the code was (perhaps accidentally) relying on Free being thread-safe, and therefore breaking now? Thoughts on this? Also if this actually turned out to be the case, what would you recommend doing here - just reverting that decision and making both Free and Dispose thread safe, or possibly fix that code relying on this (which would be much trickier to do though I'd expect)?

EDIT: pushed a test as per Tanner's suggestion, let's see if that was actually it 👀

jkotas · 2021-07-14T20:05:04Z

If it is the case, it would be useful to find the code that depends on the thread safety guarantees of Free.

stephentoub · 2021-07-14T20:07:10Z

I suspect it's not a thread-safety thing; if it were, I'd expect it to be more non-deterministic. I pulled down the PR and deterministically get a failure here on every run:

  Fatal error. Internal CLR error. (0x80131506)
     at System.Runtime.InteropServices.Marshal.GetTypedObjectForIUnknown(IntPtr, System.Type)
     at System.Runtime.InteropServices.Tests.GetTypedObjectForIUnknownTests+<>c__DisplayClass9_0.<GetTypedObjectForIUnknown_UncastableObject_ThrowsInvalidCastException>b__0()

I'll need to rebuild with a checked runtime, but presumably that internal error is related to the previously shown assert.

stephentoub · 2021-07-14T20:58:12Z

000000D29A2F93C0 00007FFFCA561A4F coreclr!HndCreateHandle + 475 at D:\repos\runtime\src\coreclr\gc\handletable.cpp:298
000000D29A2F94A0 00007FFFCA5CCBA5 coreclr!GCHandleStore::CreateHandleOfType + 69 at D:\repos\runtime\src\coreclr\gc\gchandletable.cpp:32
000000D29A2F94D0 00007FFFC9F0A79F coreclr!CreateHandleCommon + 59 at D:\repos\runtime\src\coreclr\vm\gchandleutilities.h:71
000000D29A2F9500 00007FFFC9F0A748 coreclr!BaseDomain::CreateHandle + 104 at D:\repos\runtime\src\coreclr\vm\appdomain.hpp:1027
000000D29A2F9540 00007FFFCA040176 coreclr!ThreadExceptionState::SetThrowable + 722 at D:\repos\runtime\src\coreclr\vm\exstate.cpp:160
000000D29A2F96B0 00007FFFCA187238 coreclr!Thread::SafeSetThrowables + 416 at D:\repos\runtime\src\coreclr\vm\threads.cpp:4531
000000D29A2F9860 00007FFFCA54C405 coreclr!ExceptionTracker::GetOrCreateTracker + 3645 at D:\repos\runtime\src\coreclr\vm\exceptionhandling.cpp:3852
000000D29A2F9D60 00007FFFCA55438B coreclr!ProcessCLRException + 1067 at D:\repos\runtime\src\coreclr\vm\exceptionhandling.cpp:1001
000000D29A2FA100 00007FF87CF9217F ntdll!RtlpExecuteHandlerForException + 15
000000D29A2FA130 00007FF87CF41454 ntdll!RtlDispatchException + 580
000000D29A2FA840 00007FF87CF90CAE ntdll!KiUserExceptionDispatch + 46
000000D29A2FAFD0 00007FF87A664ED9 KERNELBASE!RaiseException + 105
000000D29A2FB0B0 00007FFFCA03647F coreclr!`RaiseTheExceptionInternalOnly'::`53'::__Body::Run + 579 at D:\repos\runtime\src\coreclr\vm\excep.cpp:2806
000000D29A2FB1B0 00007FFFCA032ED8 coreclr!RaiseTheExceptionInternalOnly + 1024 at D:\repos\runtime\src\coreclr\vm\excep.cpp:2808
000000D29A2FB320 00007FFFCA03DDCF coreclr!UnwindAndContinueRethrowHelperAfterCatch + 227 at D:\repos\runtime\src\coreclr\vm\excep.cpp:8152
000000D29A2FB3F0 00007FFFCA3163C2 coreclr!MarshalNative::GetComInterfaceForObjectNative + 1138 at D:\repos\runtime\src\coreclr\vm\marshalnative.cpp:764
000000D29A2FB508                  [HelperMethodFrame_2OBJ: 000000d29a2fb508] System.Private.CoreLib.dll!System.Runtime.InteropServices.Marshal.GetComInterfaceForObjectNative(System.Object, System.Type, Boolean)
000000D29A2FB650 00007fff6ca4a157 System.Private.CoreLib.dll!System.Runtime.InteropServices.Marshal.GetComInterfaceForObject(System.Object, System.Type, System.Runtime.InteropServices.CustomQueryInterfaceMode) + 231 [D:\repos\runtime\src\coreclr\System.Private.CoreLib\src\System\Runtime\InteropServices\Marshal.CoreCLR.cs @ 382]
000000D29A2FB658 000002B277FA5570 
000000D29A2FB660 00000000000000CD 
000000D29A2FB668 000002B278028F90 
000000D29A2FB670 00007FFFC9F41A6D coreclr!Thread::BeginForbidGC + 137 at D:\repos\runtime\src\coreclr\vm\threads.h:2075
000000D29A2FB6A0 0000000100000001 
000000D29A2FB6A8 000000D29A2FB6F0 
000000D29A2FB6B0 00007fff6ca579e9 System.Runtime.InteropServices.Tests.dll!System.Runtime.InteropServices.Tests.GetComInterfaceForObjectTests+c__DisplayClass12_0.<GetComInterfaceForObject_InvalidType_ThrowsArgumentException>b__1() + 89 [D:\repos\runtime\src\libraries\System.Runtime.InteropServices\tests\System\Runtime\InteropServices\Marshal\GetComInterfaceForObjectTests.cs @ 187]
000000D29A2FB6B8 000002B2006EF640 
000000D29A2FB6C0 000002B2000AF490 
000000D29A2FB6C8 000002B200000001 
000000D29A2FB6D0 00007FFFCA287363 coreclr!ForbidGC::~ForbidGC + 91 at D:\repos\runtime\src\coreclr\vm\fcall.cpp:189
000000D29A2FB700 00007fff6c9b7fd9 xunit.assert.dll!Xunit.Assert.RecordException(System.Func`1<System.Object>) + 89 [C:\Dev\xunit\xunit\src\xunit.assert\Asserts\Record.cs @ 50]
000000D29A2FB708 000002B2006EF308 
000000D29A2FB710 000002B2006EF600 
000000D29A2FB718 000002B278028F90 
000000D29A2FB770 00007fff6c9b7e43 xunit.assert.dll!Xunit.Assert.Throws[[System.__Canon, System.Private.CoreLib]](System.Func`1<System.Object>) + 83 [C:\Dev\xunit\xunit\src\xunit.assert\Asserts\ExceptionAsserts.cs @ 41]
000000D29A2FB7D0 00007fff6c9b7f16 TestUtilities.dll!System.AssertExtensions.Throws[[System.__Canon, System.Private.CoreLib]](System.String, System.Func`1<System.Object>) + 134 [D:\repos\runtime\src\libraries\Common\tests\TestUtilities\System\AssertExtensions.cs @ 121]
000000D29A2FB840 00007fff6ca57452 System.Runtime.InteropServices.Tests.dll!System.Runtime.InteropServices.Tests.GetComInterfaceForObjectTests.GetComInterfaceForObject_InvalidType_ThrowsArgumentException(System.Type) + 258 [D:\repos\runtime\src\libraries\System.Runtime.InteropServices\tests\System\Runtime\InteropServices\Marshal\GetComInterfaceForObjectTests.cs @ 187]

stephentoub · 2021-07-14T20:58:40Z

@jkotas, does the runtime take a dependency on the method table / layout of GCHandle? If I just comment out : IDisposable, the problems go away.

jkotas · 2021-07-14T21:06:53Z

The runtime has no dependencies like that.

I think that the problem is going to be a code pattern like if (o is IDisposable d) d.Dispose();. This was no-op before this change, but starts disposing the GCHandle prematurely with this change.

stephentoub · 2021-07-14T21:29:29Z

I think that the problem is going to be a code pattern like if (o is IDisposable d) d.Dispose();

I think you're right... tracking it down...

stephentoub · 2021-07-14T21:32:52Z

Heh, the problem is related to xunit theories. The xunit code adds every item yielded from a member data to this collection:

https://github.com/xunit/xunit/blob/a8614d34999c889e1c8014f679de95d20eae1304/src/xunit.execution/Sdk/Frameworks/Runners/XunitTheoryTestCaseRunner.cs#L100

and it then disposes of those:

https://github.com/xunit/xunit/blob/a8614d34999c889e1c8014f679de95d20eae1304/src/xunit.execution/Sdk/Frameworks/Runners/XunitTheoryTestCaseRunner.cs#L151-L152

and we have tests that do things like this:

runtime/src/libraries/System.Runtime.InteropServices/tests/System/Runtime/InteropServices/GCHandleTests.cs

Lines 122 to 129 in 3ec6907

    
           public static IEnumerable<object[]> Equals_TestData() 
        
           { 
        
               GCHandle handle = GCHandle.Alloc(new object()); 
        
               yield return new object[] { handle, handle, true }; 
        
               yield return new object[] { GCHandle.Alloc(new object()), GCHandle.Alloc(new object()), false }; 
        
               yield return new object[] { GCHandle.Alloc(new object()), new object(), false }; 
        
               yield return new object[] { GCHandle.Alloc(new object()), null, false };

Makes me a little nervous about this change. Obviously we can change our tests to not do that, but I'm not sure where else this pattern might creep up and lead to silent corruption.

AraHaan · 2021-07-14T21:39:19Z

and it then disposes of those:
https://github.com/xunit/xunit/blob/a8614d34999c889e1c8014f679de95d20eae1304/src/xunit.execution/Sdk/Frameworks/Runners/XunitTheoryTestCaseRunner.cs#L151-L152

Cant the part where it disposes of them actually be patched to filter out GCHandles and dispose of them last (or not at all) as a patch to xunit itself?

GrabYourPitchforks · 2021-07-14T21:42:11Z

@AraHaan It'd be easier to remove the finally block from our test methods, relying on xunit to dispose of them instead of us performing manual disposal. Localized change, no xunit patch needed.

But this doesn't address Steve's larger issue: "I'm not sure where else this pattern might creep up and lead to silent corruption."

jkotas · 2021-07-15T16:19:36Z

Maybe we can add Dispose without implementing the IDisposable interface? It won't make it possible to use GCHandle via using pattern, but it will at least fix the IsAllocated / Free dance that a lot of places have to do today.

AraHaan · 2021-07-15T22:41:13Z

I think a better option is to create a new interface IDisposableHandle and use that for GCHandle and somehow hack the language to allow types implementing that interface to use "using"s with them.

stephentoub · 2021-07-16T12:14:53Z

This is not worth a new interface in my opinion. If it really rose to a high level of importance, we could teach the language how to pattern match Dispose, just as it already can for ref structs. I'm fine with adding Dispose here without the interface implementation. I'm waffling on whether we should add the interface implementation, anyway; the test pattern here seems very unique to a combination of test harnesses and tests only we'd write in runtime, and this xunit approach would likely have issues with any type becoming IDisposable... this one just has harder to diagnose side effects.

GSPP · 2021-07-25T06:27:20Z

xUnit auto-dispose will now mutate the boxed GCHandle instances on the heap through the IDisposable interface. That's a pretty extreme design.

AaronRobinsonMSFT · 2021-09-07T19:57:20Z

I think we might want to go back to the drawing board here and reconsider the API given the discovered semantics we've found. @jkotas 's suggestion is one option:

Maybe we can add Dispose without implementing the IDisposable interface? It won't make it possible to use GCHandle via using pattern, but it will at least fix the IsAllocated / Free dance that a lot of places have to do today.

Or perhaps simply abandon this as too many patterns have been created over the years that make this troublesome to deal with. Similar to the SafeHandle fiasco – #48193.

AaronRobinsonMSFT · 2021-11-01T15:07:54Z

@Sergio0694 Can we close this PR in lieu of the above discussion and concerns about compat? I think we need some additional thought on it.

Sergio0694 · 2021-11-01T15:16:18Z

@AaronRobinsonMSFT Sure thing! 😄

...I need to stop ever saying "this looks straightforward enough" when doing anything on the runtime 🤣

AaronRobinsonMSFT · 2021-11-01T16:56:54Z

@Sergio0694 To be fair, lots of us thought it would be too. You are in good company.

dotnet-issue-labeler bot added area-System.Runtime.InteropServices new-api-needs-documentation labels Jul 13, 2021

stephentoub reviewed Jul 13, 2021

View reviewed changes

src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/GCHandle.cs Outdated Show resolved Hide resolved

stephentoub reviewed Jul 13, 2021

View reviewed changes

src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/GCHandle.cs Outdated Show resolved Hide resolved

stephentoub reviewed Jul 13, 2021

View reviewed changes

src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/GCHandle.cs Outdated Show resolved Hide resolved

stephentoub reviewed Jul 13, 2021

View reviewed changes

src/libraries/System.Runtime/ref/System.Runtime.cs Show resolved Hide resolved

Sergio0694 requested a review from marek-safar as a code owner July 13, 2021 21:06

Sergio0694 force-pushed the feature/disposable-gchandle branch from 602b50c to 461e817 Compare July 13, 2021 21:12

GrabYourPitchforks approved these changes Jul 13, 2021

View reviewed changes

Sergio0694 force-pushed the feature/disposable-gchandle branch from 461e817 to 904a3a9 Compare July 14, 2021 00:02

sharwell reviewed Jul 14, 2021

View reviewed changes

stephentoub approved these changes Jul 14, 2021

View reviewed changes

Sergio0694 force-pushed the feature/disposable-gchandle branch from 904a3a9 to a29c100 Compare July 14, 2021 15:19

Sergio0694 added 2 commits July 16, 2021 09:28

Add IDisposable interface to GCHandle

e9a8cb8

Zero internal GC handle field before disposal

db295c4

Sergio0694 and others added 3 commits July 16, 2021 09:28

Apply code style feedbacks to GCHandle.Free()

bb5f30b

Migrate applicable patterns to GCHandle.Dispose()

5e6525e

Fix GCHandle tests to not double-dispose

cf6976d

stephentoub force-pushed the feature/disposable-gchandle branch from d457d9c to cf6976d Compare July 16, 2021 13:51

terrajobst added the community-contribution Indicates that the PR has been added by a community member label Jul 19, 2021

karelz mentioned this pull request Jul 20, 2021

Failed test SendAsync_CorrectVersionSelected_LoopbackServer #56026

Closed

karelz mentioned this pull request Jul 28, 2021

MsQuic tests hang / long running #56487

Closed

AaronRobinsonMSFT mentioned this pull request Nov 1, 2021

Add Dispose/IDisposable to GCHandle #54792

Open

Sergio0694 closed this Nov 1, 2021

ghost locked as resolved and limited conversation to collaborators Dec 1, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add IDisposable interface to GCHandle #55596

Add IDisposable interface to GCHandle #55596

Sergio0694 commented Jul 13, 2021

dotnet-issue-labeler bot commented Jul 13, 2021

sharwell Jul 14, 2021

stephentoub Jul 14, 2021

stephentoub commented Jul 14, 2021

Sergio0694 commented Jul 14, 2021 •

edited

Loading

jkotas commented Jul 14, 2021

stephentoub commented Jul 14, 2021

stephentoub commented Jul 14, 2021

stephentoub commented Jul 14, 2021 •

edited

Loading

jkotas commented Jul 14, 2021

stephentoub commented Jul 14, 2021

stephentoub commented Jul 14, 2021 •

edited

Loading

AraHaan commented Jul 14, 2021 •

edited

Loading

GrabYourPitchforks commented Jul 14, 2021

jkotas commented Jul 15, 2021

AraHaan commented Jul 15, 2021

stephentoub commented Jul 16, 2021 •

edited

Loading

GSPP commented Jul 25, 2021

AaronRobinsonMSFT commented Sep 7, 2021

AaronRobinsonMSFT commented Nov 1, 2021

Sergio0694 commented Nov 1, 2021

AaronRobinsonMSFT commented Nov 1, 2021

	IntPtr handle = _handle;
	IntPtr handle = Interlocked.Exchange(ref _handle, IntPtr.Zero);

Add IDisposable interface to GCHandle #55596

Add IDisposable interface to GCHandle #55596

Conversation

Sergio0694 commented Jul 13, 2021

Closes #54792

dotnet-issue-labeler bot commented Jul 13, 2021

sharwell Jul 14, 2021

Choose a reason for hiding this comment

stephentoub Jul 14, 2021

Choose a reason for hiding this comment

stephentoub commented Jul 14, 2021

Sergio0694 commented Jul 14, 2021 • edited Loading

jkotas commented Jul 14, 2021

stephentoub commented Jul 14, 2021

stephentoub commented Jul 14, 2021

stephentoub commented Jul 14, 2021 • edited Loading

jkotas commented Jul 14, 2021

stephentoub commented Jul 14, 2021

stephentoub commented Jul 14, 2021 • edited Loading

AraHaan commented Jul 14, 2021 • edited Loading

GrabYourPitchforks commented Jul 14, 2021

jkotas commented Jul 15, 2021

AraHaan commented Jul 15, 2021

stephentoub commented Jul 16, 2021 • edited Loading

GSPP commented Jul 25, 2021

AaronRobinsonMSFT commented Sep 7, 2021

AaronRobinsonMSFT commented Nov 1, 2021

Sergio0694 commented Nov 1, 2021

AaronRobinsonMSFT commented Nov 1, 2021

Sergio0694 commented Jul 14, 2021 •

edited

Loading

stephentoub commented Jul 14, 2021 •

edited

Loading

stephentoub commented Jul 14, 2021 •

edited

Loading

AraHaan commented Jul 14, 2021 •

edited

Loading

stephentoub commented Jul 16, 2021 •

edited

Loading