Avoid throwing exceptions in Process.KillTree during process enumeration

[Copilot]

Description

Process.KillTree throws and catches Win32Exception for every system process during enumeration (when OpenProcess fails with ERROR_ACCESS_DENIED). With debugger attached, each exception triggers debugger notification, causing ~135x slowdown (23s vs 0.17s).

Changes

Modified ProcessManager.OpenProcess to return invalid handle instead of throwing when:

• throwIfExited is false (during enumeration)
• Error is ERROR_ACCESS_DENIED (expected for system processes)

// Before: Always threw Win32Exception for access denied
if (result != Interop.Errors.ERROR_ACCESS_DENIED && !IsProcessRunning(processId))
{
    // ... handle exited process
}
throw new Win32Exception(result); // ← Threw for access denied

// After: Return invalid handle for access denied when throwIfExited=false
if (!throwIfExited && result == Interop.Errors.ERROR_ACCESS_DENIED)
{
    return SafeProcessHandle.InvalidHandle;
}
throw new Win32Exception(result);

Defensive try-catch in SafeGetHandle remains for other error conditions.

Added test Kill_EntireProcessTree_MinimalExceptions in ProcessTests.Windows.cs that:

• Creates a process tree similar to the repro code from the issue
• Uses AppDomain.CurrentDomain.FirstChanceException to count exceptions thrown during Kill(true)
• Validates that no more than 5 exceptions are thrown during the operation

Testing

• ✅ All 18 existing Kill-related tests pass
• ✅ New test validates minimal exception behavior during process tree termination

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Process.KillTree throws a lot of exceptions internally</issue_title> <issue_description>I'm seeing a situation in microsoft/testfx repo where Process.KillTree` appears to take so long.

In the above case, the process itself has already exited before Kill(true) was called. Based on the code comment here:

runtime/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Win32.cs

Lines 378 to 380 in e8812e7

	// If the process has exited, the handle is invalid.
	if (handle.IsInvalid)
	return null;

I'm really surprised this condition isn't hit and we return immediately.
I guess it's not a supported scenario to kill whole tree if the parent process already exited?

Note: I haven't tried to create a minimal repro yet.</issue_description>

<agent_instructions>Avoid throwing and catching exceptions internally inside Process.KillTree implementation on Windows Focus on exceptions thrown and caught during process enumeration where the bulk of the exceptions is thrown and caught today in the typical case. It is ok to have thrown and catch exceptions in rare cases to keep the code maintainable.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@Youssef1313 Note that even if the process itself hasn't exited, something is still going on.

And seems somehow related to whether or not debugger is attached?

Run the following with debugger attached in VS:

using System.Diagnostics;

if (args.Length == 1 && args[0] == "1")
{
    var processStartInfo = new ProcessStartInfo(Process.GetCurrentProcess().MainModule!.FileName)
    {
        Arguments = "2",
        UseShellExecute = false,
        RedirectStandardOutput = true,
        RedirectStandardError = true,
    };

    Process.Start(processStartInfo);

    while (true)
    {
        Thread.Sleep(10000);
    }
}
else if (args.Length == 1 && args[0] == "2")
{
    while (true)
    {
        Thread.Sleep(10000);
    }
}

var processStartInfo1 = new ProcessStartInfo(Process.GetCurrentProcess().MainModule!.FileName)
{
    Arguments = "1",
    UseShellExecute = false,
    RedirectStandardOutput = true,
    RedirectStandardError = true,
};

var process = Process.Start(processStartInfo1);
Thread.Sleep(1000);
var timestamp = Stopwatch.GetTimestamp();
process.Kill(true);
Console.WriteLine(Stopwatch.GetElapsedTime(timestamp));

prints ~ 00:00:23.6890791

If I run without debugger, I get ~ 00:00:00.1698470.

23seconds seems like A LOT.</comment_new>
<comment_new>@Youssef1313
Ping @dotnet/area-system-diagnostics-process. I hit this from time to time.</comment_new>
<comment_new>@jkotas

Run the following with debugger attached in VS

I doubt there is much we can do about this in the .NET libraries. It is getting stuck in the OS.</comment_new>
<comment_new>@Youssef1313
@jkotas Why is it very different when the debugger is attached though?</comment_new>
<comment_new>@jkotas
My guess is that Windows has to coordinate the killing of the process between the debuggee and the debugger. The process cannot just disappear. There may be ways in VS debugger to make it better.</comment_new>
<comment_new>@Youssef1313

My guess is that Windows has to coordinate the killing of the process between the debuggee and the debugger. The process cannot just disappear. There may be ways in VS debugger to make it better.

To the extent that it takes 23 seconds when debugger is attached and less than 0.2 seconds when it's not attached? 23 seconds feels to me extremely long and unreasonable.

This is currently slowing me down a lot when debugging our integration tests in microsoft/testfx repo.</comment_new>
<comment_new>@jkotas
Changing area to diagnostics-coreclr. It is a debugger issue, not a System.Diagnostics.Process issue.</comment_new>
<comment_new>@hoyosjs
I can't seem to repro this - I consistently get sub-second times debugging or not. </comment_new>
<comment_new>@Youssef1313
In case it matters, I'm running on Windows and from Visual Studio.

csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>

</Project>

C#:

using System.Diagnostics;

if (args.Length == 1 && args[0] == "1")
{
    var processStartInfo = new ProcessStartInfo(Process.GetCurrentProcess().MainModule!.FileName)
    {
        Arguments = "2",
        UseShellExecute = false,
        RedirectStandardOutput = true,
      ...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

- Fixes dotnet/runtime#121279

<!-- START COPILOT CODING AGENT TIPS -->
---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey).

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-diagnostics-process
See info in area-owners.md if you want to be subscribed.

[jkotas]

@copilot Refactor the repro at #121279 (comment) into a test. Add code that checks number of exceptions thrown and caught internally inside Kill(true). You can use AppDomain.CurrentDomain.FirstChanceException handler that increments a counter for that. The test should validate that there are no more than 5 exceptions thrown and caught.

[Copilot]

@copilot Refactor the repro at #121279 (comment) into a test. Add code that checks number of exceptions thrown and caught internally inside `Kill(...

Added test Kill_EntireProcessTree_MinimalExceptions based on the repro code. The test creates a process tree, uses AppDomain.CurrentDomain.FirstChanceException to count exceptions during Kill(true), and validates that no more than 5 exceptions are thrown. Commit: 5b1fbcb