Refactor Process.StartCore to accept SafeFileHandle parameters for stdin/stdout/stderr#125766
Refactor Process.StartCore to accept SafeFileHandle parameters for stdin/stdout/stderr#125766
Conversation
…ation and stream setup to Process.cs Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
…nterop.Pipe.cs to csproj Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
|
Tagging subscribers to this area: @dotnet/area-system-diagnostics-process |
adamsitnik
left a comment
There was a problem hiding this comment.
@copilot address my feedback
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Windows.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Windows.cs
Outdated
Show resolved
Hide resolved
src/libraries/System.Diagnostics.Process/src/System.Diagnostics.Process.csproj
Outdated
Show resolved
Hide resolved
…eAnonymousPipe directly, simplify Windows StartWithCreateProcess Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
All feedback addressed in commit 07d9369:
|
|
@copilot please update the PR description based on my commits and then invoke the code-review skill and post your analysis/results as a comment on this PR |
PR description updated. Code review follows below. 🤖 Copilot Code ReviewHolistic AssessmentMotivation: This refactoring lifts pipe creation out of platform-specific code into the shared Approach: The approach is sound — pipes are created uniformly via Summary: ✅ LGTM. The refactoring is correct, handles resource management properly, and the locking scheme is well-designed. Two models (Gemini 2.5 Pro and Claude Opus 4.5) reviewed this independently; their findings converge. No blocking issues found. Detailed Findings✅ Resource management — Console handles are safe to disposeThe
Disposing a non-owning ✅ Handle ownership transfer in Unix OpenStream — No double-disposeThe Unix SafePipeHandle safePipeHandle = new(handle.DangerousGetHandle(), ownsHandle: true);
handle.SetHandleAsInvalid();
handle.Dispose();The parent pipe handles ( ✅ Platform-conditional handle creation — Assertions are correctThe condition
✅ Native dup2 now unconditional — SafeThe native code now always calls ✅ Locking scheme — Correctly prevents handle inheritance leaks
The lock is shared across platforms via 💡 Minor — Comment could be more preciseThe comment at line 1372 ("We MUST close the parent copies of the child handles") is slightly misleading for the non-redirect case where handles came from Models contributing to this review: Claude Sonnet 4 (primary), Gemini 2.5 Pro, Claude Opus 4.5 |
- STARTF_USESTDHANDLES is always provided now - we may use more flags in the future - so ProcessWindowStyle tests should check only if STARTF_USESHOWWINDOW (1) was applied
adamsitnik
left a comment
There was a problem hiding this comment.
The PR LGTM and is ready for review.
@tmds The most recent conclusion from API Review is that we don't want a new option bag at all (#124334 (comment)).
This PR is just a refactor towards making it possible to start a process using ProcessStartInfo and any SafeFileHandle as standard handle.
It will allow me to implement #123959 as set of Process methods (yes, no ChildProcess) without introducing a new public API. It's not like I don't want to introduce such APIs (I still want to introduce SafeChildProcessHandle), but I can simply do it later, after providing the high-level APIs.
There was a problem hiding this comment.
Pull request overview
This PR refactors System.Diagnostics.Process startup so that platform StartCore implementations accept SafeFileHandle instances for stdin/stdout/stderr, with pipe creation moved into the shared Process.Start() path. It also updates the native/interop layers to accept provided fds instead of creating pipes in native code, enabling future scenarios where callers supply pre-existing standard stream handles.
Changes:
- Managed: Move stdio pipe creation + stream setup into
Process.Start(), and update Unix/WindowsStartCoreto acceptSafeFileHandleparameters. - Native/Interop: Change
SystemNative_ForkAndExecProcessto takestdinFd/stdoutFd/stderrFdas inputs and add a native probe forpipe2support (SystemNative_IsAtomicNonInheritablePipeCreationSupported). - Tests/Project: Update Windows window-style test expectations and add new Unix interop sources +
NullableBoolinclusion in the Process project.
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/native/libs/System.Native/pal_process_wasi.c | Updates WASI ForkAndExecProcess signature to accept stdin/out/err fds as inputs. |
| src/native/libs/System.Native/pal_process.h | Adjusts native API contract to take stdio fds by value (no redirect booleans / out fds). |
| src/native/libs/System.Native/pal_process.c | Removes native pipe creation/locking for stdio and unconditionally dup2s provided fds. |
| src/native/libs/System.Native/pal_io.h | Declares new native capability probe for atomic non-inheritable pipe creation. |
| src/native/libs/System.Native/pal_io.c | Implements the pipe2 capability probe. |
| src/native/libs/System.Native/entrypoints.c | Exposes the new native probe via System.Native entrypoints. |
| src/libraries/System.Diagnostics.Process/tests/ProcessStartInfoTests.Windows.cs | Fixes window-style assertions to mask dwFlags rather than compare full value. |
| src/libraries/System.Diagnostics.Process/src/System/Diagnostics/ProcessUtils.cs | Centralizes the cross-platform process-start ReaderWriterLockSlim. |
| src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.cs | Creates stdio handles/pipes in shared Start() and initializes redirected streams via platform OpenStream. |
| src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Windows.cs | Refactors CreateProcess path to take provided handles and duplicates them as inheritable. |
| src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Win32.cs | Updates StartCore signature, adds atomic pipe creation support flag for Windows, and moves redirect validation to shared code. |
| src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs | Updates StartCore + ForkAndExec interop usage to use provided handles; adjusts terminal detection to use IsATty(handle). |
| src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.ConfigureTerminalForChildProcesses.Unix.cs | Switches to ProcessUtils.s_processStartLock. |
| src/libraries/System.Diagnostics.Process/src/System.Diagnostics.Process.csproj | Adds new Unix interop sources and moves NullableBool to the broader non-Windows group. |
| src/libraries/Common/src/Interop/Unix/System.Native/Interop.IsAtomicNonInheritablePipeCreationSupported.cs | Adds cached managed probe for pipe2 support. |
| src/libraries/Common/src/Interop/Unix/System.Native/Interop.IsATty.cs | Adds SafeFileHandle overload for IsATty. |
| src/libraries/Common/src/Interop/Unix/System.Native/Interop.ForkAndExecProcess.cs | Updates P/Invoke to pass stdio handles (no redirect booleans / out fds). |
You can also share your feedback on Copilot code review. Take the survey.
| { | ||
| childInputPipeHandle = Console.OpenStandardInputHandle(); | ||
| } | ||
|
|
||
| if (startInfo.RedirectStandardOutput) | ||
| { | ||
| SafeFileHandle.CreateAnonymousPipe(out parentOutputPipeHandle, out childOutputPipeHandle, asyncRead: OperatingSystem.IsWindows()); | ||
| } | ||
| else | ||
| { | ||
| childOutputPipeHandle = Console.OpenStandardOutputHandle(); | ||
| } | ||
|
|
||
| if (startInfo.RedirectStandardError) | ||
| { | ||
| SafeFileHandle.CreateAnonymousPipe(out parentErrorPipeHandle, out childErrorPipeHandle, asyncRead: OperatingSystem.IsWindows()); | ||
| } | ||
| else | ||
| { | ||
| childErrorPipeHandle = Console.OpenStandardErrorHandle(); | ||
| } |
There was a problem hiding this comment.
@jkotas The alternative would be to make Console.OpenStandard*Handle not throw and just return 0/1/2. Or use File.OpenNullHandle for Android (and iOS?). What do you think?
There was a problem hiding this comment.
I do not know what works on mobile wrt. Process.Start today. I agree with copilot that we should avoid regressing it.
There was a problem hiding this comment.
@lateralusX Anything to validate here specifically on mobile platforms?
src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Windows.cs
Outdated
Show resolved
Hide resolved
| short expectedWindowFlag = short.Parse(argSplit[1]); | ||
|
|
||
| Assert.Equal(expectedDwFlag, si.dwFlags); | ||
| Assert.Equal(expectUsesShowWindow, (si.dwFlags & 0x1) != 0); // STARTF_USESHOWWINDOW is 0x1 |
There was a problem hiding this comment.
To other reviewers: This test was checking exact flag value. Now we just check if the flag is applied, allowing other flags like STARTF_USESTDHANDLES
| c_static_assert(PAL_PRIO_USER == (int)PRIO_USER); | ||
|
|
||
| #if !HAVE_PIPE2 | ||
| static pthread_mutex_t ProcessCreateLock = PTHREAD_MUTEX_INITIALIZER; |
There was a problem hiding this comment.
To other reviewers: now this lock is taken in managed code, only for the time of creating the pipes.
| bool usesTerminal = !(startInfo.RedirectStandardInput && | ||
| startInfo.RedirectStandardOutput && | ||
| startInfo.RedirectStandardError); | ||
| bool usesTerminal = Interop.Sys.IsATty(stdinHandle) || Interop.Sys.IsATty(stdoutHandle) || Interop.Sys.IsATty(stderrHandle); |
There was a problem hiding this comment.
To other reviewers: each of these handles could point to a terminal. So we perform a detailed check.
| } | ||
| else | ||
| { | ||
| childOutputPipeHandle = Console.OpenStandardOutputHandle(); |
There was a problem hiding this comment.
This and the other logic in this PR assume that stdin/stdout/stderr are distinct handles. The earlier versions had code to deal with the situation where these handle overlap. Do we need to worry about that?
There was a problem hiding this comment.
This and the other logic in this PR assume that stdin/stdout/stderr are distinct handles. The earlier versions had code to deal with the situation where these handle overlap. Do we need to worry about that?
That version was using an allow list which prohibits duplicates. It will be needed for #13943
In case of this PR, in the worst case (for example a process started with all handles redirected to NUL), we are going to perform 2 extra handle duplicates (it will be neglible when compared to the cost of starting a new process)
There was a problem hiding this comment.
It is the problem that Tom commented on. It is not clear what exact API shapes is this refactoring expected to support.
| { | ||
| childInputPipeHandle = Console.OpenStandardInputHandle(); | ||
| } | ||
|
|
||
| if (startInfo.RedirectStandardOutput) | ||
| { | ||
| SafeFileHandle.CreateAnonymousPipe(out parentOutputPipeHandle, out childOutputPipeHandle, asyncRead: OperatingSystem.IsWindows()); | ||
| } | ||
| else | ||
| { | ||
| childOutputPipeHandle = Console.OpenStandardOutputHandle(); | ||
| } | ||
|
|
||
| if (startInfo.RedirectStandardError) | ||
| { | ||
| SafeFileHandle.CreateAnonymousPipe(out parentErrorPipeHandle, out childErrorPipeHandle, asyncRead: OperatingSystem.IsWindows()); | ||
| } | ||
| else | ||
| { | ||
| childErrorPipeHandle = Console.OpenStandardErrorHandle(); | ||
| } |
There was a problem hiding this comment.
I do not know what works on mobile wrt. Process.Start today. I agree with copilot that we should avoid regressing it.
I think Deciding to no longer do
The API this is refactoring for is not worked out. It depends on what the API is whether it is useful to refactor this. |
…n Android (it's not supported)
The conclusion was that having more than one option bag would cause too much confusion (which one to use? is the old one deprecated? etc). So even if we come up with a better
We have at least two options:
If everything goes right, I am going to propose this API next Tuesday. With this refactor, exposing it publicly will be trivial. Another simple refactor is going to make it possible to move the Moreover, this refactor is going to make #13943 easier (and this API was already approved). @tmds please let me know if you need any more details. Thanks! |
| { | ||
| SafeFileHandle.CreateAnonymousPipe(out childInputPipeHandle, out parentInputPipeHandle); | ||
| } | ||
| else if (OperatingSystem.IsAndroid()) |
There was a problem hiding this comment.
This looks odd. Should we make OpenStandardInputHandle work on Android instead of this?
There was a problem hiding this comment.
@copilot address this feedback, make Console.OpenStandard*Handle APIs work on android by returning 0/1/2 values. Update the implementation, ref and tests. And then remove the Android checks from this PR
There was a problem hiding this comment.
Done in bba0144. Changed ConsolePal.Android.cs to return new SafeFileHandle(0/1/2, ownsHandle: false) (matching Unix/Wasi pattern), removed [UnsupportedOSPlatform("android")] from all three methods in both Console.cs and the ref assembly, updated tests, and removed the Android workarounds from Process.cs. All builds pass (ref, src including Android target, and Process).
I am not sure this API shape is going to fly well - it is inconsistent with what's there today. I think the SafeFileHandles should be members on ProcessStartInfo and be mutually exclusive with the other ways to redirect input/output. |
… 0/1/2; remove Android checks from Process.cs Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
| public static Stream OpenStandardError() => new LogcatStream(OutputEncoding); | ||
|
|
||
| public static SafeFileHandle OpenStandardInputHandle() => throw new PlatformNotSupportedException(); | ||
| public static SafeFileHandle OpenStandardInputHandle() => new SafeFileHandle(0, ownsHandle: false); |
There was a problem hiding this comment.
Hmm, this is inconsistent with what OpenStandardInput/OpenStandardOutput/OpenStandardError does now.
Can we just leave the standard handles alone when there is nothing to redirect?
|
We should start from the use-cases/issues, propose the API and design (which may involve some prototyping/derisking) and then the actual implementation follows. You may have a picture about the API and design. I don't think it is written down so others can understand what way we're heading and how it solves the user issues. I can technically review if this refactoring isn't breaking anything, but currently I can't say why we need it, which is the more important thing. Personally, I prefer we capture such information in an issue. PRs are already implementation centric and the coding agent workflows make it even more focused on code and code changes. |

Description
Refactors
StartCoreon both Windows and Unix to acceptSafeFileHandleinstances for child process standard streams, enabling future scenarios where callers provide pre-existing handles. Pipe creation and stream setup are lifted into the sharedProcess.cs:Start()method, with platform-specificOpenStreamand encoding helper methods defined directly in each platform file (no partial methods).Native layer (
pal_process.c,pal_process.h,pal_process_wasi.c)stdinFd/stdoutFd/stderrFdchanged fromint32_t*(out) toint32_t(in) — caller now owns pipe creationSystemNative_Pipecalls) and associatedProcessCreateLockdup2of the provided fds ontoSTDIN_FILENO/STDOUT_FILENO/STDERR_FILENOis now unconditional (CLOEXEC flags are stripped bydup2itself since it doesn't copy descriptor flags)redirectStdin/redirectStdout/redirectStderrparameters from native signatureNew native entry point (
pal_io.c,pal_io.h)SystemNative_IsAtomicNonInheritablePipeCreationSupported— returns 1 whenHAVE_PIPE2is defined, 0 otherwise; used by managed code to decide whether pipe creation requires a write lockInterop layer (
Interop.ForkAndExecProcess.cs)out int stdinFd/stdoutFd/stderrFdtoSafeFileHandleparameters (marshalled asint32_tvalues)redirectStdin/redirectStdout/redirectStderrboolean parametersNew interop declarations
Interop.IsATty.cs: AddedSafeFileHandleoverload ofIsATtyInterop.IsAtomicNonInheritablePipeCreationSupported.cs: New P/Invoke withNullableBoolcaching forSystemNative_IsAtomicNonInheritablePipeCreationSupportedManaged layer
Process.cs:Start()now creates pipes viaSafeFileHandle.CreateAnonymousPipedirectly (withConsole.OpenStandard*Handle()fallbacks for non-redirected streams). Pipe creation is guarded by!startInfo.UseShellExecute || !OperatingSystem.IsWindows(). On platforms withoutpipe2, aWriteLockwraps pipe creation to prevent accidental handle inheritance. AfterStartCore, sets up_standardInput/_standardOutput/_standardErrorstreams via platform-specificOpenStream. TheCantRedirectStreamsvalidation forUseShellExecuteis moved here from platform files.Process.Unix.cs:StartCoretakesSafeFileHandle?parameters (asserted non-null). Terminal detection (usesTerminal) changed from checking redirect booleans toIsATty()calls on the actual handles.OpenStreamtransfers ownership fromSafeFileHandletoSafePipeHandleand wraps inAnonymousPipeClientStream.GetStandardInputEncoding/GetStandardOutputEncodingreturnEncoding.Default.s_processStartLockmoved to sharedProcessUtils.Process.Windows.cs:StartWithCreateProcesstakes non-nullableSafeFileHandleparameters. Handles are duplicated as inheritable viaDuplicateAsInheritable. The oldCreatePipehelper ands_createProcessLockare removed. The lock is replaced byProcessUtils.s_processStartLock.EnterWriteLock().OpenStreamwraps handle inFileStream.GetStandardInputEncoding/GetStandardOutputEncodinguse console code pages.Process.Win32.cs:StartCoresignature updated; asserts thatUseShellExecuteimplies all-null handles.CantRedirectStreamscheck removed (moved to sharedProcess.cs).ProcessUtils.cs: Now hostss_processStartLock(shared across platforms).Process.ConfigureTerminalForChildProcesses.Unix.cs: ReferencesProcessUtils.s_processStartLockinstead of the removed instance field.Console changes (Android support)
ConsolePal.Android.cs:OpenStandardInputHandle/OpenStandardOutputHandle/OpenStandardErrorHandlenow returnSafeFileHandle(0/1/2, ownsHandle: false)instead of throwingPlatformNotSupportedException, matching the existing Unix/Wasi behavior.Console.cs: Removed[UnsupportedOSPlatform("android")]from all threeOpenStandard*Handlemethods.System.Console.cs): Removed[UnsupportedOSPlatformAttribute("android")]from all three methods.ConsoleHandles.cs): Updated platform filters to include Android in valid-handle tests and exclude it from throws-on-unsupported tests.Build verification
TestGetProcesses)🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.