ssh/tailssh: fix double race condition with non-pty command #8405
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dsnet
force-pushed
the
dsnet/ssh-output-race
branch
from
58a3d75
to
789504d
Compare
There are two race conditions in output handling. The first race condition is due to a misuse of exec.Cmd.StdoutPipe. The documentation explicitly forbids concurrent use of StdoutPipe with exec.Cmd.Wait (see golang/go#60908) because Wait will close both sides of the pipe once the process ends without any guarantees that all data has been read from the pipe. To fix this, we allocate the os.Pipes ourselves and manage cleanup ourselves when the process has ended. The second race condition is because sshSession.run waits upon exec.Cmd to finish and then immediately proceeds to call ss.Exit, which will close all output streams going to the SSH client. This may interrupt any asynchronous io.Copy still copying data. To fix this, we close the write-side of the os.Pipes after the process has finished (and before calling ss.Exit) and synchronously wait for the io.Copy routines to finish. Fixes #7601 Signed-off-by: Joe Tsai <joetsai@digital-static.net> Co-authored-by: Maisem Ali <maisem@tailscale.com>
dsnet
force-pushed
the
dsnet/ssh-output-race
branch
from
789504d
to
1c3cd6f
Compare
maisem
approved these changes
Jun 22, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There are two race conditions in output handling.
The first race condition is due to a misuse of exec.Cmd.StdoutPipe. The documentation explicitly forbids concurrent use of StdoutPipe with exec.Cmd.Wait (see golang/go#60908) because Wait will close both sides of the pipe once the process ends without any guarantees that all data has been read from the pipe. To fix this, we allocate the os.Pipes ourselves and manage cleanup ourselves when the process has ended.
The second race condition is because sshSession.run waits upon exec.Cmd to finish and then immediately proceeds to call ss.Exit, which will close all output streams going to the SSH client. This may interrupt any asynchronous io.Copy still copying data. To fix this, we close the write-side of the os.Pipes after the process has finished (and before calling ss.Exit) and synchronously wait for the io.Copy routines to finish.
Fixes #7601