Skip to content
This repository was archived by the owner on Sep 9, 2020. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 22 additions & 5 deletions internal/gps/cmd_unix.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import (
"context"
"os"
"os/exec"
"syscall"
"time"

"github.com/pkg/errors"
Expand All @@ -25,11 +26,26 @@ type cmd struct {
}

func commandContext(ctx context.Context, name string, arg ...string) cmd {
// Grab the caller's context and pass a derived one to CommandContext.
c := cmd{ctx: ctx}
ctx, cancel := context.WithCancel(ctx)
c.Cmd = exec.CommandContext(ctx, name, arg...)
c.cancel = cancel
// Create a one-off cancellable context for use by the CommandContext, in
// the event that we have to force a Process.Kill().
ctx2, cancel := context.WithCancel(context.Background())
Copy link
Contributor

@tamird tamird Oct 13, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comment I left in #1269; we should just not use a context here and just c.Cmd.Process.Kill directly instead.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think that's racy, unfortunately. IIRC, the code that your original PR got rid of in favor of exec.CommandContext() had all these nasty workarounds, including passing around integer address pointers for atomic ops, in order to be able to safely call c.Cmd.Process.Kill() directly without any possibility of that call being made on a nil pointer.

i'll go back and double-check, but i know that the thing i really liked about using exec.CommandContext in the first place was that it gave us a safe way of invoking Kill() by taking advantage of timing information that is otherwise only accessible from within the os/exec package.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do double check - as far as I know, there's no possibility of Cmd.Process being nil after Start returns successfully.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, so, it was a bit more complicated than that - while i think you're correct that there's no possibility of Command.Process being nil, it was Command.ProcessState that gave problems:

// If the process doesn't exit immediately, check every 50ms, up to 3s,
// after which send a hard kill.
//
// Cannot rely on cmd.ProcessState.Exited() here, as that is not set
// correctly when the process exits due to a signal. See
// https://github.com/golang/go/issues/19798 . Also cannot rely on it
// because cmd.ProcessState will be nil before the process exits, and
// checking if nil create a data race.
if !atomic.CompareAndSwapInt32(isDone, 1, 1) {
to := time.NewTimer(3 * time.Second)
tick := time.NewTicker(50 * time.Millisecond)
defer to.Stop()
defer tick.Stop()
// Loop until the ProcessState shows up, indicating the proc has exited,
// or the timer expires and
for !atomic.CompareAndSwapInt32(isDone, 1, 1) {
select {
case <-to.C:
return cmd.Process.Kill()
case <-tick.C:
}
}
}

the goal there was avoiding sending a signal to a process that had already terminated, and that's the thing that's difficult to ascertain from anywhere other than the thread that's Wait()ing. all i can remember about why i went to those lengths is because i wanted to avoid returning an error to the broader system when that error was simply a case of our mismanagement of the process (signal-after-termination), rather than an actual error from the process. yes, we could sniff the error, but...well, it's an unexported, potentially os-specific error there. not ideal.

(and yes, that implies that this system should also probably be returning the cancelation error, rather than one from the underlying process, when cancelation is instructed)

so yeah, i'd prefer we stick with the reuse of context for the kill semantics here, as it both insulates us from having to think about any of the above, and may also provide a more useful point for understanding the causal ordering of events here (signal send start, signal send complete, process terminated, etc.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand what you're saying here. The code in os/exec is no more resilient to any of these problems; it just ignores the error returned from Process.Kill.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

¯\_(ツ)_/¯ - maybe it's all equivalent. i don't have more time i can afford to invest in this right now, and figuring out if there was something else that motivated the original code. so - if you and @jmank88 can agree that just calling Process.Kill() is equivalent, then i'll relent, and we can just use that.

if not, this route feels less dragon-y to me, and without adequate time to dispel the fog, i've gotta follow my gut.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'm with @tamird here. We could do this in another PR though, right? I can put something up after this is merged for us to stare at some more in isolation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, good point, can do it in a follow-up.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought that's what #1269 is.


c := cmd{
Cmd: exec.CommandContext(ctx2, name, arg...),
cancel: cancel,
ctx: ctx,
}

// Force subprocesses into their own process group, rather than being in the
// same process group as the dep process. Because Ctrl-C sent from a
// terminal will send the signal to the entire currently running process
// group, this allows us to directly manage the issuance of signals to
// subprocesses.
c.Cmd.SysProcAttr = &syscall.SysProcAttr{
Setpgid: true,
Pgid: 0,
}

return c
}

Expand All @@ -47,6 +63,7 @@ func (c cmd) CombinedOutput() ([]byte, error) {
var b bytes.Buffer
c.Cmd.Stdout = &b
c.Cmd.Stderr = &b

if err := c.Cmd.Start(); err != nil {
return nil, err
}
Expand Down
23 changes: 23 additions & 0 deletions internal/gps/source_manager.go
Original file line number Diff line number Diff line change
Expand Up @@ -741,6 +741,29 @@ const (
ctExportTree
)

func (ct callType) String() string {
switch ct {
case ctHTTPMetadata:
return "Retrieving go get metadata"
case ctListVersions:
return "Retrieving latest version list"
case ctGetManifestAndLock:
return "Reading manifest and lock data"
case ctListPackages:
return "Parsing PackageTree"
case ctSourcePing:
return "Checking for upstream existence"
case ctSourceInit:
return "Initializing local source cache"
case ctSourceFetch:
return "Fetching latest data into local source cache"
case ctExportTree:
return "Writing code tree out to disk"
default:
panic("unknown calltype")
}
}

// callInfo provides metadata about an ongoing call.
type callInfo struct {
name string
Expand Down