New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use syscall.Dup2 capture cgo output #9

Merged
merged 1 commit into from Aug 4, 2018
File filter...
Filter file types
Jump to file or symbol
Failed to load files and symbols.
+35 −18
Diff settings

Always

Just for now

Use syscall.Dup2 capture cgo output

In go1.10, C.stdout and C.stderr are no longer assignable. This method
is similar to how shells perform redirects and should be more generic.

Fixes #8
  • Loading branch information...
lutzky committed Jul 26, 2018
commit f15804f0e6285e5634cf78f703ca544a6936a8fa
Copy path View file
@@ -8,18 +8,13 @@ import "C"

import (
"bytes"
"errors"
"io"
"os"
"sync"
"syscall"
"unsafe"
)

var (
// ErrFDOpenFailed indicates that C's fdopen has failed
ErrFDOpenFailed = errors.New("fdopen returned nil")
)

var lockStdFileDescriptorsSwapping sync.Mutex // FIXME our solution is not concurrent-safe. Find a better solution because this might be a bottleneck in the future.

// Capture captures stderr and stdout of a given function call.
@@ -72,11 +67,29 @@ func CaptureWithCGo(call func()) (output []byte, err error) {
lockStdFileDescriptorsSwapping.Lock()
defer lockStdFileDescriptorsSwapping.Unlock()

originalStdErr, originalStdOut := os.Stderr, os.Stdout
originalCStdErr, originalCStdOut := C.stderr, C.stdout
originalStdout, e := syscall.Dup(syscall.Stdout)
if e != nil {
return nil, e
}

originalStderr, e := syscall.Dup(syscall.Stderr)
if e != nil {
return nil, e
}

defer func() {
os.Stderr, os.Stdout = originalStdErr, originalStdOut
C.stderr, C.stdout = originalCStdErr, originalCStdOut
if e := syscall.Dup2(originalStdout, syscall.Stdout); e != nil {
err = e
}
if e := syscall.Close(originalStdout); e != nil {
err = e
}
if e := syscall.Dup2(originalStderr, syscall.Stderr); e != nil {
err = e
}
if e := syscall.Close(originalStderr); e != nil {
err = e
}
}()

r, w, err := os.Pipe()
@@ -93,14 +106,12 @@ func CaptureWithCGo(call func()) (output []byte, err error) {
cw := C.CString("w")
defer C.free(unsafe.Pointer(cw))

f := C.fdopen((C.int)(w.Fd()), cw)
if f == nil {
return nil, ErrFDOpenFailed
if e := syscall.Dup2(int(w.Fd()), syscall.Stdout); e != nil {
return nil, e
}
if e := syscall.Dup2(int(w.Fd()), syscall.Stderr); e != nil {
return nil, e
}
defer C.fclose(f)

os.Stderr, os.Stdout = w, w
C.stderr, C.stdout = f, f

out := make(chan []byte)
go func() {
@@ -116,12 +127,18 @@ func CaptureWithCGo(call func()) (output []byte, err error) {

call()

C.fflush(f)
C.fflush(C.stdout)

err = w.Close()
if err != nil {
return nil, err
}
if e := syscall.Close(syscall.Stdout); e != nil {

This comment has been minimized.

@zimmski

zimmski Aug 4, 2018

Owner

These changes work perfectly, THANK YOU! However, I have one question that I could not figure out alone since I am not that familiar with the "dup" syscalls.

Are these close calls needed because syscall.Stdout/Stderr are copies due to Dup2 of w.Fd()?

This comment has been minimized.

@lutzky

lutzky Aug 4, 2018

Contributor

Yes, the Dup and Dup2 syscalls are used by shells to perform redirects, so you can just look at the source code of bash for another example of how this is used.

Assume that when we enter the function the only open file descriptors are stdin (0), stdout (1) and stderr (2). Then we open the pipe, so that becomes file descriptors 3,4. We save (using Dup) 1 and 2, so those become 5 and 6. Then we use Dup2 to copy 3 into 1 and 2, capturing the output. At this point, the underlying file descriptor belonging to the write end of the pipe has three references - 1,2,3. It will only count as closed once all three of those have been closed. The io.Copy call writing into out will block until this happens.

Does that clarify?

This comment has been minimized.

@zimmski

zimmski Aug 4, 2018

Owner

That clarifies it perfectly, thank you! You should write a blog post about that.

This comment has been minimized.

@lutzky

lutzky Aug 4, 2018

Contributor

Will do! Hell, the reason I fixed this bug was a yak shave as part of wanting to write a post about mutation testing...

This comment has been minimized.

@zimmski

zimmski Aug 4, 2018

Owner

That sounds also pretty neat! Please ping me when your material is online :-)

This comment has been minimized.

This comment has been minimized.

return nil, e
}
if e := syscall.Close(syscall.Stderr); e != nil {
return nil, e
}

return <-out, err
}
ProTip! Use n and p to navigate between commits in a pull request.