Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: cmd/cgo: unsafe FFI calls #42469

Open
DemiMarie opened this issue Nov 9, 2020 · 2 comments
Open

proposal: cmd/cgo: unsafe FFI calls #42469

DemiMarie opened this issue Nov 9, 2020 · 2 comments
Labels
Projects
Milestone

Comments

@DemiMarie
Copy link

@DemiMarie DemiMarie commented Nov 9, 2020

The Glasgow Haskell Compiler (GHC) differentiates between “safe” and “unsafe” FFI calls. “safe” FFI calls are allowed to block and call back into Haskell, but have a substantial overhead. “unsafe” FFI calls are not allowed to block, but are as fast as a C function call.

While Go FFI will always be slower due to stack switching, this seems to account for only a small amount of the overhead that others have observed. If the C function is guaranteed to be short-running, a significant speedup can be obtained by making a direct call into C code, without involving the Go scheduler. Of course, if the C function blocks, this is bad, but in many cases, it can be guaranteed not to. Calling back into Go from an unsafe FFI call is undefined behavior, but in many cases such calls are known not to occur.

@gopherbot gopherbot added this to the Proposal milestone Nov 9, 2020
@gopherbot gopherbot added the Proposal label Nov 9, 2020
@ianlancetaylor ianlancetaylor changed the title Proposal: unsafe FFI calls proposal: cmd/cgo: unsafe FFI calls Nov 9, 2020
@ianlancetaylor ianlancetaylor added this to Incoming in Proposals Nov 9, 2020
@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Nov 9, 2020

This will produce programs that usually work but sometimes hang for inexplicable reasons. I don't think we've come close to the theoretical limit on speeding up cgo calls without removing safety.

Can you point to some documentation for GHC unsafe calls? Thanks.

@smasher164
Copy link
Member

@smasher164 smasher164 commented Nov 10, 2020

@ianlancetaylor Here is a section in the GHC User Guide about guaranteed call safety: https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/ffi-chap.html#guaranteed-call-safety

The Haskell 2010 Report specifies that safe FFI calls must allow foreign calls to safely call into Haskell code. In practice, this means that the garbage collector must be able to run while these calls are in progress, moving heap-allocated Haskell values around arbitrarily.

This greatly constrains library authors since it implies that it is not safe to pass any heap object reference to a safe foreign function call. For instance, it is often desirable to pass an unpinned ByteArray#s directly to native code to avoid making an otherwise-unnecessary copy. However, this can only be done safely if the array is guaranteed not to be moved by the garbage collector in the middle of the call.

The Chapter does not require implementations to refrain from doing the same for unsafe calls, so strictly Haskell 2010-conforming programs cannot pass heap-allocated references to unsafe FFI calls either.

In previous releases, GHC would take advantage of the freedom afforded by the Chapter by performing safe foreign calls in place of unsafe calls in the bytecode interpreter. This meant that some packages which worked when compiled would fail under GHCi (e.g. #13730).

However, since version 8.4 this is no longer the case: GHC guarantees that garbage collection will never occur during an unsafe call, even in the bytecode interpreter, and further guarantees that unsafe calls will be performed in the calling thread.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Proposals
Incoming
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.