compiler: implement go:cgo_import_dynamic and lower Darwin cgo import trampolines (#5604) - #5612
Open
neomantra wants to merge 3 commits into
Open
compiler: implement go:cgo_import_dynamic and lower Darwin cgo import trampolines (#5604)#5612neomantra wants to merge 3 commits into
neomantra wants to merge 3 commits into
Conversation
Signed-off-by: Evan Wies <evan@neomantra.net>
Route the remaining variadic imports (open, openat, fcntl) through fixed-signature C wrappers the way ioctl already is: the syscall engine calls imported addresses through fixed-signature function pointers, and a variadic callee reads its variadic arguments from the stack on darwin/arm64, so direct calls silently receive garbage arguments (an x/sys unix.Open created files with mode 0 instead of the requested mode). Of the symbols darwin's generated syscall wrappers import, exactly open, openat, fcntl, and ioctl are variadic. Also accept the one- and two-operand forms of //go:cgo_import_dynamic like the gc compiler does, ignore the unused library operand, and only replace loads of uintptr-typed trampoline globals. Signed-off-by: Evan Wies <evan@neomantra.net>
The standard library's function-based trampoline pattern has the same variadic problem as the address-global pattern: of the libc functions darwin's zsyscall wrappers import, open, openat, fcntl, and ioctl are variadic, but createDarwinFuncPCABI0Call only routed open through a fixed-signature wrapper. As a result syscall.SetNonblock (fcntl F_SETFL with the new flags in the variadic slot) observably wrote garbage file flags on darwin/arm64, and the stdlib ioctl and openat paths were equally unsound. Use the shared darwinVariadicImports table for this path too, replacing the open-only special case. Signed-off-by: Evan Wies <evan@neomantra.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
As noted in #5604, this was worked through with LLM. It adds parsing of the
go:cgo_import_dynamicdirective and implementation for Darwin.The other Tinygo-supported platform that could use this is OpenBSD, but is more important for Darwin. It targets
loadASTCommentsgenerally because these directives are not necessarily attached to a function/module/etc; the comment there explains it.The following description of the PR is LLM-generated from our journey; it is verbose but comprehensive. I have read it and I have reviewed all the submitted code, both humanually and with multiple LLMs. Thanks for the review.
Summary
Extend TinyGo's existing Darwin stdlib trampoline lowering to the variable-based pattern used by
golang.org/x/sys/unixandinternal/syscall/unix, and route all four variadic libc imports (open,openat,fcntl,ioctl) through fixed-signature wrappers on both trampoline paths.Fixes #5604.
Fixes #5365.
Builds on #5401.
The companion
_ioctl/___sincos_stretstubs (macos-minimal-sdk#5) are already indevvia the macos-minimal-sdk v0.1.0 submodule update.Problem
Darwin syscall wrappers generated by x/sys declare globals such as
libc_ioctl_trampoline_addrand associate them with dylib symbols using file-level//go:cgo_import_dynamicpragmas. TinyGo does not compile the assembly that initializes those globals, so the existing libc syscall engine receives address zero and crashes beforemainin Bubble Tea programs.TinyGo already avoids the equivalent assembly trampoline for the standard library:
createDarwinFuncPCABI0Callrecognizesabi.FuncPCABI0(libc_*_trampoline)and substitutes the address of an external libc declaration. This change applies that same lowering to the x/sys global-load pattern.That existing stdlib lowering also carries a silent bug of its own: among its variadic imports it routes only
openthrough a fixed-signature wrapper, so stdlibfcntl,ioctl, andopenatcalls corrupt their variadic arguments on darwin/arm64. This PR fixes that too, since the new lowering needs the same wrapper set anyway.Scope
This is a targeted implementation of the
//go:cgo_import_dynamicpattern used by Darwin's generated syscall wrappers, not general cross-platform support for the directive. Although the file-level metadata is parsed during compilation, thelibc_*_trampoline_addrsymbol-address substitution is restricted toGOOS=darwin; non-Darwin targets retain their existing behavior.Upstream Go also uses
//go:cgo_import_dynamicon platforms such as OpenBSD, AIX, Solaris, and illumos. Supporting those platforms, the directive's library operand, other use patterns, and platform-specific linking or calling conventions remains out of scope.Implementation
//go:cgo_import_dynamicpragmas into a package-local map for Darwin trampoline lowering, accepting the same one-, two-, and three-operand forms as the gc compiler (local [remote ["library"]]); the remote symbol defaults to the local one and the library operand is ignored (the linker already resolves against libSystem).libc_*_trampoline_addrglobals withptrtointof an external declaration for the pragma's remote symbol. Globals of any other type keep their normal load.$INODE64variants.darwinVariadicImportstable. Of the symbols Darwin's generated syscall wrappers import (zsyscall_darwin_*.goin x/sys and the standard library), exactlyopen,openat,fcntl, andioctlare variadic. The syscall engine calls imported addresses through fixed-signature function pointers (tinygo_syscallXand friends) that pass every argument in a register, while a variadic callee reads its variadic arguments from the stack on darwin/arm64, so direct calls silently receive garbage in the variadic slot (observed:EFAULTfromioctl;unix.Opencreating files with mode 0 instead of the requested mode).FuncPCABI0trampoline lowering (createDarwinFuncPCABI0Call) at the same table, replacing itsopen-only special case. This fixes the pre-existing silent bug on the stdlib path:syscall.SetNonblock(fcntlF_SETFLwith the new flags in the variadic slot) observably wrote garbage file flags on darwin/arm64, and the stdlibioctlandopenattrampolines were equally unsound.The companion
_ioctland___sincos_stretstub additions landed upstream in macos-minimal-sdk#5 and reacheddevwith the macos-minimal-sdk v0.1.0 submodule update, so this branch (rebased onto that dev) builds and links darwin x/sys programs out of the box with no SDK changes of its own. No new stubs are needed foropen,openat, orfcntl— those symbols are already in the stock stub list, and their prototypes come from the SDK'ssys/fcntl.h.Verification
go test -tags llvm22 ./compiler -count=1golang.org/x/sys/unix.IoctlGetTermiosrepro under a PTY: returned a populated termios value and<nil>.github.com/charmbracelet/x/term.IsTerminalunder a PTY:true.term.MakeRawandterm.Restoreunder a PTY: both<nil>.examples/spinner: initialized, rendered animated frames, and exited onq.examples/list-default: initialized, rendered the list UI, and exited onq.unix.Openwith mode0o640created the file with mode640exactly,unix.FcntlInt(F_DUPFD, 50)returned fd50, andunix.Openatwith mode0o600created mode600exactly. The pre-fix compiler produced mode0, fd49, and mode41on the same program — the silent-corruption failure mode.FuncPCABI0path):syscall.SetNonblock(fd, true)followed by anF_GETFLreadback yields flags exactly0o4(O_NONBLOCK), and a read from the empty pipe returnsEAGAINimmediately. Without theFuncPCABI0change, the same program showed garbage flags (0o20000110),O_NONBLOCKunset, and the read blocked forever; the blocking read was also reproduced on stockdevat f71b630 with a stdlib-only variant.No startup panic or
os/signal/SIGWINCH failure surfaced in this ladder.Context
This is the actual current Darwin-native root cause behind #5365. It is the "class 2" failure described by @mparrett in #4794 and is independent of the public
syscall.Syscall*API discussed there.An earlier attempt in #5403 made public
syscall.Syscall*calls work by lowering syscall numbers directly to raw Darwin kernel instructions. That was a separate route which bypassed Darwin's libc-based dispatch and was never used by the x/sys/Bubble Tea call chain.After #5401 made x/sys's private linknames resolve, the failing program reached TinyGo's existing
syscall.syscalln/syscall.rawsyscallnlibc dispatcher with the correctioctlarguments but a zero function pointer. The dispatcher was not broken; the assembly-generated bridge that supplied its libc function pointer was missing. This change supplies that address and lets x/sys use TinyGo's existing libc path as intended.This does not implement public
syscall.Syscall*-by-number on Darwin, nor does it guarantee that every libc symbol is already present in the macOS minimal SDK.