Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: go build -buildmode=c-shared -linkshared produces .so that isn't loadable by dlopen #60957

Closed
kevin-luke-ctct opened this issue Jun 22, 2023 · 5 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@kevin-luke-ctct
Copy link

kevin-luke-ctct commented Jun 22, 2023

What version of Go are you using (go version)?

go 1.17

Does this issue reproduce with the latest release?

Yes, tried with 1.19 and 1.20

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/kluke/.cache/go-build"
GOENV="/home/kluke/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/kluke/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/kluke/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.20"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.20/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.20.4"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/kluke/test/go.mod"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build788265043=/tmp/go-build -gno-record-gcc-switches"

What did you do?

We have a project that builds multiple plugins using c-go and -buildmode=c-share. We would like to use -linkshared to reduce the total disk usage of the plugins.

So I suppose I should start with the question, is this a reasonable thing to try and do?

$go install -buildmode=shared -linkshared std // To install golang shared libs
$go build -buildmode=c-shared -linkshared -o ./myplugin.so ./pluginsource

What did you expect to see?

Expected ./myplugin.so to be produced that is linked to stdlib.so and also be loadable by dlopen.

What did you see instead?

The plugin compiles successfully but is not loadable by dlopen.

Example

// main.go
package main

func main() {}
// cstuff/cstuff.go
package cstuff

import (
  "fmt"
  "C"
)

//export TestFunction
func TestFunction() {
  fmt.Println("Hello World")
}
// test.py
from ctypes import *
test = cdll.LoadLibrary("./test.so")
test.TestFunction()

Steps:

go install -buildmode=shared -linkshared std
go build -buildmode=c-shared -linkshared -o test.so .
# Library looks OK
❯ ldd ./test.so
	linux-vdso.so.1 (0x00007fff59d19000)
	libstd.so => /usr/lib/go-1.20/pkg/linux_amd64_dynlink/libstd.so (0x00007f6a9b394000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6a9b168000)
	libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f6a9b14c000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f6a9b146000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6a9b123000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f6a9e0dc000)
	
❯ python ./test.py
Traceback (most recent call last):
  File "./test.py", line 3, in <module>
    test = cdll.LoadLibrary("./test.so")
  File "/usr/lib/python3.8/ctypes/__init__.py", line 451, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python3.8/ctypes/__init__.py", line 373, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /usr/lib/go-1.20/pkg/linux_amd64_dynlink/libstd.so: undefined symbol: main.main
@dmitshur dmitshur changed the title affected/package: go build -buildmode=c-shared -linkshared cmd/compile: go build -buildmode=c-shared -linkshared produces .so that isn't loadable by dlopen Jun 23, 2023
@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 23, 2023
@dmitshur dmitshur added this to the Backlog milestone Jun 23, 2023
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jun 23, 2023
@dmitshur
Copy link
Contributor

CC @golang/compiler.

@cherrymui
Copy link
Member

So I suppose I should start with the question, is this a reasonable thing to try and do?

As you may have thought of, currently the c-shared build mode and -linkshared don't work together. And there is no plan to support that. Thanks.

@cherrymui cherrymui closed this as not planned Won't fix, can't repro, duplicate, stale Jun 23, 2023
@kevin-luke-ctct
Copy link
Author

@cherrymui could you elaborate as to why building c-shared libs in -linkshared mode isn't supported?

Is it a matter of difficulty, or is there an architectural reason to not support that mode of operation?

@ianlancetaylor
Copy link
Contributor

As far as I can see, it's not inherently impossible. But in general -buildmode=shared is not well supported (see #47788) and nobody has worked on making -buildmode=shared and -buildmode=c-shared work together.

@laboger
Copy link
Contributor

laboger commented Jun 30, 2023

Shouldn't the unsupported buildmode option combinations result in error or warning messages to avoid generating invalid .so files?

@golang golang locked and limited conversation to collaborators Jun 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

6 participants