New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: -buildmode=c-shared and dlopen-ing a shared library #16805

Open
sbinet opened this Issue Aug 19, 2016 · 4 comments

Comments

Projects
None yet
7 participants
@sbinet
Member

sbinet commented Aug 19, 2016

consider the following GOPATH:

sh> tree .
.
├── mylib.so
└── src
    ├── main.go
    ├── my-cmd
    │   └── main.go
    ├── pkg1
    │   └── pkg.go
    └── pkg2
        └── pkg.go

5 directories, 6 files

with:

// src/pkg1
package pkg1

import "fmt"

var Int = 0
var Map = make(map[string]int)

func init() {
    fmt.Printf("pkg1.Int: %p\n", &Int)
    fmt.Printf("pkg1.Map: %p\n", &Map)
}
// src/pkg2
package pkg2

import "C"
import (
    "fmt"
    "pkg1"
)

//export Load
func Load() {
    fmt.Printf("pkg2.Load...\n")
}

func init() {
    fmt.Printf(">>> pkg2: pkg1.Int: %p\n", &pkg1.Int)
    fmt.Printf(">>> pkg2: pkg1.Map: %p\n", &pkg1.Map)
}
// src/my-cmd/main.go
package main

import "C"

import _ "pkg2"

func main() {}

and:

// src/main.go
package main

// #include <dlfcn.h>
// #cgo LDFLAGS: -ldl
// #include <stdlib.h>
// #include <stdio.h>
//
// void loadPlugin(void *lib) {
//   void (*f)(void) = NULL;
//   char *error = NULL;
//   f = (void (*)(void))(dlsym(lib, "Load"));
//   error = dlerror();
//   if (f == NULL || error != NULL) {
//      fprintf(stderr, "ERROR no such symbol!!! (%s)\n", error);
//      return;
//   }
//   fprintf(stderr, "symbol 'Load' loaded...\n");
//   (*f)();
// }
import "C"

import (
    "fmt"
    "log"
    "pkg1"
    "unsafe"
)

func main() {
    fmt.Printf("main.pkg1.Int: %p\n", &pkg1.Int)
    fmt.Printf("main.pkg1.Map: %p\n", &pkg1.Map)

    fmt.Printf("loading DLL...\n")
    cstr := C.CString("./mylib.so")
    defer C.free(unsafe.Pointer(cstr))
    h := C.dlopen(cstr, C.RTLD_NOW)
    if h == nil {
        log.Fatalf("error loading %s\n", C.GoString(cstr))
    }
    defer C.dlclose(h)

    fmt.Printf("loading plugin...\n")
    C.loadPlugin(h)
    fmt.Printf("loading plugin... [done]\n")

}

running the following command, gives:

sh> go build -buildmode=c-shared -o mylib.so ./src/my-cmd && go run ./src/main.go 
pkg1.Int: 0x728c48
pkg1.Map: 0x70e1e0
main.pkg1.Int: 0x728c48
main.pkg1.Map: 0x70e1e0
loading DLL...
loading plugin...
symbol 'Load' loaded...
pkg1.Int: 0x7f925f7f9a48
pkg1.Map: 0x7f925f7df038
>>> pkg2: pkg1.Int: 0x7f925f7f9a48
>>> pkg2: pkg1.Map: 0x7f925f7df038
pkg2.Load...
loading plugin... [done]

ie: the addresses of pkg1.Int and pkg1.Map are not the same when inspected from the go.main() or pkg1.init and when inspected from the dynamically loaded mylib.so shared library.
also, pkg1.init() is run twice.

here is my environment:

sh> go version
go version go1.7 linux/amd64

sh> go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/binet/work/igo/src/github.com/go-interpreter/example/cshared-bug"
GORACE=""
GOROOT="/usr/lib/go"
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build549233304=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"

from my reading of https://docs.google.com/document/d/1nr-TQHw_er6GOQRsF6T43GGhFDelrAP0NqSS_00RgZQ/edit# this shouldn't happen, even when using the c-shared buildmode instead of the plugin buildmode.

@ianlancetaylor @crawshaw : right ?

@crawshaw

This comment has been minimized.

Contributor

crawshaw commented Aug 19, 2016

I think it's fair to say at this point that what you're doing is not supported yet. Even if the symbols were correctly de-duplicated, the dlopen'ed c-shared library will attempt to initialize the runtime again, which I don't think we have adequate defenses against yet.

(I'll try to get my buildmode=plugin CLs out soon, which take care of this.)

@quentinmit quentinmit added this to the Go1.8Maybe milestone Sep 6, 2016

@quentinmit

This comment has been minimized.

Contributor

quentinmit commented Oct 10, 2016

@crawshaw Is this now resolved by saying "use buildmode=plugin"?

@quentinmit quentinmit added the NeedsFix label Oct 10, 2016

@crawshaw

This comment has been minimized.

Contributor

crawshaw commented Oct 11, 2016

There's a slight variant of this that deserves fixing (and wouldn't be too much work, but I'm a bit short on time): which is using dlopen from a C program on two separately built Go c-shared libraries.

The second library's global constructor should not re-initialize the runtime, but should do most of the work in plugin_lastmoduleinit.

@rsc

This comment has been minimized.

Contributor

rsc commented Oct 20, 2016

If buildmode=plugin works, great. Otherwise, this will need to wait for Go 1.9.

@rsc rsc modified the milestones: Go1.9Early, Go1.8Maybe Oct 20, 2016

@bradfitz bradfitz modified the milestones: Go1.10Early, Go1.9Early May 3, 2017

@bradfitz bradfitz modified the milestones: Go1.10Early, Go1.10 Jun 14, 2017

@rsc rsc modified the milestones: Go1.10, Go1.11 Dec 1, 2017

@bradfitz bradfitz modified the milestones: Go1.11, Go1.12 May 18, 2018

@ianlancetaylor ianlancetaylor added this to the Go1.12 milestone Jun 1, 2018

@bcmills bcmills modified the milestones: Go1.12, Unplanned Oct 24, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment