runtime: support dlclose with -buildmode=c-shared

[mattn]

package main

import (
    "C"
    "fmt"
)

var (
    c chan string
)

func init() {
    c = make(chan string)
    go func() {
        n := 1
        for {
            switch {
            case n%15 == 0:
                c <- "FizzBuzz"
            case n%3 == 0:
                c <- "Fizz"
            case n%5 == 0:
                c <- "Buzz"
            default:
                c <- fmt.Sprint(n)
            }
            n++
        }
    }()
}

//export fizzbuzz
func fizzbuzz() *C.char {
    return C.CString(<-c)
}

func main() {
}

build this with

$ go build -buildmode=c-shared -o libfizzbuzz.so libfizzbuzz.go

then go

from ctypes import *
import _ctypes
lib = CDLL("./libfizzbuzz.so")
lib.fizzbuzz.restype = c_char_p
print lib.fizzbuzz()
print lib.fizzbuzz()
print lib.fizzbuzz()
print lib.fizzbuzz()
print lib.fizzbuzz()
print lib.fizzbuzz()
_ctypes.dlclose(lib._handle)

1
2
Fizz
4
Buzz
Fizz
Segmentation fault

[minux]

I don't understand. What do you expect otherwise?
The code is still running, and you've unmapped its
pages.

In general, it's impossible to dlclose a Go shared
library (or a plugin).

[mattn]

eventhough close(c) and wait exiting goroutine, it reproduce.

On 6/7/15, Minux Ma notifications@github.com wrote:

I don't understand. What do you expect otherwise?
The code is still running, and you've unmapped its
pages.

---

Reply to this email directly or view it on GitHub:
#11100 (comment)

• Yasuhiro Matsumoto

[minux]

You can't stop the runtime, which uses its own OS threads.

[ianlancetaylor]

I think this is a legitimate feature request. Although we currently do not support calling dlclose on a Go shared library, and it would be difficult to make it work, it is not fundamentally impossible.

[minux]

It's possible if the user can be certain it doesn't hold onto any Go objects and all resources allocated by Go code has been freed (esp. no background goroutines). However, as there is no way to kill a goroutine, I think all sufficiently sophisticated Go shared library will not be unloadable. For example, the os/signal package contains a background goroutine to check for newly arrived signals. Any use of os/signal.Handle will leave the Go shared library un-unloadable. I'm sure there are other cases. What's more, the runtime can't reliably detect whether it's safe to unload the Go shared library, so rather than make dlclose potentially trigger segmentation fault later, I'd rather make dlclose always fail and document that.

[mattn]

This also reproduce.

package main

import (
        "C"
        "fmt"
)

var (
        c chan string
        q chan struct{}
)

func init() {
        c = make(chan string)
        q = make(chan struct{})
        go func() {
                defer func() {
                        recover()
                        q <- struct{}{}
                }()
                n := 1
                for {
                        switch {
                        case n%15 == 0:
                                c <- "FizzBuzz"
                        case n%3 == 0:
                                c <- "Fizz"
                        case n%5 == 0:
                                c <- "Buzz"
                        default:
                                c <- fmt.Sprint(n)
                                println("stop")
                        }
                        n++
                }
        }()
}

//export fizzbuzz
func fizzbuzz() *C.char {
        return C.CString(<-c)
}

//export finish
func finish() {
        close(c) // occur panic of sending closed channel in above
        <-q      // wait goroutine
}

func main() {
}

from ctypes import *
import _ctypes
lib = CDLL("./libfizzbuzz.so")
lib.fizzbuzz.restype = c_char_p
print lib.fizzbuzz()
print lib.fizzbuzz()
print lib.fizzbuzz()
print lib.fizzbuzz()
print lib.fizzbuzz()
print lib.fizzbuzz()
lib.finish()
_ctypes.dlclose(lib._handle)

[mattn]

i added runtime.LockOSThread() in top of init() but not.

[gcatlin]

I have a need for this functionality too. My goal is to be able to make changes to a running Go game engine without needing to completely reload the game.

The implementation idea is to have 2 layers, the platform layer and the game layer.

The platform layer is GOOS/GOARCH-specific and is written in C. It is responsible for the game loop wherein it gathers controller input and manages image and sound buffers that it provides to the game layer for writing. It uses OS-provided functionality to output graphics and sound.

The game layer is platform agnostic, contains game logic only, and is written in Go. It is called by the platform layer once per iteration of the game loop. The game layer takes the controller input and buffers provided by the platform layer, updates the game state, and writes to the buffers.

The game layer is a shared library, built using -buildmode=c-shared, that can be edited and recompiled at any time. When the platform layer detects that the shared library was modified, it unloads the previous version of the library (dlclose) then loads the new version of the library (dlopen).

See https://gist.github.com/gcatlin/e09359f6e53f37e74a82

I'm trying this on darwin/amd64 and getting the following error when calling dlopen on the shared library for the second time (i.e. after detecting that the shared library has changed):

runtime/cgo: could not obtain pthread_keys
    tried 0x101 0x102 0x103 0x104 0x105 0x106 0x107 0x108 0x109 0x10a 0x10b 0x10c 0x10d 0x10e 0x10f 0x110 0x111 0x112 0x113 0x115 0x116 0x117 0x118 0x119 0x11a 0x11b 0x11c 0x11d 0x11e 0x11f 0x120 0x121 0x122 0x123 0x124 0x125 0x126 0x127 0x128 0x129 0x12a 0x12b 0x12c 0x12d 0x12e 0x12f 0x130 0x131 0x132 0x133 0x134 0x135 0x136 0x137 0x138 0x139 0x13a 0x13b 0x13c 0x13d 0x13e 0x13f 0x140 0x141 0x142 0x143 0x144 0x145 0x146 0x147 0x148 0x149 0x14a 0x14b 0x14c 0x14d 0x14e 0x14f 0x150 0x151 0x152 0x153 0x154 0x155 0x156 0x157 0x158 0x159 0x15a 0x15b 0x15c 0x15d 0x15e 0x15f 0x160 0x161 0x162 0x163 0x164 0x165 0x166 0x167 0x168 0x169 0x16a 0x16b 0x16c 0x16d 0x16e 0x16f 0x170 0x171 0x172 0x173 0x174 0x175 0x176 0x177 0x178 0x179 0x17a 0x17b 0x17c 0x17d 0x17e 0x17f 0x180 0x181
fatal error: cgo callback before cgo call

Is there a different way to achieve this with Go?