New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: support dlclose with -buildmode=c-shared #11100

Open
mattn opened this Issue Jun 6, 2015 · 11 comments

Comments

Projects
None yet
9 participants
@mattn
Member

mattn commented Jun 6, 2015

package main

import (
    "C"
    "fmt"
)

var (
    c chan string
)

func init() {
    c = make(chan string)
    go func() {
        n := 1
        for {
            switch {
            case n%15 == 0:
                c <- "FizzBuzz"
            case n%3 == 0:
                c <- "Fizz"
            case n%5 == 0:
                c <- "Buzz"
            default:
                c <- fmt.Sprint(n)
            }
            n++
        }
    }()
}

//export fizzbuzz
func fizzbuzz() *C.char {
    return C.CString(<-c)
}

func main() {
}

build this with

$ go build -buildmode=c-shared -o libfizzbuzz.so libfizzbuzz.go

then go

from ctypes import *
import _ctypes
lib = CDLL("./libfizzbuzz.so")
lib.fizzbuzz.restype = c_char_p
print lib.fizzbuzz()
print lib.fizzbuzz()
print lib.fizzbuzz()
print lib.fizzbuzz()
print lib.fizzbuzz()
print lib.fizzbuzz()
_ctypes.dlclose(lib._handle)
1
2
Fizz
4
Buzz
Fizz
Segmentation fault

@minux minux closed this Jun 6, 2015

@minux

This comment has been minimized.

Member

minux commented Jun 6, 2015

I don't understand. What do you expect otherwise?
The code is still running, and you've unmapped its
pages.

In general, it's impossible to dlclose a Go shared
library (or a plugin).

@mikioh mikioh changed the title from c-shared crash with unload to runtime: dlclose in shared library causes segmentation fault Jun 6, 2015

@mattn

This comment has been minimized.

Member

mattn commented Jun 7, 2015

eventhough close(c) and wait exiting goroutine, it reproduce.

On 6/7/15, Minux Ma notifications@github.com wrote:

I don't understand. What do you expect otherwise?
The code is still running, and you've unmapped its
pages.


Reply to this email directly or view it on GitHub:
#11100 (comment)

  • Yasuhiro Matsumoto
@minux

This comment has been minimized.

Member

minux commented Jun 7, 2015

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Jun 7, 2015

I think this is a legitimate feature request. Although we currently do not support calling dlclose on a Go shared library, and it would be difficult to make it work, it is not fundamentally impossible.

@ianlancetaylor ianlancetaylor reopened this Jun 7, 2015

@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Jun 7, 2015

@minux

This comment has been minimized.

Member

minux commented Jun 7, 2015

@mattn

This comment has been minimized.

Member

mattn commented Jun 8, 2015

This also reproduce.

package main

import (
        "C"
        "fmt"
)

var (
        c chan string
        q chan struct{}
)

func init() {
        c = make(chan string)
        q = make(chan struct{})
        go func() {
                defer func() {
                        recover()
                        q <- struct{}{}
                }()
                n := 1
                for {
                        switch {
                        case n%15 == 0:
                                c <- "FizzBuzz"
                        case n%3 == 0:
                                c <- "Fizz"
                        case n%5 == 0:
                                c <- "Buzz"
                        default:
                                c <- fmt.Sprint(n)
                                println("stop")
                        }
                        n++
                }
        }()
}

//export fizzbuzz
func fizzbuzz() *C.char {
        return C.CString(<-c)
}

//export finish
func finish() {
        close(c) // occur panic of sending closed channel in above
        <-q      // wait goroutine
}

func main() {
}
from ctypes import *
import _ctypes
lib = CDLL("./libfizzbuzz.so")
lib.fizzbuzz.restype = c_char_p
print lib.fizzbuzz()
print lib.fizzbuzz()
print lib.fizzbuzz()
print lib.fizzbuzz()
print lib.fizzbuzz()
print lib.fizzbuzz()
lib.finish()
_ctypes.dlclose(lib._handle)
@mattn

This comment has been minimized.

Member

mattn commented Jun 8, 2015

i added runtime.LockOSThread() in top of init() but not.

@gcatlin

This comment has been minimized.

gcatlin commented Jul 13, 2015

I have a need for this functionality too. My goal is to be able to make changes to a running Go game engine without needing to completely reload the game.

The implementation idea is to have 2 layers, the platform layer and the game layer.

The platform layer is GOOS/GOARCH-specific and is written in C. It is responsible for the game loop wherein it gathers controller input and manages image and sound buffers that it provides to the game layer for writing. It uses OS-provided functionality to output graphics and sound.

The game layer is platform agnostic, contains game logic only, and is written in Go. It is called by the platform layer once per iteration of the game loop. The game layer takes the controller input and buffers provided by the platform layer, updates the game state, and writes to the buffers.

The game layer is a shared library, built using -buildmode=c-shared, that can be edited and recompiled at any time. When the platform layer detects that the shared library was modified, it unloads the previous version of the library (dlclose) then loads the new version of the library (dlopen).

See https://gist.github.com/gcatlin/e09359f6e53f37e74a82

I'm trying this on darwin/amd64 and getting the following error when calling dlopen on the shared library for the second time (i.e. after detecting that the shared library has changed):

runtime/cgo: could not obtain pthread_keys
    tried 0x101 0x102 0x103 0x104 0x105 0x106 0x107 0x108 0x109 0x10a 0x10b 0x10c 0x10d 0x10e 0x10f 0x110 0x111 0x112 0x113 0x115 0x116 0x117 0x118 0x119 0x11a 0x11b 0x11c 0x11d 0x11e 0x11f 0x120 0x121 0x122 0x123 0x124 0x125 0x126 0x127 0x128 0x129 0x12a 0x12b 0x12c 0x12d 0x12e 0x12f 0x130 0x131 0x132 0x133 0x134 0x135 0x136 0x137 0x138 0x139 0x13a 0x13b 0x13c 0x13d 0x13e 0x13f 0x140 0x141 0x142 0x143 0x144 0x145 0x146 0x147 0x148 0x149 0x14a 0x14b 0x14c 0x14d 0x14e 0x14f 0x150 0x151 0x152 0x153 0x154 0x155 0x156 0x157 0x158 0x159 0x15a 0x15b 0x15c 0x15d 0x15e 0x15f 0x160 0x161 0x162 0x163 0x164 0x165 0x166 0x167 0x168 0x169 0x16a 0x16b 0x16c 0x16d 0x16e 0x16f 0x170 0x171 0x172 0x173 0x174 0x175 0x176 0x177 0x178 0x179 0x17a 0x17b 0x17c 0x17d 0x17e 0x17f 0x180 0x181
fatal error: cgo callback before cgo call

Is there a different way to achieve this with Go?

@crawshaw crawshaw changed the title from runtime: dlclose in shared library causes segmentation fault to runtime: support dlclose with -buildmode=c-shared Jul 17, 2015

@crawshaw crawshaw modified the milestones: Go1.6, Unplanned Jul 17, 2015

ianlancetaylor added a commit that referenced this issue Oct 8, 2015

cmd/link: pass -Wl,-z,nodelete when creating an ELF shared library
Go shared libraries do not support dlclose, and there is no likelihood
that they will suppose dlclose in the future.  Set the DF_1_NODELETE
flag to tell the dynamic linker to not attempt to remove them from
memory.  This makes the shared library act as though every call to
dlopen passed the RTLD_NODELETE flag.

Fixes #12582.
Update #11100.
Update #12873.

Change-Id: Id4b6e90a1b54e2e6fc8355b5fb22c5978fc762b4
Reviewed-on: https://go-review.googlesource.com/15605
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>

@rsc rsc modified the milestones: Unplanned, Go1.6 Nov 5, 2015

@z505

This comment has been minimized.

z505 commented May 1, 2017

Minux said:

It's possible if the user can be certain it doesn't hold onto
any Go objects and all resources allocated by Go code
has been freed (esp. no background goroutines).

However, as there is no way to kill a goroutine, I think all
sufficiently sophisticated Go shared library will not be unloadable.

Hi, so how does go know it is safe for the exe/elf to close if a user kills the application or hits the close button on a win32 gui window... There must be some way of finding out if it is safe? or go program just exits badly with hanging data around?
Or how do you halt/exit a go app safely (an exe, not a dll)..? Would exiting a go app (exe) safely be the same as unloading a dll safely?

The go runtime does not keep a reference count of the number of goroutines currently open? This should not be a performance hit to keep track of N number of goroutines open? Each goroutine opened increments a ref count by 1. But maybe there is already something implemented like this, that could be tapped into and used for dll's to know if all goroutines are finished.

Minux says:

For example, the os/signal package contains a background
goroutine to check for newly arrived signals.

Can you increment a reference counter and unload the library later once all signals are finished (goroutine ref count hit zero)? How does the exe/elf normally exit safely and kill the program if goroutines could be still running? Simply make the DLL unload function act like how a normal go exe program safely exits? easier said than done, likely :-)

@glycerine

This comment has been minimized.

glycerine commented May 1, 2017

Minux wrote,

You can't stop the runtime, which uses its own OS threads.

This is the root of the problem, @z505. I don't believe there is any graceful shutdown of the runtime threads under normal process exit/termination. Hence they may still have references to DLL memory. So currently there's no safety problem with process termination; just doing an exit() call, it simply unceremoniously stops the process, and doesn't need to do any graceful cleanup.

or go program just exits badly with hanging data around?

Yep.

@brutestack

This comment has been minimized.

brutestack commented May 16, 2018

I'm using Go c-shared library in my Unity3D game.
Unity itself does not support unloading libraries, but when you try to exit Unity Editor or Unity based game it starts waiting untill all libraries complete their work (wait untill all threads stop)

In case of using Go c-shared library which creates at least one goroutine Unity will wait forever and will not be able to exit. This will not happen if library does not utilize goroutines.
So, go creates thead for my goroutine and never terminates it even if there is nothing to do in this thread.

Here is sample library code to reproduce:

package main

import "C"
import "time"

//export TestLib
func TestLib() {
	go func() {
		for i := 0; i < 10; i++ {
			fmt.Println("testlib.dll is here")
			time.Sleep(3 * time.Second)
		}
		fmt.Println("testlib.dll thread Done")
	}()
}

func main() {
}

I'm compiling that library on Linux that way:
env GOOS=windows GOARCH=amd64 CGO_ENABLED=1 CXX=x86_64-w64-mingw32-g++ CC=x86_64-w64-mingw32-gcc go build -o $(go env GOPATH)/lib/win64/testlib.dll -buildmode c-shared unity3d.com/testlib

Here is c# class that utilizes this library:

using System;
using System.Runtime.InteropServices;
using UnityEngine;

public class TestLibrary : IDisposable {
	public TestLibrary()
	{
		ServerProcess();
	}

	private void ServerProcess()
	{
		TestLib();
	}
		
	public void Dispose()
	{
	}


	[DllImport ("testlib")]
	private extern static void TestLib ();

}

Unity is besed on Mono, that's why c#.

There is a very-very dirty trick that fixes my problem and allows to exit my game without wating forever - I have implemented function in my Go library that causes panic and library crashes when I need to stop everything and exit game. Actually nobody should do that, but it works.
Additional code in Go library:

//export Panic
func Panic() {
	var panicChan chan bool
	close(panicChan)
}

Wortst of it is that I don't know what exactly crashes after calling Panic() from C# - Unity(Mono) or Go library, but I suspect Unity.
Anfortunately crash is the only way to exit Unity game that uses Go c-shared library with goroutines...

Another workaround is to compile Go code as executable and run it in separate process, but this is not sutible for iOS version of my game because iOS does not let executing separate processes (even included in the same application boundle) without using private APIs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment