plugin: add Windows support

[QuestionPython]

hi
Plugin Pkg Work for Windows!?
i want use this for mac,linux,win,... os.
when(what time) fix this?

https://golang.org/pkg/plugin/

[bradfitz]

There is currently nobody working on it, as far as I know.

/cc @alexbrainman @crawshaw

[QuestionPython]

mean in go 1.8 , plugin pkg work for apple-mac,windows and more ?

[bradfitz]

@QuestionPython, yes, it's even documented in multiple places:

https://golang.org/pkg/plugin/

Currently plugins only work on Linux.

https://golang.org/doc/go1.8#plugin

Plugin support is currently only available on Linux.

[alexbrainman]

There is currently nobody working on it, as far as I know.

I am not working on it. Sorry.

Alex

[bradfitz]

We delete all "me too" voting comments per https://golang.org/wiki/NoMeToo. Vote with emoji reactions at top instead.

[bradfitz]

The "me too" comments wouldn't stop, so this issue is now locked. If there are updates, they will be made here.

[0xdevalias]

Out of curiosity, do we know how much effort it would take to implement windows support? Or if there are any blockers to it (and what they are?)

[akavel]

Notably, the recently published Go kernel for Jupyter notebooks is using buildmode=shared, and thus doesn't currently support Windows natively. This is a very cool use case, adding a REPL-like live coding feature to the Go ecosystem, thus it would be really awesome if someone tried to start work on buildmode=shared on Windows to support this use case.

Similar to @0xdevalias , I'm quite interested in some hints as to what is missing for this to work on Windows? I'm especially curious what extra work is needed given that c-shared is already implemented on Windows?

[alexbrainman]

@0xdevalias and @akavel I don't have any effort estimation or any hints as to what missing here. I have not actually looked at what is involved. I am so much behind at fixing old issues ...

Alex

[akavel]

@alexbrainman Thanks! I'll ask on golang-dev then, maybe someone else can shed some light (edit: link to the thread)

[0xdevalias]

There doesn't seem to be a huge amount to it in the src: https://github.com/golang/go/tree/master/src/plugin

My completely naive guess would be figuring the windows equivalents to the C-bindings in plugin_dlopen.go.

The main functions I can see there are:

• https://linux.die.net/man/3/dlopen : The function dlopen() loads the dynamic library file named by the null-terminated string filename and returns an opaque "handle" for the dynamic library.
• https://linux.die.net/man/3/dlsym : The function dlsym() takes a "handle" of a dynamic library returned by dlopen() and the null-terminated symbol name, returning the address where that symbol is loaded into memory.

Googling for "dlopen equivalent windows" led me to the following:

• https://github.com/alainfrisch/flexdll : an implementation of a dlopen-like API for Windows
  • "FlexDLL implements mostly the usual dlopen POSIX API, without trying to be fully conformant though (e.g. it does not respect the official priority ordering for symbol resolution). This should make it easy to port applications developped for Unix."
• LoadLibrary: https://msdn.microsoft.com/en-us/library/windows/desktop/ms684175(v=vs.85).aspx

And "dlsym equivalent windows":

• https://stackoverflow.com/questions/4184017/dlopen-dlsym-on-the-main-executable-how-portable-is-it
  • "Windows uses LoadLibrary() and GetProcAddress() instead."
• GetProcAddress: https://msdn.microsoft.com/en-us/library/windows/desktop/ms683212(v=vs.85).aspx
• https://www.symantec.com/connect/articles/dynamic-linking-linux-and-windows-part-two
  • "Both Linux and Windows provide routines (such as dlopen() and dlsym() in Linux, and LoadLibrary() ,GetProcAddress() in Windows)"
• https://www.experts-exchange.com/questions/22762973/Want-to-create-a-wrapper-class-for-dlopen-dlsym-dlclose-to-work-on-both-UNIX-and-windows-platform.html
  • "On windows, I believe LoadLibrary() == dlopen(), GetProcAddress() == dlsym(), and FreeLibrary() == dlclose()"

So from that, it sounds like we have the following premise to work from:

• dlopen in *nix roughly maps to LoadLibrary in windows
• dlsym in *nix roughly maps to GetProcAddress in windows

The main definitions in FlexDLL don't look too complex.. but there is quite a bit of extra code around those that may be required too:

• https://github.com/alainfrisch/flexdll/blob/master/flexdll.c#L83-L117

Hopefully this helps scope out what will be required/start pointing in the right direction :)

[jclc]

All the posts seem to be concerned with the loading of symbols, but does the compiler support producing a plugin (presumably DLL) on Windows?

[alexbrainman]

does the compiler support producing a plugin (presumably DLL) on Windows?

It is possible to build Windows DLL from your Go code. You want -buildmode=c-shared 'go build' flag for that. See #26714 for an example. 'go build' command uses gcc under covers to build DLL.

Alex

[jclc]

I've been hacking on this issue for a while and it seems to be going well for now. I've managed to load a dll built with -buildmode=c-shared and call its init function. The only limitation of this is that the plugin needs to have a main function or it won't compile. I'm developing on Linux using GCC and Wine. Just a few questions if anyone could clarify:

What exactly is going on in this function? The dlopen implementation calls this function and apparently returns the symbols; it doesn't work with Windows's shared objects.
https://github.com/golang/go/blob/master/src/runtime/plugin.go#L10

Secondly, I couldn't find any consistent guidelines for using wide strings with CGO so I ended up depending on unicode/utf16 and unicode/utf8. However, go/build/deps_test.go has pretty strict restrictions on which packages you can import. Is this a problem?

Edit: I guess this isn't so straightforward as I thought. -buildmode=plugin adds some metadata that is needed to find the exported symbols. Reading the PE data (using pev's readpe) doesn't show any of the functions that the plugin is meant to export, only init and main. When loading it, the init function is run implicitly.

[Indribell]

This seems invaluable, to me. I work on Windows, primarily. Is this difficult to implement, or is it just not a priority?

Its not a priority seems to be the right answer. There are issues with Windows/Mac support. There is the issue with gopath. The issue with different go versions and they all date back to 2017.

The plugin feature is mostly a tech demo that for some unholy reason got released as a stable feature of the language. When in reality it needed to have been hidden until it was ready. Its not ready and definably not useable as a feature for clients ( we have burned our hands on that also! ). Imagine wanting to offer plugin support for your program using go... They run into the platform issues. The different compile version issues. The gopath issues...

For a language that pride itself in delaying for years, when it comes down to new features ( like generics ) because it needs to be done correctly. And then seeing Plugins in this state 4 years after its release is at best mindboggling.

If you want plugins, go the way slower grpc route, as their are plugins ( more stand alone solution that communicate over grpc ) that work. But at the cost of a 40 a 50 times performance hit, so never run hot code using those.

[edwingeng]

FYI: I made a solution, https://github.com/edwingeng/hotswap, for hot reloading based on the plugin mechanism, with which you can develop and debug a plugin on Windows, and then run the plugin on Linux without changing any code.

P.S. Don't miss the --staticLinking argument.

[zandercodes]

Hey guys, I have tried to implement plugin for windows. Unfortunately the dynlink in the ASM for windows does not work. R15 can not be used as dynlink because this is a global variable.

Since I am not familiar with ASM it is hard to implement. Or can someone help?

sys_windows_amd64.s:219: when dynamic linking, R15 is clobbered by a global variable access and is used here: 00171

[scottctaylor12]

@zandercodes I'm not familiar with ASM either, but I would also like to implement plugins for windows. Is there a branch or fork that I can review to see what you've done so far?

[Linkangyis]

https://github.com/Linkangyis/GoLangLinkDLL
我认为这个方案应该可行，只是没办法实现跟plugin一样的共享包使用共享内存空间的效果

[godcong]

The current Plugin model has the following disadvantages

1. now the Plugin can not do dynamic loading and destruction.
2. only support Linux system, can not be used in other systems.

I think to do this, the following ideas can be considered.

1. when compiling, you need to export the interface to call and compile the file xxx.plugin and xxx_plugin.go, one is a binary file, a go file
2. In addition, the compilation can be with the system suffix, because the binaries of different systems are different. like xxx.plugin.x86.linux
3. if the method needs to be called by Plugin, you need to implement the bridge method in the plugin.go file. Usually these should be generated automatically
4. the automatically generated code may be more complex, you can specify some specifications. For example, the exported code must be commented with //plugin.
5. loading can be done locally or from a repository if possible, and automatically device your own system configuration. plugin.Load(/mnt/d/xxx.plugin) or plugin.Load(github/golang/go/plugin/xxx.plugin) like this.
   If there is no corresponding system implementation, it can return not found.

If you can follow the above method to achieve. Then go will have its own plug-in system. Instead of relying on c or windows dll or something.

[eh-steve]

People might be interested in my (still janky) fork of pkujhd's goloader (tested on windows/amd64 as well as linux/amd64, linux/arm64, darwin/amd64, darwin/arm64) and how it differs from plugin.

It doesn't depend on libdl doing the legwork of symbol deduplication - it directly reads Go archive files and loads code and applies relocations.

It doesn't necessarily require -dynlink because it controls the address of the mmap'd segment to ensure PCREL relocs are within 32-bits of the first module's symbols.

[lvyonghuan]

I'm currently working on an embedded project that requires strong versatility. It has the potential to run on Linux or Windows. The fact that the plugin package only works on Linux is causing me a lot of headaches.

[cchamplin]

@lvyonghuan I think the best path forward here at the moment is to leverage compilation to wasm via either tinygo or go with wasip1, and then load the the wasm 'plugin' in with something like https://github.com/tetratelabs/wazero. You can get pretty far with this approach, it checks a lot of boxes for dynamically loading and executing code.

[RichiCoder1]

Yah, it's not as clean as the go plugin interface, but https://extism.org/ is a great plugin option at this point.

[lvyonghuan]

@lvyonghuan I think the best path forward here at the moment is to leverage compilation to wasm via either tinygo or go with wasip1, and then load the the wasm 'plugin' in with something like https://github.com/tetratelabs/wazero. You can get pretty far with this approach, it checks a lot of boxes for dynamically loading and executing code.

I don't have much knowledge about WebAssembly. Does it typically require a browser environment to function properly? In our environment, there may not be a browser.

[lesomnus]

@lvyonghuan Does it typically require a browser environment to function properly? In our environment, there may not be a browser.

You probably want to look out something like WASM runtime -> https://github.com/appcypher/awesome-wasm-runtimes.
Indeed WASM was developed target to browsers but you can run WASM outside of the browser (with proper runtime).
Anyway, it doesn't seem appropriate to discuss that issue (about WASM) further in this thread.

[Linkangyis]

I'm currently working on an embedded project that requires strong versatility. It has the potential to run on Linux or Windows. The fact that the plugin package only works on Linux is causing me a lot of headaches.

给你提供一个思路

go->c->go 能跑起来，c做桥，但是在windows下，dll和主程序是几乎完全不同的内存空间，撑死加载一下func，引入相同的包实际内存地址也是完全不一样的

[Linkangyis]

I'm currently working on an embedded project that requires strong versatility. It has the potential to run on Linux or Windows. The fact that the plugin package only works on Linux is causing me a lot of headaches.

https://github.com/Linkangyis/GoLangLinkDLL

你可以尝试参考这个实现