plugin: add Windows support

[QuestionPython]

hi
Plugin Pkg Work for Windows!?
i want use this for mac,linux,win,... os.
when(what time) fix this?

https://golang.org/pkg/plugin/

[bradfitz]

There is currently nobody working on it, as far as I know.

/cc @alexbrainman @crawshaw

[QuestionPython]

mean in go 1.8 , plugin pkg work for apple-mac,windows and more ?

[bradfitz]

@QuestionPython, yes, it's even documented in multiple places:

https://golang.org/pkg/plugin/

Currently plugins only work on Linux.

https://golang.org/doc/go1.8#plugin

Plugin support is currently only available on Linux.

[alexbrainman]

There is currently nobody working on it, as far as I know.

I am not working on it. Sorry.

Alex

[bradfitz]

We delete all "me too" voting comments per https://golang.org/wiki/NoMeToo. Vote with emoji reactions at top instead.

[bradfitz]

The "me too" comments wouldn't stop, so this issue is now locked. If there are updates, they will be made here.

[0xdevalias]

Out of curiosity, do we know how much effort it would take to implement windows support? Or if there are any blockers to it (and what they are?)

[akavel]

Notably, the recently published Go kernel for Jupyter notebooks is using buildmode=shared, and thus doesn't currently support Windows natively. This is a very cool use case, adding a REPL-like live coding feature to the Go ecosystem, thus it would be really awesome if someone tried to start work on buildmode=shared on Windows to support this use case.

Similar to @0xdevalias , I'm quite interested in some hints as to what is missing for this to work on Windows? I'm especially curious what extra work is needed given that c-shared is already implemented on Windows?

[alexbrainman]

@0xdevalias and @akavel I don't have any effort estimation or any hints as to what missing here. I have not actually looked at what is involved. I am so much behind at fixing old issues ...

Alex

[akavel]

@alexbrainman Thanks! I'll ask on golang-dev then, maybe someone else can shed some light (edit: link to the thread)

[0xdevalias]

There doesn't seem to be a huge amount to it in the src: https://github.com/golang/go/tree/master/src/plugin

My completely naive guess would be figuring the windows equivalents to the C-bindings in plugin_dlopen.go.

The main functions I can see there are:

• https://linux.die.net/man/3/dlopen : The function dlopen() loads the dynamic library file named by the null-terminated string filename and returns an opaque "handle" for the dynamic library.
• https://linux.die.net/man/3/dlsym : The function dlsym() takes a "handle" of a dynamic library returned by dlopen() and the null-terminated symbol name, returning the address where that symbol is loaded into memory.

Googling for "dlopen equivalent windows" led me to the following:

• https://github.com/alainfrisch/flexdll : an implementation of a dlopen-like API for Windows
  • "FlexDLL implements mostly the usual dlopen POSIX API, without trying to be fully conformant though (e.g. it does not respect the official priority ordering for symbol resolution). This should make it easy to port applications developped for Unix."
• LoadLibrary: https://msdn.microsoft.com/en-us/library/windows/desktop/ms684175(v=vs.85).aspx

And "dlsym equivalent windows":

• https://stackoverflow.com/questions/4184017/dlopen-dlsym-on-the-main-executable-how-portable-is-it
  • "Windows uses LoadLibrary() and GetProcAddress() instead."
• GetProcAddress: https://msdn.microsoft.com/en-us/library/windows/desktop/ms683212(v=vs.85).aspx
• https://www.symantec.com/connect/articles/dynamic-linking-linux-and-windows-part-two
  • "Both Linux and Windows provide routines (such as dlopen() and dlsym() in Linux, and LoadLibrary() ,GetProcAddress() in Windows)"
• https://www.experts-exchange.com/questions/22762973/Want-to-create-a-wrapper-class-for-dlopen-dlsym-dlclose-to-work-on-both-UNIX-and-windows-platform.html
  • "On windows, I believe LoadLibrary() == dlopen(), GetProcAddress() == dlsym(), and FreeLibrary() == dlclose()"

So from that, it sounds like we have the following premise to work from:

• dlopen in *nix roughly maps to LoadLibrary in windows
• dlsym in *nix roughly maps to GetProcAddress in windows

The main definitions in FlexDLL don't look too complex.. but there is quite a bit of extra code around those that may be required too:

• https://github.com/alainfrisch/flexdll/blob/master/flexdll.c#L83-L117

Hopefully this helps scope out what will be required/start pointing in the right direction :)

[jclc]

All the posts seem to be concerned with the loading of symbols, but does the compiler support producing a plugin (presumably DLL) on Windows?

[alexbrainman]

does the compiler support producing a plugin (presumably DLL) on Windows?

It is possible to build Windows DLL from your Go code. You want -buildmode=c-shared 'go build' flag for that. See #26714 for an example. 'go build' command uses gcc under covers to build DLL.

Alex

[jclc]

I've been hacking on this issue for a while and it seems to be going well for now. I've managed to load a dll built with -buildmode=c-shared and call its init function. The only limitation of this is that the plugin needs to have a main function or it won't compile. I'm developing on Linux using GCC and Wine. Just a few questions if anyone could clarify:

What exactly is going on in this function? The dlopen implementation calls this function and apparently returns the symbols; it doesn't work with Windows's shared objects.
https://github.com/golang/go/blob/master/src/runtime/plugin.go#L10

Secondly, I couldn't find any consistent guidelines for using wide strings with CGO so I ended up depending on unicode/utf16 and unicode/utf8. However, go/build/deps_test.go has pretty strict restrictions on which packages you can import. Is this a problem?

Edit: I guess this isn't so straightforward as I thought. -buildmode=plugin adds some metadata that is needed to find the exported symbols. Reading the PE data (using pev's readpe) doesn't show any of the functions that the plugin is meant to export, only init and main. When loading it, the init function is run implicitly.

[r3inbowari]

Maybe someone who needs to promote this plan...

[davecheney]

Thank you to all the commenters. Can I remind everyone of the No +1 policy. It's clear that there is a desire for pluggin support for windows, there is no doubt about that, but at the moment nobody from the Go team or the community has stepped up to do the work to implement this feature.

[goapifirst]

@davecheney Understood.. however I am a little surprised that this is something the Go team put in a release.. half baked. I would have thought by now, multiple releases later that it would have been on a list somewhere to get completed and not left hanging given it was implemented already for one platform then fixed for a 2nd. I mean if it was a 3rd party addon/lib/etc and built this way, sure.. but that it is part of the official golang distribution.. but not complete.. seems like it should bubble up to the top on things to finish. I dont know if there are other features like this in the language/core that are half baked and only work on one or two platforms, but Golang is largely understood to produce binaries on all three platforms, so to leave out support for one on a feature that, at least to me is actually quite a big deal.. but is receiving less use because it's not complete, just seems out of ordinary for an otherwise pristine piece of kit. Again I'll say.. if this was not part of the core language and already in a release for a while now, I think many could understand. I have to believe someone among the many of you amazing developers could spend a little time on this and figure it out.

[ianlancetaylor]

It is really indisputable that plugins are a half baked feature. They have significant problems even on platforms where they (mostly) work. It was perhaps a mistake to add them at all. I think I approved the change to add plugins myself; I'm sorry for the trouble they are causing.

[goapifirst]

@ianlancetaylor I am going to assume it would be difficult at best to remove the feature at this point? Maybe Golang 2.0 can remove that feature.. or do it right. I am a nut for "plugins" as they do have a lot of value.. being able to load code dynamically as needed.. and more so in Golang if it worked the same across all platforms, would be amazing. There is clearly ways for this to work as languages like C can load .dll and .so dynamically on all platforms. I am going to assume then that it is more a matter of not having the right expertise within the team or anyone for that matter to not only get it working, but in a way that it is consistent across all platforms.
I will say though that today, with things mostly done via PWA/web based apps, I can see why it may not have much significance in Go where it is not typically used for desktop based apps where the notion of plugins is more common (e.g. audio plugins, editing plugins in video editors, game addons, etc). The RPC style plugin engine seems usable in most situations that don't need extreme performance like real time audio processing, etc.
Still.. if it can't be removed.. would be great to see it resolved. :)

[ianlancetaylor]

Unfortunately it's more subtle than just generating a DLL. Given the way that Go works, the plugin and the main executable will inevitably both import some of the same packages. Those packages will then exist twice as code, but in the general case they must share the same data variables. This is straightforward in ELF, where a symbol in the main executable will override and replace a symbol in a shared library. This is not straightforward in the PE/COFF format used on Windows. I expect that there is some way to do it, but there is no obvious simple way.

[marcogrecopriolo]

Here's my stab at it: https://github.com/marcogrecopriolo/plugin
Little snag - windows doesn't support buildmode plugin or shared.
I've tried testing using a c-shared built package, but the the next snag I hit is that windows doesn't seem to keep track of dynamic module data, so we panic in lastmoduleinit().
I guess somebody will have to enable plugin builds for windows?

[g13013]

@ianlancetaylor for what it worth, Go plugins are useful for us!

[tuhao1020]

Any progress on this issue?

[ambition0802]

Any progress on this issue~~~

[mdempsky]

You can find progress updates on this here.

[eaglexiang]

You can find progress updates on this here.

@mdempsky you linked this page itself

[mdempsky]

Sorry, my joke seems to have fallen flat, so I'll give a direct answer instead.

I linked to this page, because this page is where progress updates would be posted. There are no recent updates here because there has been no recent progress on this issue.

[naikrovek]

This seems invaluable, to me. I work on Windows, primarily. Is this difficult to implement, or is it just not a priority?

[ianlancetaylor]

It is difficult to implement.

[Indribell]

This seems invaluable, to me. I work on Windows, primarily. Is this difficult to implement, or is it just not a priority?

Its not a priority seems to be the right answer. There are issues with Windows/Mac support. There is the issue with gopath. The issue with different go versions and they all date back to 2017.

The plugin feature is mostly a tech demo that for some unholy reason got released as a stable feature of the language. When in reality it needed to have been hidden until it was ready. Its not ready and definably not useable as a feature for clients ( we have burned our hands on that also! ). Imagine wanting to offer plugin support for your program using go... They run into the platform issues. The different compile version issues. The gopath issues...

For a language that pride itself in delaying for years, when it comes down to new features ( like generics ) because it needs to be done correctly. And then seeing Plugins in this state 4 years after its release is at best mindboggling.

If you want plugins, go the way slower grpc route, as their are plugins ( more stand alone solution that communicate over grpc ) that work. But at the cost of a 40 a 50 times performance hit, so never run hot code using those.

[edwingeng]

FYI: I made a solution, https://github.com/edwingeng/hotswap, for hot reloading based on the plugin mechanism, with which you can develop and debug a plugin on Windows, and then run the plugin on Linux without changing any code.

P.S. Don't miss the --staticLinking argument.