plugin: add Windows support

[QuestionPython]

hi
Plugin Pkg Work for Windows!?
i want use this for mac,linux,win,... os.
when(what time) fix this?

https://golang.org/pkg/plugin/

[bradfitz]

There is currently nobody working on it, as far as I know.

/cc @alexbrainman @crawshaw

[QuestionPython]

mean in go 1.8 , plugin pkg work for apple-mac,windows and more ?

[bradfitz]

@QuestionPython, yes, it's even documented in multiple places:

https://golang.org/pkg/plugin/

Currently plugins only work on Linux.

https://golang.org/doc/go1.8#plugin

Plugin support is currently only available on Linux.

[alexbrainman]

There is currently nobody working on it, as far as I know.

I am not working on it. Sorry.

Alex

[bradfitz]

We delete all "me too" voting comments per https://golang.org/wiki/NoMeToo. Vote with emoji reactions at top instead.

[bradfitz]

The "me too" comments wouldn't stop, so this issue is now locked. If there are updates, they will be made here.

[0xdevalias]

Out of curiosity, do we know how much effort it would take to implement windows support? Or if there are any blockers to it (and what they are?)

[akavel]

Notably, the recently published Go kernel for Jupyter notebooks is using buildmode=shared, and thus doesn't currently support Windows natively. This is a very cool use case, adding a REPL-like live coding feature to the Go ecosystem, thus it would be really awesome if someone tried to start work on buildmode=shared on Windows to support this use case.

Similar to @0xdevalias , I'm quite interested in some hints as to what is missing for this to work on Windows? I'm especially curious what extra work is needed given that c-shared is already implemented on Windows?

[alexbrainman]

@0xdevalias and @akavel I don't have any effort estimation or any hints as to what missing here. I have not actually looked at what is involved. I am so much behind at fixing old issues ...

Alex

[akavel]

@alexbrainman Thanks! I'll ask on golang-dev then, maybe someone else can shed some light (edit: link to the thread)

[0xdevalias]

There doesn't seem to be a huge amount to it in the src: https://github.com/golang/go/tree/master/src/plugin

My completely naive guess would be figuring the windows equivalents to the C-bindings in plugin_dlopen.go.

The main functions I can see there are:

• https://linux.die.net/man/3/dlopen : The function dlopen() loads the dynamic library file named by the null-terminated string filename and returns an opaque "handle" for the dynamic library.
• https://linux.die.net/man/3/dlsym : The function dlsym() takes a "handle" of a dynamic library returned by dlopen() and the null-terminated symbol name, returning the address where that symbol is loaded into memory.

Googling for "dlopen equivalent windows" led me to the following:

• https://github.com/alainfrisch/flexdll : an implementation of a dlopen-like API for Windows
  • "FlexDLL implements mostly the usual dlopen POSIX API, without trying to be fully conformant though (e.g. it does not respect the official priority ordering for symbol resolution). This should make it easy to port applications developped for Unix."
• LoadLibrary: https://msdn.microsoft.com/en-us/library/windows/desktop/ms684175(v=vs.85).aspx

And "dlsym equivalent windows":

• https://stackoverflow.com/questions/4184017/dlopen-dlsym-on-the-main-executable-how-portable-is-it
  • "Windows uses LoadLibrary() and GetProcAddress() instead."
• GetProcAddress: https://msdn.microsoft.com/en-us/library/windows/desktop/ms683212(v=vs.85).aspx
• https://www.symantec.com/connect/articles/dynamic-linking-linux-and-windows-part-two
  • "Both Linux and Windows provide routines (such as dlopen() and dlsym() in Linux, and LoadLibrary() ,GetProcAddress() in Windows)"
• https://www.experts-exchange.com/questions/22762973/Want-to-create-a-wrapper-class-for-dlopen-dlsym-dlclose-to-work-on-both-UNIX-and-windows-platform.html
  • "On windows, I believe LoadLibrary() == dlopen(), GetProcAddress() == dlsym(), and FreeLibrary() == dlclose()"

So from that, it sounds like we have the following premise to work from:

• dlopen in *nix roughly maps to LoadLibrary in windows
• dlsym in *nix roughly maps to GetProcAddress in windows

The main definitions in FlexDLL don't look too complex.. but there is quite a bit of extra code around those that may be required too:

• https://github.com/alainfrisch/flexdll/blob/master/flexdll.c#L83-L117

Hopefully this helps scope out what will be required/start pointing in the right direction :)

[jclc]

All the posts seem to be concerned with the loading of symbols, but does the compiler support producing a plugin (presumably DLL) on Windows?

[alexbrainman]

does the compiler support producing a plugin (presumably DLL) on Windows?

It is possible to build Windows DLL from your Go code. You want -buildmode=c-shared 'go build' flag for that. See #26714 for an example. 'go build' command uses gcc under covers to build DLL.

Alex

[jclc]

I've been hacking on this issue for a while and it seems to be going well for now. I've managed to load a dll built with -buildmode=c-shared and call its init function. The only limitation of this is that the plugin needs to have a main function or it won't compile. I'm developing on Linux using GCC and Wine. Just a few questions if anyone could clarify:

What exactly is going on in this function? The dlopen implementation calls this function and apparently returns the symbols; it doesn't work with Windows's shared objects.
https://github.com/golang/go/blob/master/src/runtime/plugin.go#L10

Secondly, I couldn't find any consistent guidelines for using wide strings with CGO so I ended up depending on unicode/utf16 and unicode/utf8. However, go/build/deps_test.go has pretty strict restrictions on which packages you can import. Is this a problem?

Edit: I guess this isn't so straightforward as I thought. -buildmode=plugin adds some metadata that is needed to find the exported symbols. Reading the PE data (using pev's readpe) doesn't show any of the functions that the plugin is meant to export, only init and main. When loading it, the init function is run implicitly.

[marcogrecopriolo]

from https://github.com/marcogrecopriolo/sqsl/blob/master/oslib/coslc.c

/*
** dlopen
*/
DLLDECL void *fgw_dlopen(char *f)
{
return (void *) LoadLibrary(f);
}

/*
** dlsym
*/
DLLDECL void *fgw_dlsym(void *h, char *s)
{
return (void *) GetProcAddress(h, s);
}

/*
** dlclose
*/
DLLDECL void *fgw_dlclose(void *h)
{
FreeLibrary(h);
}

static char errbuf[20];
/*
** dlerror
*/
DLLDECL char *fgw_dlerror()
{
int rc;

rc=GetLastError();
sprintf((char *) &errbuf, "%d", rc);
return (char *) &errbuf;

}

This is really all there is to it. Could we please have it now?
Also, could we have func(p *Plugin) Close()?

These two would make golang user defined functions inside couchbase's N1QL very easy to implement.

[xtxy]

any plan for this feature?

[goapifirst]

Seems as Go is one of the hot languages and growing fast now, it would benefit to support plugins on all three platforms and hopefully in a way that doesn't require the main app to be compiled using the same version of go. That is a lesser evil, but would be nice if say an app compiled in go 1.14 and load a plugin compiled in 1.17 and not crash, and vice versa. Here is to hoping someone will add this feature for Windows and see some examples of building a simple app that uses a plugin that is also built on all 3 platforms.

[r3inbowari]

Maybe someone who needs to promote this plan...

[davecheney]

Thank you to all the commenters. Can I remind everyone of the No +1 policy. It's clear that there is a desire for pluggin support for windows, there is no doubt about that, but at the moment nobody from the Go team or the community has stepped up to do the work to implement this feature.

[goapifirst]

@davecheney Understood.. however I am a little surprised that this is something the Go team put in a release.. half baked. I would have thought by now, multiple releases later that it would have been on a list somewhere to get completed and not left hanging given it was implemented already for one platform then fixed for a 2nd. I mean if it was a 3rd party addon/lib/etc and built this way, sure.. but that it is part of the official golang distribution.. but not complete.. seems like it should bubble up to the top on things to finish. I dont know if there are other features like this in the language/core that are half baked and only work on one or two platforms, but Golang is largely understood to produce binaries on all three platforms, so to leave out support for one on a feature that, at least to me is actually quite a big deal.. but is receiving less use because it's not complete, just seems out of ordinary for an otherwise pristine piece of kit. Again I'll say.. if this was not part of the core language and already in a release for a while now, I think many could understand. I have to believe someone among the many of you amazing developers could spend a little time on this and figure it out.

[ianlancetaylor]

It is really indisputable that plugins are a half baked feature. They have significant problems even on platforms where they (mostly) work. It was perhaps a mistake to add them at all. I think I approved the change to add plugins myself; I'm sorry for the trouble they are causing.

[goapifirst]

@ianlancetaylor I am going to assume it would be difficult at best to remove the feature at this point? Maybe Golang 2.0 can remove that feature.. or do it right. I am a nut for "plugins" as they do have a lot of value.. being able to load code dynamically as needed.. and more so in Golang if it worked the same across all platforms, would be amazing. There is clearly ways for this to work as languages like C can load .dll and .so dynamically on all platforms. I am going to assume then that it is more a matter of not having the right expertise within the team or anyone for that matter to not only get it working, but in a way that it is consistent across all platforms.
I will say though that today, with things mostly done via PWA/web based apps, I can see why it may not have much significance in Go where it is not typically used for desktop based apps where the notion of plugins is more common (e.g. audio plugins, editing plugins in video editors, game addons, etc). The RPC style plugin engine seems usable in most situations that don't need extreme performance like real time audio processing, etc.
Still.. if it can't be removed.. would be great to see it resolved. :)

[ianlancetaylor]

Unfortunately it's more subtle than just generating a DLL. Given the way that Go works, the plugin and the main executable will inevitably both import some of the same packages. Those packages will then exist twice as code, but in the general case they must share the same data variables. This is straightforward in ELF, where a symbol in the main executable will override and replace a symbol in a shared library. This is not straightforward in the PE/COFF format used on Windows. I expect that there is some way to do it, but there is no obvious simple way.

[marcogrecopriolo]

Here's my stab at it: https://github.com/marcogrecopriolo/plugin
Little snag - windows doesn't support buildmode plugin or shared.
I've tried testing using a c-shared built package, but the the next snag I hit is that windows doesn't seem to keep track of dynamic module data, so we panic in lastmoduleinit().
I guess somebody will have to enable plugin builds for windows?

[g13013]

@ianlancetaylor for what it worth, Go plugins are useful for us!

[tuhao1020]

Any progress on this issue?