Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unify support for dlopen/LoadLibrary/.. #38825

Open
ThomasKrenn opened this issue May 3, 2020 · 7 comments
Open

Unify support for dlopen/LoadLibrary/.. #38825

ThomasKrenn opened this issue May 3, 2020 · 7 comments

Comments

@ThomasKrenn
Copy link

@ThomasKrenn ThomasKrenn commented May 3, 2020

It would be nice to finally have some cross platform support
for loading shared objects, without the need for cgo (and complex tool chains).

The syscall package has support for
LoadLibrary and
GetProcAddress

and the plugin package has support for
dlopen and
dlsym

Why there is still no package with cross
platform suppport?

I have a shared object with 150 entry points
compiled with _cdecl / x64 and I can load and use it
cross platform in LUA and COBOL but
not with GO (without a complex cgo toolchain).

Best regards

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented May 4, 2020

It's not a simple problem. What language is your shared library written in? If it's written in C, or any language that uses a C-compatible API, then you will have to call any shared library functions via cgo. The plugin package only works with shared libraries that are written in Go. Methods like syscall.(*Proc).Call only work to call Windows DLL functions; calling shared library functions on, say, Unix, is somewhat more complex.

Go is an open source project. If you want to propose a approach to this general problem, I encourage you to do so. There are many subtleties. You may want to look at https://golang.org/s/execmodes to get a start on the problem space. Thanks.

@ThomasKrenn
Copy link
Author

@ThomasKrenn ThomasKrenn commented May 4, 2020

Hello Ian,
thanks for your quick response.
I wrote the first comment after I played with
-buildmode=c-shared

Out of the box I got a working .so (or .dll on Windows).
The amazing part is that is has nearly zero dependencies
(on Windows only kernel32.dll and msvcrt.dll).
I tried all kinds of parameters (e.g. (uintptr, *uintptr, *C.char))
and got everthing working (A C executable calling Go code in a .so/.dll).
Really great work!

-- What language is your shared library written in?
C (x64 with __cdecl)

What is confusing to me is that I cannot write a
Go executable and load a .so (Linux) .dll (Win)
(The opposite of what I described above.)

Yes there could be struct alignment problems and on Win
someone might try to load a dll with __stcall functions.
But for a start Go could support __cdecl x64.

In Java one can use JNA:
https://java-native-access.github.io/jna/4.2.1/

Ruby has a ffi interface:
https://github.com/ffi/ffi/wiki/Examples

(Lua was a bad example, because it currently does not
implement dlopen)

Maybe I miss something, but the other discussions I have
seen didn't really help me understand the issue.

Best regards
Thomas

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented May 4, 2020

Go also has a FFI interface: cgo.

It seems to me that you are suggesting that Go have a way to call into shared libraries written in C without using cgo. That won't work. As I said above, if the shared library is written in C, or any language that uses a C-compatible API, then you will have to call any shared library functions via cgo.

@ThomasKrenn
Copy link
Author

@ThomasKrenn ThomasKrenn commented May 4, 2020

-- That won't work.
It works already using:
syscall.LoadLibrary
.NewProc
unfortunately on Windows only.
That's why I wrote the request.

-- Go also has a FFI interface: cgo.
Yes, but it works more like the old Java JNI, because it requires a complex toolchain and C know how.

From the JNA Website:
JNA provides Java programs easy access to native shared libraries without writing anything but Java code - no JNI or native code is required. This functionality is comparable to Windows' Platform/Invoke and Python's ctypes.

Three major languages have the feature, but it seems that Go got stuck in the C world.
(BTW even Cobol (Microfocus and GNUcobol) have the feature)

@ThomasKrenn ThomasKrenn closed this May 4, 2020
@ThomasKrenn ThomasKrenn reopened this May 4, 2020
@networkimprov
Copy link

@networkimprov networkimprov commented May 4, 2020

I think what you're asking is for Go to implicitly wrap a member of a shared library when it's requested, which is what cgo does at compile time. (Note that Java & Ruby are not compiled languages.) That might not be too hard if the function signature only has primitive types and no pointers. Structs and pointers would complicate matters.

There haven't been many requests for this (I don't think), so making it happen probably requires a volunteer with relevant experience.

@ThomasKrenn
Copy link
Author

@ThomasKrenn ThomasKrenn commented May 4, 2020

Java is a compiled language and has/had to solve the same technical issues.
Pointers uintptr, *uintptr should work out of the box. x64 has 8 byte per pointer an all platforms.
__cdecl is standard for x64 (even on Windows). No fancy name mangling.

The funny thing is that the plugin interface
https://golang.org/src/plugin/
supports
dlopen but not win/loadlibary
and syscall supports
win/Loadlibrary but not dlopen. (?!)
I think the volunteers/developers just need the right approach.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented May 4, 2020

Fair enough, the Windows code hides some of the cgo approach from the user. It still uses the cgo mechanisms underneath, and it provides a limited list of types, so it has limited flexibility.

Yes, using cgo requires a limited amount of C knowledge. I personally don't consider that to be a problem. If you are calling C code from Go code you need to understand C memory management. I don't see cgo as comparable to JNI. Using JNI requires writing different kinds of code in C. That is not true of cgo. You can use cgo exactly like the Windows syscall API by writing only C #include statements, nothing else.

If someone wants to make a detailed proposal for how to simplify cgo, perhaps to work likely the syscall WIndows API, go ahead. We can consider that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.