-
-
Notifications
You must be signed in to change notification settings - Fork 416
add high-level Library wrapper … #617
Conversation
And why does github sort commits in chronological instead of genealogical order? |
Shouldn't we name them
|
I find that annoyingly long, names where you always make a typo, and you have to write them much more often than loadLibrary. If it was only me, I'd name them
We cannot add all functionality so people need to be able to use the underlying handle when necessary.
That refers to the state of the super type. You can also test for equality with zero or use |
Yes, the handle must be gettable, but I disagree that it should be arbitrarily settable which it currently is. I don't think it should be implicitly gettable either, but that's the issue quoted below.
Well, I think subtyping
As AliasThis only deals with implicit conversions from the UDT, the initialization/assignment issue is more appropriately solved with a constructor and
Right, I guess it's alright not to mention it as long as |
As for this issue, I don't buy these arguments; I don't think the full names are particularly long, and common words like "function" and "variable" seem to me like they should be less likely to be misspelled if anything. However, I concede that it's hard to argue that the full names are inherently better on grounds other than the consistency-with-loadLibrary argument, so I think we'll have to agree to disagree. |
Seriously yes, I just tried it stopping at the first mistake, see for yourself.
It shouldn't be too surprising that short-term memorizing the movement to type a shorter word is easier and less error prone than for a long word. But usually I hit M-/ after 5-6 characters to hippie-expand the rest of the name. Looking at the alternatives I think it's a good compromise. auto lib = load("libfoo.so"); // though that too short to be imported
auto pf = lib.func!bar();
auto pv = lib.var!baz(); auto lib = loadLibrary("libfoo.so");
auto pf = lib.loadFunction!bar();
auto pv = lib.loadVariable!baz(); auto lib = loadLib("libfoo.so");
auto pf = lib.loadFunc!bar();
auto pv = lib.loadVar!baz(); |
Well, then I'd like to invoke the argument that code is first and foremost supposed to be readily readable and understandable, not readily writable. |
Sure, that's most important, but at the same time short names that convey the same amount of information ARE easier to read. |
Another idea. We could ditch the |
So you'd rather deprecate two symbols and be inconsistent with |
We have a chance at type safety here. Let's not push |
They were supposed to be deprecated anyhow, because adding a
Yes, these things are important because brief lines aid readability. Overlong names are a plaque as you can see by the terrible Java approach. In this case it doesn't matter so much, but I'd rather go with the short names if possible. |
Well, again let's agree to differ on which naming is better. How about |
👍 |
done |
|
||
auto var1 = lib.loadVar!(size_t)("core.runtime.TestStruct.var"); | ||
auto var2 = lib.loadVar!(size_t, "core.runtime.TestStruct.var")(); | ||
auto var3 = lib.loadVar!(TestStruct.var)(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
May be better another using of this function:
struct Foo
{
int* bar;
}
lib.load(Foo.bar); //set Foo.bar to pointer of `Foo.bar` symbol which mangled as a int variable?
The existing implementation I confused by the following fact:
Foo.bar variable need only to get name, type and linkage to the loadVar. We can not modify external Foo.bar through host Foo.bar.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is intended to be used with .di headers for the library so it operates on the actual declarations.
What you do with the resulting address is up to you.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it typical situation, when user need to load shared library dynamically and you have an .di interface?
Why user can't link this library at compile time?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@IgorStepanov Any time that you do plugins, you need dynamically load them at runtime. They might not even exist yet when the program is compiled. Dynamically loading a library like this is the real benefit of shared libraries. There are obviously benefits for programs that just link against everything, but it's still debatable as to whether statically linking or dynamic linking is better, whereas for stuff like plugins, dynamically linking a library with a particular interface is really the only way to do it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jmdavis Yes, but when you use plugin, you haven't .di interface? Or I'm wrong? And you can write many different plugins for one .di header? But what about ODR? If you declare class Foo in you header 'foo.di', you can have only one foo.Foo class definition in your program.
You can't declare first foo.Foo in plugin_1.so, second foo.Foo in plugin_2.so and load both plugins at a same time. Is not it? May be I do not fully understand the mechanism of shared libraries.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can't declare first foo.Foo in plugin_1.so, second foo.Foo in plugin_2.so and load both plugins at a same time. Is not it? May be I do not fully understand the mechanism of shared libraries.
Yes you can, it's the normal plugin use-case (see DConf Talk). This works because libraries aren't loaded into the global lookup scope (unless you specify RTLD_GLOBAL).
Recommended reading Drepper's DSO How to.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, it really opens up big opportunities. May be we also need to add function loadClass
, which returns ClassInfo for this class? In can be usefull when host system declare interface for the plugins and plugin class implements this interface. In this case we can load ClassInfo for plugin from library, call create()
, cast to interface and work with plugin through it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You'll be able to use the existing ModuleInfo.localClasses
metadata for that.
We will add a function to iterate over the modules of a loaded library.
But apparently Object.factory
wouldn't know how to disambiguate identical FQNs.
I like the new interface.
Should I wish we could make it an error or exception to use the We probably have to deal with Windows' reference counting on |
It's better.
Yes.
I'm a little undecided about this. From a composability standpoint I'd prefer to leave this as is, because simply testing the existence of a symbol is a valid use-case. I usually wrap all of these calls in
Do you really think there is any doubt about this? And there are 2-levels of reference counting on linux, btw (#593). |
Testing the existence of a symbol in a loaded library should be perfectly valid, but attempting to load a symbol from a previously unloaded library is also going to return null, while it seems to me to be more of a logic error (akin to accessing an object after it has been explicitly |
Are the two calls to |
I haven't touched this code, only moved it, and despite it's ugliness I wouldn't want to stuff more than necessary into this pull request. Feel invited to make a pull once this is merged :). |
Roger that. Otherwise this pull LGTM, and it's green. Are there any objections for pulling it?
If this is due to the ddoc'ed unittest bugs, apologies. I couldn't figure out a simpler way to implement them other than in the parser, since there's really no concept of "what is the previous visible symbol (w.r.t. version statements and static if)". |
I'm not too sure about the |
Yeah I also thought about this during the implementation but I forgot it. I used
Not really, though now the implementation structure is somewhat confusing.
I think the iterations over the API were quite useful, so let's wait until there is no more input. |
Somewhat OT: With regard to the DLL linking mess, there's also these: Issue 9220 (you've replied here) and Issue 6019. I've filed 6019 a long time ago though. |
{ | ||
auto lib = loadLib(null); // the executable | ||
scope (exit) lib.unloadLib(); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why doesn't Windows and Posix use the same API here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe I should remove this from the documentation, you can't use Windows's LoadLibrary with null to obtain a handle to the executable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't you use loadLib
and when null
is passed on Windows call GetModuleHandleA(null)
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this seems like a sensible approach, even though I'm always a bit wary of using special values like that.
How often is this expected to turn up in actual client code?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about rejecting null
for the current loadLib
and then providing a zero-argument overload that returns a reference to the current module?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about rejecting null for the current loadLib
Yep, let's add a separate function for dlopen(RTLD_SELF, ...)
later. The implementation is tricky, because which Library is returned depends on the caller, so this function could not live in druntime or we'd need to fake the return address.
No idea whether we can implement this semantic on Windows.
Maybe we could also come up with a better name than "loadLib no arguments".
I just have to say that I really don't like these abbreviated function names. I would hate to see D become like C in this regard where basically every symbol name is abbreviated. The 70's are over, the length of symbol names doesn't matter anymore. D already has too many abbreviated symbol names, we don't need to repeat the same mistake again. And no, just because we don't abbreviate symbols names doesn't mean it will have overly verbose names an in Java or Objective-C. |
- Move unittest to directly after version (D_Ddoc) followed by the implementations. - Dummy symbol for optlink's Bug 3956 needs to be placed before the unittest.
- no implicit conversion to make Library more typesafe - use Library in the new loadLib/unloadLib API - deprecate void* API Runtime.loadLibrary/Runtime.unloadLibray - add Library.opCast(T:bool) to test for initialized lib
- add handle !is null invariant to Library - remove opCast and disable default constructor
- Adapt the docs to emphasize that a search is performed and null is returned on failure.
The problem is that once it's in a release, it will be around for a long time, at least if it isn't accompanied with a big red warning saying otherwise. |
(edit: oops, sorry about not replying as a line comment; but then again, I've mentioned this like three times throughout) |
Right, we should come up with a good API here. |
So what should I do about the changelog? I could mention shared libraries being supported, but without a nice usable wrapper such as this I don't know if it's worth mentioning it. I have to give some use-cases on how to use the feature after all. |
Better not. The feature is still incomplete and experimental. |
Ok. Anyway the changelog was merged as soon as I made the pull, so if you decide you want to mention shared libs you can make a pull against dlang.org, no need to go through me. |
If we no longer use druntime's changelog.dd we should probably remove it. |
So what should we do about this?
I'd prefer either 1 or 3. |
Ping @JakobOvrum |
I have no idea what the best solution is :( It's an awkward spot because library handles are reference counted in both libdl (right?) and on Windows, but we can't get at that count with libdl. A non-copyable type is probably just fine for most use cases where the library is never unloaded (think libraries like Derelict), but when uses like plugins are involved, the One note about 4 is that the type system prevents non-null instances of |
@MartinNowak any decisions were made? |
I will continue to work on this soonish. |
ping pliz pliz |
This got delayed because the release building took much longer. |
ok thanks, please email me if blocked |
Ping @MartinNowak @andralex, status update please. |
ping |
I believe this is of critical importance to consider D having good support for shared libraries. |
@MartinNowak You seems a bit abstracted recent months. Are you ok? When does you return to us? :) |
@MartinNowak this seemed to go on the backburner... close until you can get back to it? |
@MartinNowak Is this branch still viable? This feature is pretty valuable. |
Yes, but I'm busy with other topics currently. |
@MartinNowak 4 years passed! Any plan to update this? |
The single most annoying pull request I ever prepared. Too many long-standing unrelated bugs for such a straightforward implementation.