Unable to load plugin OFS lib

[esindril]

While trying to load the OFS plugin for the fst nodes in EOS, I get the following error:

++++++ File system initialization started.
141103 09:46:41 8806 FstOfs_Finder: Network i/f undefined; unable to self-locate.

This comes from the fact that my OFS Configure method is called with a 0 EnvInfo parameter since when loading the library in XrdXrootdConfig.cc:294 the function is called like this:
osFS = XrdXrootdloadFileSystem(&eDest, osFS, FSLib[1], pi->ConfigFN);

Therefore the EnvInfo is not passed but it's set later on (4 lines below) using: osFS->EnvInfo(&myEnv);

But when the call is done in XrdXrootdloadFileSystem, I also need to call the Configure on my OFS plugin and I don't have then EnvInfo object. And setting it later does not help me at this point, since I don't have another opportunity to call the ofs Configure method.

Therefore, I believe either the XrdSfsGetFileSystem signature should be modified to pass the EnvInfo object or you can first pass the EnvInfo object to XrdXrootdloadFileSystem and then after calling XrdSfsGetFileSystem set it for the new ofs object and also call the Configure method - so that I am not forced to call it myself in the XrdSfsGetFileSystem.

It's a bit twisted, but feel free to aske me further questions if it's not clear ...

Thanks,
Elvin

[abh3]

Hi Elvin,

I do understand your problem. What is it in EnvInfo() that you need that prevents you from initializing? Couldn't that part be deferred until EnvInfo() is actually called? As for the proposals, we could add the envinfo to the call XrdSfsGetFileSystem() but that would only work in R4.1 and if your plugin gets loaded by any earlier version you are guaranteed to crash. Not such a nice prospect.

The other proposed alternative (i.e. setting envinfo then calling Configure) is not workable because there is no such method in the Sfs Interface. It only exists in the built-in version.

The only safe way of doing this is to define XrdSfsGetFileSystem2() which has the new signature and if we find it we use it with the extra parameter. Otherwise, we use the old function call. I still have to look to see if that is even doable via the plugin manager.

[abh3]

Hi Elvin,

I looked and trying first XrdSfsGetFileSystem2() followed by XrdSfsGetFileSystem()it's potentially doable but would take some amount of work and make the code pretty ugly simply because plugins can be named versioned (i.e. libXrdxxx-4.so otherwise linXrdxxx.so) which complicates things.

So, I propose the following solution:

1. We find the right library by searching for XrdSfsGetFileSystem(). So, this entry point must exists (in your case it can simply return 0).
2. Once we establish the right library, we check if it also contains XrdSfsGetFileSystem2() and if it does, we will invoke that entry point with the extra parameter in lieu of XrdSfsGetFileSystem().

Would that be an acceptable solution?

Andy

[esindril]

Hi Andy,

Our OFS plugin in built on top of the default OFS one. Therefore, in our library I have implemented the entry point for the plugin which has the following signature:

XrdSfsFileSystem XrdSfsGetFileSystem (XrdSfsFileSystem* native_fs,
XrdSysLogger* lp,
const char* configfn)

which is called from XrdXrootdLoadLib.cc using the function XrdXrootdloadFileSystem which in turn is called from XrdXrootdConfig.cc:294. At this point in the file you can notice the difference on how the default OFS implementation and the plugin one are called (i.e the EnvInfo object at the end):

osFS = XrdSfsGetDefaultFileSystem(0,eDest.logger(),pi->ConfigFN,&myEnv);

Therefore in the entry point function XrdSfsGetFileSystem, I create my OFS plugin object and I also call its Configure methods which in turn calls the Configure for the default OFS object (this was our choice). But since I don't have the EnvInfo object which holds information about EnvInfo->GetPtr("XrdNetIF*"), it crashes in line XrdOfsConfig.cc:116.

The only difference comes from the fact that in the default OFS object you have access to the env variable holding information about the networking interface while in the plugin object I have but only after the initialisation and the configuration step ....

Yes, the solution sounds fine for me, I can implement both entry points and if the first one retrurns 0 then you try to call the second one.

Thanks,
Elvin

[abh3]

Ah, so before I change any code, please read the following description of the xrootd.fslib directive:

http://xrootd.org/doc/dev41/xrd_config.htm#_Toc399620860

Notice that it allows you to wrap your own ofs plugin around a fully configured one. So, if you say

xrootd.fslib /path/myOfsPlugin.so default

the default pluigin will be initialized and they your plugin will be loaded and given a pointed to the initialized one so you can intercept all the calls. This seems to be exactly what you are doing but your doing it using internal calls. Will using this supported scheme solve your problem?

[esindril]

Hi Andy,

This will not work for us as it would mean rewriting most of the ofs plugin and more over we only use a limited set of functionality from the default ofs - such as this part of calling the Configure method to parse the default configuration options.

But generally speaking, wouldn't it be desirable for someone who implements their own OFS standalone plugin to have access during the configuration step to the following components set in XrdXrootdConfig:

myEnv.PutPtr("XrdInet_", (void *)(pi->NetTCP));
myEnv.PutPtr("XrdNetIF_", (void )(&(pi->NetTCP->netIF)));
myEnv.PutPtr("XrdSecGetProtocol", (void )secGetProt);
myEnv.PutPtr("XrdScheduler", Sched);

Because the way the XrdSfsGetFileSystem entry point is currently defined this is not possible ... unless I'm missing something.

Another option would be to drop the check done in XrdOfsConfig.cc:139 since myIF seems not be used during the configure step of the default OFS and then set it when the EnvInfo() method of the OFS is called. But this might have some other side effects as the EnvInfo is 0 while initialising XrdConfigPI & others. I will have a closer look on Monday and try it out and let you know the outcome ...

Cheers,
Elvin

[abh3]

Hi Elvin,

Largely, the clases in the Env object are not public and should not be
used by plugin writes (never mind that they are and we have a special
category of private but made accessible classes). Do, given that you only
over-ride a small section of ofs classes; would it work if I gae you a
class whechi would look similar to the ofs class and could be used as a
wrapper to that class bt allow you easily over-ride the methods that you
want. It would basically be a class with all of the XrdSfs methods but
would invoke the buil-in XrdOfs method unless you over-ride it. From what
you say that would work for you and, in fact, you could even write one
like that.

Andy

On Sat, 15 Nov 2014, Elvin Sindrilaru wrote:

Hi Andy,

This will not work for us as it would mean rewriting most of the ofs plugin and more over we only use a limited set of functionality from the default ofs - such as this part of calling the Configure method to parse the default configuration options.

But generally speaking, wouldn't it be desirable for someone who implements their own OFS standalone plugin to have access during the configuration step to the following components set in XrdXrootdConfig:

myEnv.PutPtr("XrdInet_", (void *)(pi->NetTCP));
myEnv.PutPtr("XrdNetIF_", (void )(&(pi->NetTCP->netIF)));
myEnv.PutPtr("XrdSecGetProtocol", (void )secGetProt);
myEnv.PutPtr("XrdScheduler", Sched);

Because the way the XrdSfsGetFileSystem entry point is currently defined this is not possible ... unless I'm missing something.

Another option would be to drop the check done in XrdOfsConfig.cc:139 since myIF seems not be used during the configure step of the default OFS and then set it when the EnvInfo() method of the OFS is called. But this might have some other side effects as the EnvInfo is 0 while initialising XrdConfigPI & others. I will have a closer look on Monday and try it out and let you know the outcome ...

Cheers,
Elvin

---

Reply to this email directly or view it on GitHub:
#161 (comment)

[ljanyst]

I don't really see how this would help...

[esindril]

Hi Andy,

The above pull request completely solves my problem and I belive it also makes sense logiacally. First of all, given that you now expose the EnvInfo() function in the XrdSfs layer (XrdSfsInterface.hh:308) a user can decide to provide the env info for the OFS plugin using this method (after calling for example the configure method) therefore failing in the XrdOfsConfigure doing the check
if (!EnvInfo || !(myIF = (XrdNetIF )EnvInfo->GetPtr("XrdNetIF")))
is a bit of a too strong constrain that could be relaxed (as in this pull request) and could even be implemented in the EnvInfo() function since the myIF object is not used anywhere but in the XrdOfs::fsctl (in the OFS layer). This could also be an opportunity to make the code in XrdXrootdConfig a bit more symmetric when it comes to loading the default OFS or any other plugin OFS implementation.

Thanks,
Elvin

[abh3]

Hi Elvin,

The patch would cause as many problems as it solves. Should EnvInfo be zero then myIF becomes undefined. Unfortunately, it is used elsewhere with the implication via the existing test that if it would become undefined configuration would fail before myIF could ever be used. I'll look to see if that can be fixed, though I already have the GetFileSystem2() patch and could just as easily go with that. These are the kinds of issues always come up when private code is used -- there is no guarantee that we won't change it in a way that you can't easily use it.

[esindril]

Hi Andy,

Give me a few more days before doing any modifications as I'll try to adapt our code to work with the current interface. If this works then there is no need for GetFileSystem2 & others.

Thanks,
Elvin

[esindril]

Hi Andy,

I believe you can safely close this request as we can work around this issue on our side.

Thanks,
Elvin

[ljanyst]

Merci! :)