Plugin configuration should be conformant

[abh3]

Currently, many plugin are configured by adding additional parameters after the plugin specification (e.g. xrootd.fslib []). This leads, many times, to inconsistent syntax and tortuously long lines (e.g. gsi) that are pretty opaque. There should be a standard way that plug-in writers can hook into the standard xrootd configuration file syntax mechanism.

[abh3]

So, I've looked at this in several angles and he only way to do this is for plugin writers to use the standard interface to get to read the configuration file and parse out what is of interest. I suppose we could make it slightly easier but it would reduce the options available. I'll open it up to suggestions of what plugin writers would like to have and after a little while, if I don't get any feedback, close this issue as one that is not particularly interesting.

[bbockelm]

This would be fantastic - the most problematic plugin one is the one mentioned in the ticket (GSI plugin). I would still be interested in exporting an alternate entry point that receives the configuration file (maintaining backward compatibility by still exporting the existing entry point).

[abh3]

So, the question I posed in another thread remains on the table. Is it acceptable to simply have a "blob parameter" creator. That is, if you don't specify the gobbledygook on the plugin line you can use he standard format (i.e. "gsi.xxx") and we will construct that blob from those parameters and feed them into the plugin. It's not ideal as any error messages will be delayed and won't necessarily be obviously tied to the parameter you may have specified. However, this is the fastest way of getting this done with the least amount of error.

[bbockelm]

This will help with the standard gsi.xxx options you mention - but what about the authz plugin parameters?

For example, what happens to authzfun and authzfunparms here:

sec.protocol /home/xrootd-build/release_dir/lib64 gsi \
  -certdir:/etc/grid-security/certificates \
  -cert:/etc/grid-security/hostcert.pem \
  -key:/etc/grid-security/hostkey.pem \
  -crl:1 \
  -authzfun:libXrdLcmaps.so \
  -authzfunparms:no-authz \
  -gmapopt:0 \
  -authzto:3600

Will plugin writers still need to squash all parameter setting into a single line without spaces?

[abh3]

That's just naturally falls out where gsi.authzfun gets converted to "-authzfun:". As for long lines, there is no need and, in fact, not allowed. Each "-:" specification would be entered on a separate line prefixed by "gsi." using the scheme . All of them would be blobbed and passed onward.

[abh3]

Ah, I see what you are saying. The authzfunparms itself might be an incomprehensible blob, much like vomsfunparms. This is more complicated simply because each of these plugins uses an arbitrary syntax, for no particularly good reason. The answer is we could given a template. So,I see that

-vomsfunparms:certfmt=pem|vos=atlas,cms,dteam|grps=/atlas,/cms,/dteam|grpopt=10|dbg

is sort of an ungodly mess. Here we know that '=' is the key-value separator (as opposed to colon) and the option separator is '|' (as opposed to space) and there is no leading character (as opposed to dash). So, yes, if I were told that's the blobbing template I could apply it when you see multiple gsi.authzfunparms directives to construct that blob. The question is where does the template come from? The config file? A builtin template? From the protocol itself? If it's a builtin template then there would need to be a way to override it if you wanted to use another plugin that has some other kind of template going on. If it comes from the protocol then the protocol would have to instantiate a blobber itself and provide the template but then each plugin would have to do the same. authz and voms plugins really use different syntax based on who wrote the plugin? Do we have multiple instances of this going on already? If not, we can kill this divergence now before it starts I think.

[bbockelm]

Hi @abh3 -

I'm sure that all plugins use a different syntax - as you suggest, it's likely from plugin writers working independently (and, I'd add, not having access to the configuration subsystem).

However, setting up a class that generates the input string seems to be quite complicated (example question: if I specify a configuration variable multiple times, is it additive or does it replace earlier invocations) compared to simply providing access to the config system like other plugins do.

No?

Brian

[abh3]

Yep, it is not straight forward. Though for gsi and the plugins it uses it all seems to not be additive as you have to concatenate all the key values in one go in the parameter blob. It would seem the best idea is to look over and enable blobbing on a case by case basis. As for access to the config system... that was made public years ago. Notice that XrdOucStream.hh is available in the public headers. That's the only thing you need to get the config file directives specific to your machine (ok, it likely needs somewhat better documentation). As far as I can tell people just don't want to use it for what they think are simple parameters. I mean I can simplify it even more (e.g. you give me the tags you want in the config file and I will give you an std::vector of key-value pairs to do whatever you want with). Anyway, doing all of the conversion will take time as there are a lot of other tasks that have higher priority when it comes to similar time allocation (e.g. CRL refresh, HTTPS-xroots openSSL merging, etc). If it's easy we can usually fit it in the cracks but for longer tasks, it will have to wait.

…

On Mon, 20 Apr 2020, Brian P Bockelman wrote: Hi @abh3 - I'm sure that all plugins use a different syntax - as you suggest, it's likely from plugin writers working independently (and, I'd add, not having access to the configuration subsystem). However, setting up a class that generates the input string seems to be quite complicated (example question: if I specify a configuration variable multiple times, is it additive or does it replace earlier invocations) compared to simply providing access to the config system like other plugins do. No? Brian -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #1016 (comment)

[bbockelm]

Yup - agreed that there are higher priority things out there!

I'm aware that XrdOucStream.hh is public; I think all we really need here is the configuration file name! I guess what I was envisioning was a new plugin invocation that includes both the config parameters (-vomsfunparms:certfmt=pem|vos=atlas,cms,dteam|grps=/atlas,/cms,/dteam|grpopt=10|dbg above) and is also given the config filename.

[xrootd-dev]

Yep, I get it. The issue is that sub-plugins are the wild west. However, irrespective how the plugin was designed (or not), you can always get the path to the configuration file via the "XRDCONFIGFN" envar (I think that's documented). Andy

…

On Thu, 23 Apr 2020, Brian P Bockelman wrote: Yup - agreed that there are higher priority things out there! I'm aware that `XrdOucStream.hh` is public; I think all we really need here is the configuration file name! I guess what I was envisioning was a new plugin invocation that includes both the config parameters (`-vomsfunparms:certfmt=pem|vos=atlas,cms,dteam|grps=/atlas,/cms,/dteam|grpopt=10|dbg ` above) and is also given the config filename. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #1016 (comment) ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1

[bbockelm]

Ah, good to know! I think that might allow me to do what I want.

For now, let's put this ticket on pause until I can track down the XRDCONFIGFN route.

Brian

[abh3]

OK, I suggest that I give you a config file extractor. Pro: you don't have to deal with all the gory details. Con: you wn't be able to issue inline error messages as all you get is an extraction of the congig file that is relevant to you. Let me know how you'd like to go with that. Andy

…

On Thu, 23 Apr 2020, Brian P Bockelman wrote: Ah, good to know! I think that might allow me to do what I want. For now, let's put this ticket on pause until I can track down the `XRDCONFIGFN` route. Brian -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #1016 (comment)

[bbockelm]

You mean the sort of thing you summarized in #1016 (comment)? If so, I don't think that's necessary - once I know how to get the config filename, I've done the parsing in several other plugins.

I do think a simplified config file interface would be fantastic - that's a separate discussion, however.

[abh3]

If you are referring to an std::vector of config line lines, yes that's what I mean. Otherwise, look at how XrdHttpProtocol.cc sets it up. As for a simplified interface, suggestions are welcomed. If you'd like, start a new ticket for that.

[abh3]

We have a new method to extract relevant directives from a config file making this a trivial process for new plugins. We will not be going back and fixing old plugins as the labor cost is excessive.