systemd and directories an EL7

[martbhell]

Hello!

Trying out xrootd - redirector to a dCache instance on EL7 - (it's working - these are just some errors I get in the xrootd logs).

setup:
The exact same /etc/xrootd/xrootd-clustered.cfg config, storage.xml and Authfile on an EL6 and EL7.
Both have OSG33 and xrootd 4.3.0 and xrootd-cmstfc 1.5.1-10. dCache 2.15
Started the services with systemctl start xrootd@clustered.service
selinux is in permissive mode

Issues on EL7:
A) I had to manually create /var/run/xrootd and set ownerships. This means if I reboot the server this doesn't work. does not survive a reboot.
I guess this is because I have

all.pidpath /var/run/xrootd

If I change this to /var/run

then it tries to create /var/run/clustered

B) It also tries to create a //clustered directory
------ xrootd protocol initialization completed.
160512 15:05:34 6059 XrdConfig: Unable to create home directory //clustered; permission denied

I couldn't find what setting to change the home directory in http://xrootd.org/doc/prod/xrd_config.htm

This is the adminpath:

all.adminpath /var/run/xrootd

[zvada]

We've found same issue, though I've set adminpath to spool dir as follows: all.adminpath /var/spool/xrootd. This would solve your part A I believe.
Still, for the B, there is presence of the following upon (re)start of the xrootd service:

------ xrootd protocol initialization completed.
160513 14:37:46 1355 XrdConfig: Unable to create home directory //clustered; permission denied
160513 14:37:46 1355 XrdConfig: Unable to create envfile /var/run/xrootd/xrootd.clustered.env; no such file or directory
------ xrootd clustered@xrootd-local.unl.edu:1094 initialization completed.

Should we need to change all.adminpath from /var/run to /var/spool for el7 or what is causing //clustered; permission denied?

[abh3]

The details of what files are created anw where they are created can be found here:
http://xrootd.org/doc/dev44/xrd_config.htm#_Toc448774978

In case that is not clear (and please let me know if it's not), some comments:

1. Unable to create home directory
   The home directory is whatever the current working directory is at start-up. If the '-n' option was specified, then a directory with the name matching the '-n' argument is created in that directory and xrootd makes it the home directory. This is part of automatic fencing which allows you to trivially run multiple xrootd's on the same machine. Fencing is described here:
   http://xrootd.org/doc/dev44/xrd_config.htm#_Toc448774975

In this case, it appears that a) the current working directory was "/" (bad idea) and "-n clustered" was specified as an argument. Obviously, one cannot create directories in "/" unless you are root.

2. Unable to create envfile
   The envfile is a file that contains the information about the running environment and can be used by external scripts to locate various adminitrative files xrootd creates. Where it's placed depends on what was specified by the pidpath directive (if any), the "-s" command line option option (is specified), and "-n" option. Whatever you specify the last component of that path (e.g. xrootd in this case) needs to have owner rwx permisions and be owned by the userid running the daemon. This seems to not have been the case here. Whether you shoose /var/runxrootd or /var/spool/xrootd is a site preference as xrootd doesn't care.

In general, all directories you specify in the config file or command line options must be owned by xrootd with user permissions set to rwx (i.e. the last component). I am closing this ticket since it's not a bug but a configuration problem on how the directories were setup. If you feel otherwise, please re-open the thicket and let me know what you think is the endemic xrootd-specific problem here.

[martbhell]

"-s /var/run/xrootd" is the default in the systemd service script.

If I make an systemctl edit xrootd.. and change the -s to point to /var/spool/ I get rid of the unable to create envfile. (I'd also have to do this for the cmsd service).

Unfortunately //clustered is still trying to be created.

I fixed that by setting WorkingDirectory=/var/spool/xrootd in the systemd override file.

https://github.com/xrootd/xrootd/blob/422ae9cc6ca0dcb08f573e1dfe4a6850abf29f54/packaging/common/xrootd%40.service

I would like to be able to set these settings without having to modify / override the systemd service file from xrootd. Would it be possible to introduce variables?

p.s. I can't re-open - I'd have to open a new one so I'll reply here and hope that's enough :)

[bbockelm]

Hi,

Reviving this ticket because other folks are complaining about it.

The XRootD packaging is obviously wrong. While, yes, it's technically possible to correct the out-of-the-box issues, I don't understand why these can't be fixed in the packaging?

• XRootD RPM packages default to /var/run/xrootd, yet the packaging does not create /var/run/xrootd using the OS's mechanisms (tmpfiles.d).
• XRootD is set to start in / and, on RHEL7, requires you to specify a named instance (-n option). Hence the attempt to create //clustered. Suggestion: set the startup directory to not-`/ in the packaging.

Brian

[simonmichal]

@bbockelm :

Regarding your first suggestion: I fixed that exactly one week ago ;-) :
#485

Regarding the second one: given that the first problem is already solved, I suppose we can set the working directory to /var/run/xrootd. Although, this is not permanent storage on cc7, any objections?

Michal

[bbockelm]

Hi Michal,

Locally, we drop in this override:

[Service]
...
WorkingDirectory=/var/spool/xrootd

[olifre]

This issue still affects cmsd on EL7:

170809 17:02:19 16835 XrdConfig: Unable to create home directory //clustered; permission denied

It's also expected, since the mentioned commit sadly only fixes the xrootd@ service. The same fix should be done for cmsd@ in packaging, I believe.

[simonmichal]

Thanks for pointing it out! Just in time before the next RC :-)

Michal

[simonmichal]

Done in: 378da3a

Michal

[olifre]

Many thanks! 👍