Default adminpath may be pruned by systemd-tmpfilesd

[olifre]

Not sure if this is a "bug" or just an annoyance, or what is the best way to go about it:
I observed our TPC transfers today failing in case X.509 delegation was required. The cause was:

210901 08:19:47 393183 ofs_TPC: Unable to create credentials file /tmp/grid/.ofs/.tpccreds/fts3.26515:30@fts-atlas-003.cern.ch#17.creds; no such file or directory

which in turn was caused by /tmp/grid/.ofs/.tpccreds not existing anymore after some days without transfers using X.509 delegation.

Checking the code, it seems the directory is only created once, and then assumed to stay. Sadly, tmpfiles cleaners such as systemd-tmpfilesdor tmpreaper regularly pass by nowadays to prune such directories when not used for a while. And since XRootD is running very stable, it runs for many days in a row for us ;-).

I think there are several ways this could be handled:

• Warn in the documentation about it, and delegate choice of a different directory to the user.
• Add a systemd-tmpfilesd exception and ship it with the rpm packages (might also be necessary to do this for tmpreaper and other such things).
• Always recreate directories in the adminpath in case they go missing at runtime.

I'm not sure what is the best way, and maybe you have an even better idea ;-).

[abh3]

Hi Oliver, In fact we specifically have a warning about using /tmp as the admin directory in the documentation: https://xrootd.slac.stanford.edu/doc/dev53/xrd_config.htm#_Toc60181749 Additionally, certain features are turned off when the admin path is located in /tmp (e.g. checkpointing). The only real solution is to not use /tmp. The only reason it defaults to /tmp is because that's the only place that's common to all possible sites not because it's the place that should be used (in fact, it really shouldn't be placed there). It's pretty easy to locate it elsewhere and there are a number of ways to accomplish this (see the documentation above). Andy

…

On Wed, 1 Sep 2021, Oliver Freyermuth wrote: Not sure if this is a "bug" or just an annoyance, or what is the best way to go about it: I observed our TPC transfers today failing in case X.509 delegation was required. The cause was: ``` 210901 08:19:47 393183 ofs_TPC: Unable to create credentials file ***@***.***#17.creds; no such file or directory ``` which in turn was caused by `/tmp/grid/.ofs/.tpccreds` not existing anymore after some days without transfers using X.509 delegation. Checking the code, it seems the directory is only created once, and then assumed to stay. Sadly, `tmpfiles` cleaners such as `systemd-tmpfilesd`or `tmpreaper` regularly pass by nowadays to prune such directories when not used for a while. And since XRootD is running very stable, it runs for many days in a row for us ;-). I think there are several ways this could be handled: * Warn in the documentation about it, and delegate choice of a different directory to the user. * Add a `systemd-tmpfilesd` exception and ship it with the `rpm` packages (might also be necessary to do this for `tmpreaper` and other such things). * Always recreate directories in the adminpath in case they go missing at runtime. I'm not sure what is the best way, and maybe you have an even better idea ;-). -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #1502

[olifre]

Hi Andy,

oh my — indeed, that warning is pretty prominent in the documentation. I must have had my eyes half-closed when looking for it, the docs even mention the warning right next to the default value.

So yeah, this is a classical case of user error, thanks for pointing me to the documentation! I assume relocating it may also require some caution with activated SELinux, but I'm used to that ;-).

Thanks and cheers,
Oliver

[bbockelm]

Before we close things off --

If there's a Big Red Warning Sign in the docs, why do we have this (do we have this?) as the default value? Wouldn't it be better to provide an improved default in the packaging (probably something in /run?).

Brian

[abh3]

Historically, site admins get annoyed when you make non-obvious default decisions for them (i.e. path placement), especially when it may require non-default security settings. As far as I can tell, they are OK with a bad default that they can change because semi-bad defaults require more work to figure out how bad they really are. Frankly, I sort of agree with that approach. While /tmp is a bad default there really is no simple "good" default as there is no common agreement on what "good" means for the admin path placement. It all depends....

…

On Wed, 1 Sep 2021, Brian P Bockelman wrote: Before we close things off -- If there's a Big Red Warning Sign in the docs, why do we have this (do we have this?) as the default value? Wouldn't it be better to provide an improved default in the packaging (probably something in `/run`?). Brian -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: #1502 (comment)

[olifre]

For reference, I have settled on:
all.adminpath /run/xrootd
which seems to work fine with SELinux, since the directory is shipped with the RPMs (with correct SELinux contexts) and contains the PID files (used by the systemd units). That seems to work out of the box.
Of course, Andy's point still holds — while I'm personally a fan of using /run/xrootd, since it matches FHS, I'm probably not the standard admin out there ;-).