Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFE: Add template specifier for path to unit file #6308

Closed
matthijskooijman opened this issue Jul 8, 2017 · 23 comments · Fixed by #20262 or #22195
Closed

RFE: Add template specifier for path to unit file #6308

matthijskooijman opened this issue Jul 8, 2017 · 23 comments · Fixed by #20262 or #22195
Labels
pid1 RFE 🎁 Request for Enhancement, i.e. a feature request

Comments

@matthijskooijman
Copy link

matthijskooijman commented Jul 8, 2017

Submission type

  • Request for enhancement (RFE)

I would like to see one or two extra specifiers to be used in unit files that would expand to:

  • The full path to the current unit file.
  • The full path to the directory the current unit file is in.

This would help to support bits of software that are not installed into a fixed place, without having to hardcode paths into the unit file.

E.g. consider a git clone of a bit of software. I cloned this into /home/matthijs/foo, but this could be any other path. To start a daemon from that directory, I'm including a service file into the git repo, that looks like this:

[Service]
ExecStart=/home/matthijs/foo/foo-daemon
User=matthijs

(more stuff omitted)

After making the git clone, I can set this up using:

sudo systemctl enable /home/matthijs/foo/foo.service

This makes the needed links in /etc/systemd to make this service run.

However, the service file hardcodes the path to the git checkout, which makes it not so suitable for inclusion in the git repository itself.

If, for example, %d would refer to the unit file's directory, I would be able to use:

ExecStart=%d/foo-daemon

and this service file would work regardless of where the git checkout
was made.

@poettering poettering added pid1 RFE 🎁 Request for Enhancement, i.e. a feature request labels Jul 10, 2017
@keszybz
Copy link
Member

keszybz commented Jul 26, 2017

Hmm. You could easily have a script in the git repo which would generate the unit file with the right path and call systemctl enable for you. I don't see anything wrong with your idea, but OTOH, it's such a fringe case that I'm not sure it deserves support in systemd itself.

@ams-tschoening
Copy link

ams-tschoening commented Jan 4, 2018

I have exactly the same use case and would like to vote for the proposed enhancement as well. A lot of people deploy software using their SCMs, including things like systemd configs. Enhancements for those use cases have been implemented in the past already as well.

While of course each and every project could implement custom scripts, this seems like redundant and error prone work to me. systemd already provides the concept of specifiers into which the newly suggested ones would perfectly fit. I have lots of different projects and customers per system, so would most likely implement some different approach, like a system wide dispatcher forwarding ExecStart and Co., to ease maintenance of systemd-handling in my environment. Therefore I would be forced to collect the needed data differently than other scripts, data which ist most likely already available to systemd itself. And that wouldn't even work with setting working or root directories, which is systemd already capable of doing.

Additionally, dynamically generating those scripts makes deployment in general more difficult: Where should those generated scripts be stored? In the working copy and if so, should it be versioned by users or not? How about updates, which might change some template unit file? With custom generated unit files the generator might need to run again, which would not be needed if there's only one versioned file containing all the needed specifiers already.

So, in my opinion, this is not a fringe case anymore, but instead becomes more and more common and additional format specifiers would make systemd support such use cases better out of the box. And from my understanding, things better/easier working OOB was often used as argument for other discussions as well.

@fgervais
Copy link
Contributor

fgervais commented Aug 6, 2018

Yes same use-case here

@keszybz
Copy link
Member

keszybz commented Aug 6, 2018

And how would those specifiers expand in drop-ins?

@ams-tschoening
Copy link

AFAIK there are two forms of drop-ins, one overriding a file completely and the other overriding individual settings. So it depends on how the specifier is defined in the end and which override is used: If a file is overridden completely and the specifier implements the easiest case of resolving to the unit file where it is defined in, do the same with overrides and assume users know what they are doing.

In case of overriding individual settings, I would expect that if the setting containing the specifier is NOT overridden, it resolves to the file in which it is actually specified. If it is overridden, assume the same argument as before and resolve based on the overriding file. Depending on how overriding individual settings is implemented, resolving based on the original file might be difficult of course.

But don't make things too complicated, this is a 99 % vs. 1 %-scenario. In 99 % of the use-cases things won't be overridden and users can write "wrong" unit files anyway for any reason. Drop-ins might simply be not supported well like the whole use-case described in this issue is currently. In the end, even without proper support for drop-ins things like %d would be an enhancement to all users of the described use-case.

@elgow
Copy link

elgow commented Jan 14, 2019

Another vote in favor here.

For units that are not overridden it's possible to determine the source file path from the FragmentPath property. If a unit is overridden using systemctl edit --full then the FragmentPath gives the path of the override file rather than the actual source file path. In that case the source path becomes unknowable. It would be good to have specifiers to get the path to the unit source file for the use case described above.

@sshymko
Copy link

sshymko commented Mar 13, 2019

A related feature would be support of executive paths relative to WorkingDirectory:
https://www.digitalocean.com/community/questions/trying-to-change-workingdirectory-of-systemd-service-unit

@elgow
Copy link

elgow commented Mar 13, 2019

I'd like to second and elaborate on what @ams-tschoening said about this. I try to have my enabled units link directly to source files in a git checkout directory that is the deployment. If forced to override the contents of the source file as described by @keszybz then all or part of the source file becomes inoperative. It is MUCH more useful to have symbols (either %x or ${x}), in the source file that get replaced by path components at runtime. That's available for the home dir with %h and ${HOME}. Why can't it be provided for the location of the source file as well? As it stands, the .path units are almost useless because so much has to be hard-coded a priori in the source file that the deployment becomes far too brittle.

@sergei66666
Copy link

sergei66666 commented Mar 20, 2019

Also another way which help reduce errors

[Service]
WorkingDirectory=/home/admin/testapp
ExecStartPre=/bin/chmod +x %wd/app-executable
ExecStart=%wd/app-executable

#where %wd is value of WorkingDirectory

Another words - it help if we need to take some actions on the same folder.

@jansohn
Copy link

jansohn commented Oct 28, 2020

I'm also looking for a way to not having to repeat the working directory path in ExecStart and ExecStop... It violates the DRY principle IMHO...

WorkingDirectory=/opt/apache-tomcat
ExecStart=/opt/apache-tomcat/bin/startup.sh
ExecStop=/opt/apache-tomcat/bin/shutdown.sh

@anitazha
Copy link
Member

anitazha commented Jul 1, 2021

I'm onboarding @alexlzhu and am proposing he works on a feature BinaryDirectory= (name TBD) that will use the specified directories as additional, higher priority look up paths for executables without absolute paths. I think that would cover the use cases here and was something we asked for in ASG 2019 (though we never filed an RFE for it).

alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Jul 14, 2021
…xecuted by Exec*= should be found │A way to specify a directory where binaries executed by Exec*= should be found (in addition to the PATH set by systemd).

Closes systemd#6308
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Jul 14, 2021
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Jul 14, 2021
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Jul 14, 2021
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Jul 14, 2021
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Jul 16, 2021
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Jul 18, 2021
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Jul 20, 2021
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Jul 21, 2021
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Jul 21, 2021
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Jul 21, 2021
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Jul 21, 2021
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Jul 21, 2021
alexlzhu added a commit to alexlzhu/systemd that referenced this issue Jul 21, 2021
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Jul 21, 2021
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Sep 24, 2021
…ies executed by Exec*= should be found

Currently there does not exist a way to specify a path relative to which
all binaries executed by Exec should be found. The only way is to
specify the absolute path.

This change adds the functionality to specify a path relative to which
binaries executed by Exec*= can be found.

Closes systemd#6308
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Sep 24, 2021
…ies executed by Exec*= should be found

Currently there does not exist a way to specify a path relative to which
all binaries executed by Exec should be found. The only way is to
specify the absolute path.

This change adds the functionality to specify a path relative to which
binaries executed by Exec*= can be found.

Closes systemd#6308
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Sep 24, 2021
…ies executed by Exec*= should be found

Currently there does not exist a way to specify a path relative to which
all binaries executed by Exec should be found. The only way is to
specify the absolute path.

This change adds the functionality to specify a path relative to which
binaries executed by Exec*= can be found.

Closes systemd#6308
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Sep 24, 2021
…ies executed by Exec*= should be found

Currently there does not exist a way to specify a path relative to which
all binaries executed by Exec should be found. The only way is to
specify the absolute path.

This change adds the functionality to specify a path relative to which
binaries executed by Exec*= can be found.

Closes systemd#6308
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Sep 25, 2021
…to which binaries executed by Exec*= should be found

Currently there does not exist a way to specify a path relative to which
all binaries executed by Exec should be found. The only way is to
specify the absolute path.

This change adds the functionality to specify a path relative to which
binaries executed by Exec*= can be found.

Closes systemd#6308
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Sep 27, 2021
…to which binaries executed by Exec*= should be found

Currently there does not exist a way to specify a path relative to which
all binaries executed by Exec should be found. The only way is to
specify the absolute path.

This change adds the functionality to specify a path relative to which
binaries executed by Exec*= can be found.

Closes systemd#6308
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Sep 27, 2021
…to which binaries executed by Exec*= should be found

Currently there does not exist a way to specify a path relative to which
all binaries executed by Exec should be found. The only way is to
specify the absolute path.

This change implements the functionality to specify a path relative to which
binaries executed by Exec*= can be found.

Closes systemd#6308
alexlzhu pushed a commit to alexlzhu/systemd that referenced this issue Sep 27, 2021
…to which binaries executed by Exec*= should be found

Currently there does not exist a way to specify a path relative to which
all binaries executed by Exec should be found. The only way is to
specify the absolute path.

This change implements the functionality to specify a path relative to which
binaries executed by Exec*= can be found.

Closes systemd#6308
cdown pushed a commit that referenced this issue Sep 28, 2021
…to which binaries executed by Exec*= should be found

Currently there does not exist a way to specify a path relative to which
all binaries executed by Exec should be found. The only way is to
specify the absolute path.

This change implements the functionality to specify a path relative to which
binaries executed by Exec*= can be found.

Closes #6308
@ams-tschoening
Copy link

ams-tschoening commented Sep 28, 2021

How does the new ExecSearchPath closes this request? From my understanding one still needs to maintain a hard-coded path, which is NOT what this request is about. It's about the exact opposite, to NOT need to maintain those paths in SCM-checkouts. Just look at the examples:

ExecStart=%d/foo-daemon
ExecStart=%wd/app-executable

This should be reopened in my opinion, ExecSearchPath might make the current setup easier, but doesn't really address this RFE.

@elgow
Copy link

elgow commented Sep 28, 2021

I agree that the ExecSearchPath doesn't solve the problem unless it could be specified with a wildcard indicating the path of the current unit, or the cwd or other non-explicit values.

@anitazha
Copy link
Member

specified with a wildcard indicating the path of the current unit, or the cwd or other non-explicit values.

There is %I which you can use to pass arbitrary stuff as the instance name:

# service_name@.service
[Service]
ExecStart=foo-binary
ExecSearchPath=%I:/bin:/usr/bin:/your_favorite_dir

And then you can run like so: systemctl start service_name@/path/to/bin
Which results in the path being: /path/to/bin:/bin:/usr/bin:/your_favorite_dir

But actually looking at this issue again you could have used specifiers and templates to do this even without ExecSearchPath= to pass arbitrary paths:

# service_name@.service
[Service]
ExecStart=%I/foo-binary

ExecSearchPath= allows you to use drop-in overrides though which is nicer if you already know which paths to use.

If you really want a specifier specifically for working directory though we can reopen but I agree with keszybz that it's a bit of a fringe use case.

@elgow
Copy link

elgow commented Sep 28, 2021

If a service is set for auto-start at boot would the systemctl start service_name@/path/to/bin persist through a reboot?

@ams-tschoening
Copy link

ams-tschoening commented Sep 29, 2021

And then you can run like so: systemctl start service_name@/path/to/bin

Besides the reboot-question, this already looks like an ugly hack.

[...]but I agree with keszybz that it's a bit of a fringe use case.

Only if one ignores the LIKE-votes and somewhat detailed comments explaining absolutely valid and pretty standard use-cases. :-/

@anitazha
Copy link
Member

anitazha commented Sep 29, 2021

If a service is set for auto-start at boot would the systemctl start service_name@/path/to/bin persist through a reboot?

Yes if you do systemctl enable on it which creates the appropriate symlinks. You need to escape the / by replacing with -. Additional support questions can be taken to the systemd-devel mailing list.

@anitazha anitazha reopened this Sep 29, 2021
@matthijskooijman
Copy link
Author

matthijskooijman commented Sep 29, 2021

Edit: Seems my post and the reopening crossed each other, thanks @anitazha for reopening!

I would indeed agree that the linked PR does not solve my original request, or even improve it at all (it does solve #6308 (comment) and #6308 (comment), but IMHO those are really different problems from the original issue). I would suggest reopening this.

Of course, this is a bit of a fringe case, so closing this as wontfix would be a valid path (I personally would prefer adding the specifier, obviously, but I can see that choices must be made in what to implement and what to leave out), but then I would suggest reopening this and then closing it with a clear wontfix statement, for clarity.

As for the %I approach, I had not thought of that. AFAICS it would require renaming the unit file to end with an @ (to become a template). It would still require the user to specify the full pathname to the unit-file/git checkout but then in a command when enabling the unit, rather than hardcoded in the file. So this could use a git-tracked unit file without modifications indeed. It does not implement the request of this issue, but it does provide a less ideal but IMHO workable alternative for the underlying problem.

If a service is set for auto-start at boot would the systemctl start service_name@/path/to/bin persist through a reboot?

AFAIU, yes, though you'd need enable, not start here. For example getty is also managed like this, I have a getty@tty1.service symlink in my /etc/systemd/system/getty.target.wants which persists this instance argument.

pdmorrow pushed a commit to pdmorrow/systemd that referenced this issue Nov 17, 2021
…to which binaries executed by Exec*= should be found

Currently there does not exist a way to specify a path relative to which
all binaries executed by Exec should be found. The only way is to
specify the absolute path.

This change implements the functionality to specify a path relative to which
binaries executed by Exec*= can be found.

Closes systemd#6308
keszybz added a commit to keszybz/systemd that referenced this issue Jan 20, 2022
Fixes systemd#6308: people want to be able to link a unit file via 'systemctl enable'
from a git checkout or such and refer to other files in the same repo.
The new specifiers make that easy.

%y/%Y is used because other more obvious choices like %d/%D or %p/%P are
not available because at least on of the two letters is already used.

The new specifiers are only available in units. Technically it would be
trivial to add then in [Install] too, but I don't see how they could be
useful, so I didn't do that.

I added both %y and %Y because both were requested in the issue, and because I
think both could be useful, depending on the case. %Y to refer to other files
in the same repo, and %y in the case where a single repo has multiple unit files,
and e.g. each unit has some corresponding asset named after the unit file.
@keszybz
Copy link
Member

keszybz commented Jan 20, 2022

@matthijskooijman (and anyone else who commented): please check if #22195 DTRT for you.

keszybz added a commit to keszybz/systemd that referenced this issue Jan 20, 2022
Fixes systemd#6308: people want to be able to link a unit file via 'systemctl enable'
from a git checkout or such and refer to other files in the same repo.
The new specifiers make that easy.

%y/%Y is used because other more obvious choices like %d/%D or %p/%P are
not available because at least on of the two letters is already used.

The new specifiers are only available in units. Technically it would be
trivial to add then in [Install] too, but I don't see how they could be
useful, so I didn't do that.

I added both %y and %Y because both were requested in the issue, and because I
think both could be useful, depending on the case. %Y to refer to other files
in the same repo, and %y in the case where a single repo has multiple unit files,
and e.g. each unit has some corresponding asset named after the unit file.
keszybz added a commit to keszybz/systemd that referenced this issue Jan 20, 2022
Fixes systemd#6308: people want to be able to link a unit file via 'systemctl enable'
from a git checkout or such and refer to other files in the same repo.
The new specifiers make that easy.

%y/%Y is used because other more obvious choices like %d/%D or %p/%P are
not available because at least on of the two letters is already used.

The new specifiers are only available in units. Technically it would be
trivial to add then in [Install] too, but I don't see how they could be
useful, so I didn't do that.

I added both %y and %Y because both were requested in the issue, and because I
think both could be useful, depending on the case. %Y to refer to other files
in the same repo, and %y in the case where a single repo has multiple unit files,
and e.g. each unit has some corresponding asset named after the unit file.
@yuwata yuwata linked a pull request Jan 20, 2022 that will close this issue
keszybz added a commit to keszybz/systemd that referenced this issue Jan 21, 2022
Fixes systemd#6308: people want to be able to link a unit file via 'systemctl enable'
from a git checkout or such and refer to other files in the same repo.
The new specifiers make that easy.

%y/%Y is used because other more obvious choices like %d/%D or %p/%P are
not available because at least on of the two letters is already used.

The new specifiers are only available in units. Technically it would be
trivial to add then in [Install] too, but I don't see how they could be
useful, so I didn't do that.

I added both %y and %Y because both were requested in the issue, and because I
think both could be useful, depending on the case. %Y to refer to other files
in the same repo, and %y in the case where a single repo has multiple unit files,
and e.g. each unit has some corresponding asset named after the unit file.
@matthijskooijman
Copy link
Author

@keszybz Thanks! I haven't tested the PR, but from the documentation it seems like it would indeed fix my case exactly. Awesome!

@5310
Copy link

5310 commented Aug 12, 2023

@matthijskooijman Did fragment specifiers resolve your use-case in the end?

I would like to use fragment specifiers in my systemd configs as well, but I can't find any information whatsoever as to what they even are in context of systemd, let alone how to use them. The only mention of fragments related to unit files I can find are in this PR that implements fragment specifiers and the documentation it adds. And that's it.

Could you please direct me to what you found out about them?

@matthijskooijman
Copy link
Author

@matthijskooijman Did fragment specifiers resolve your use-case in the end?

I'm not sure I've actually tested yet, but from what I see from the docs, they should solve my case.

Could you please direct me to what you found out about them?

They are documented in the systemd.unit manpage, in the "FRAGMENT SPECIFICERS" section.

@matthijskooijman
Copy link
Author

For future reference, the %y and %Y specifiers are available from systemd 251 and up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pid1 RFE 🎁 Request for Enhancement, i.e. a feature request