fwupd 1.3.3 misconfigured ConfigurationDirectory #1479
Comments
|
our distro is stateless (in the systemd sense), we might need to patch
fwupd to first search in /etc and if the file is not there, fall back to
/usr/share, which is how we normally want this to work for our users
…On Mon, Nov 11, 2019 at 5:38 PM Mario Limonciello ***@***.***> wrote:
Upstream fwupd received a bug report that on Clear they have no remotes
and can't enable any.
The bug report was fwupd/fwupd#1555
<fwupd/fwupd#1555>.
After some investigation it was noted that the debug logs showed:
FuConfig using config path of /etc/fwupd
As the package is configured with --sysconfdir=/usr/share/fwupd/ this was
surprising.
It turns out that it is caused by the ConfigurationDirectory directive in
the systemd unit. This takes precedence
<https://github.com/fwupd/fwupd/blob/679d1c0f9a49de15e247657d1fb284205bc34f5e/src/fu-common.c#L1028>over
sysconfdir.
Systemd provides this directive for unit files to use for their
configuration directory in the distribution. So this leaves 3 options that
we find. I'll try to compare and contrast them.
1.
Modify the fwupd package to use --sysconfdir=/etc.
This would make it more like other distributions.
Configuration files wouldn't be overwritten by distribution updates.
I think this would fix the problem effectively.
2.
Fix the systemd package. If the clearlinux distribution expects all
packages to store data in /usr/share it should be configured such that ConfigurationDirectory
<https://www.freedesktop.org/software/systemd/man/systemd.exec.html>is
correct.
This should fix both fwupd and any other package that encounters a
similar problem.
3.
In the clearlinux fwupd package comment out the ConfigurationDirectory
directive in the fwupd.service unit. This will only fix the fwupd
package, but it is the easiest and potentially least contentious option if
1 or 2 are not doable options. It was verified in the upstream bug already
to solve the issue.
CC original reporter @Queuecumber <https://github.com/Queuecumber> and
@hughsie <https://github.com/hughsie>
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#1479?email_source=notifications&email_token=AAJ54FPYACZXDLAZO7OMTG3QTICJ5A5CNFSM4JL47572YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HYSCLNQ>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAJ54FK7ZICOND6KMYP47H3QTICJ5ANCNFSM4JL4757Q>
.
|
|
So normally if a user wants to change a configuration file they are supposed to duplicate it from |
|
there are two modes;
1) copy and modify
and
2) overlay /etc on top (eg incremental settings)
not all software can deal with #2.
(this is also how systemd works and more and more other things fwiw)
…On Mon, Nov 11, 2019 at 5:47 PM Mario Limonciello ***@***.***> wrote:
So normally if a user wants to change a configuration file they are
supposed to duplicate it from /usr/share to /etc themselves and then
modify it in /etc?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1479?email_source=notifications&email_token=AAJ54FOMZ6L5DEMXKXZCJM3QTIDJXA5CNFSM4JL47572YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDYXUCY#issuecomment-552696331>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAJ54FLAO7RCYSEQPT7HXYLQTIDJXANCNFSM4JL4757Q>
.
|
|
(specifically, we as OS will not put or update files in /etc. those are
fully owned by the user and whatever tools they use to manage. yes this is
different from some other distros, but a lot of our users really like the
strict separation of OS and user provided configuration files)
On Mon, Nov 11, 2019 at 5:48 PM Arjan van de Ven <arjanvandeven@gmail.com>
wrote:
… there are two modes;
1) copy and modify
and
2) overlay /etc on top (eg incremental settings)
not all software can deal with #2.
(this is also how systemd works and more and more other things fwiw)
On Mon, Nov 11, 2019 at 5:47 PM Mario Limonciello <
***@***.***> wrote:
> So normally if a user wants to change a configuration file they are
> supposed to duplicate it from /usr/share to /etc themselves and then
> modify it in /etc?
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <#1479?email_source=notifications&email_token=AAJ54FOMZ6L5DEMXKXZCJM3QTIDJXA5CNFSM4JL47572YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDYXUCY#issuecomment-552696331>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAJ54FLAO7RCYSEQPT7HXYLQTIDJXANCNFSM4JL4757Q>
> .
>
|
|
fwupdmgr has lots of commands that modify those files. For example:
and with fwupd 1.3.3 after performing an update you may be prompted to upload a report automatically in the future. Depending what you say a key to be modified in each remote your preference. So in an ideal scenario how should that work in your mind? Should the |
|
Ideal case is where it would add a new file for the new remote, in /etc
and at start, all settings are read first from the reference file in /usr,
and then /etc gets processed where only options in those files overwrite
the defaults that came out of /usr
that way the tool just writes one small file with just the new content and
does not need to know about anything else in those files
and if a new upstream version comes out with more stuff in the default
config... it just works nicely due to the overlay nature... the os updates
the file in /usr with no conflicts
…On Mon, Nov 11, 2019, 17:58 Mario Limonciello ***@***.***> wrote:
fwupdmgr has lots of commands that modify those files. For example:
#fwupdmgr enable-remote lvfs-testing would cause daemon to modify a key
in /etc/fwupd/remotes.d/lvfs-testing.conf
and with fwupd 1.3.3 after performing an update you may be prompted to
upload a report automatically in the future. Depending what you say a key
to be modified in each remote your preference.
So in an ideal scenario how should that work in your mind? Should the
fwupdmgr tool know about this stateless configuration and clone the file
from /usr/share/fwupd to /etc/fwupd/?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1479?email_source=notifications&email_token=AAJ54FNIH65IIEOGHZIVM2LQTIETTA5CNFSM4JL47572YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDYYHDQ#issuecomment-552698766>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAJ54FO2IJIUM3K76HT5USLQTIETTANCNFSM4JL4757Q>
.
|
|
or maybe if this is more similar to how systemd handles units; have a
directory with possible repos, and an 'enabled' directory where the tool
places symlinks.
you'd have two enabled directories, one in /usr/share and one in /etc...
and the one in /etc supplements and overrides the /usr/share one.
this is also nice in that now enabling/disabling is a file(well, symlink),
which is something you can also ship in a package.....
On Mon, Nov 11, 2019 at 6:04 PM Arjan van de Ven <arjanvandeven@gmail.com>
wrote:
… Ideal case is where it would add a new file for the new remote, in /etc
and at start, all settings are read first from the reference file in /usr,
and then /etc gets processed where only options in those files overwrite
the defaults that came out of /usr
that way the tool just writes one small file with just the new content and
does not need to know about anything else in those files
and if a new upstream version comes out with more stuff in the default
config... it just works nicely due to the overlay nature... the os updates
the file in /usr with no conflicts
On Mon, Nov 11, 2019, 17:58 Mario Limonciello ***@***.***>
wrote:
> fwupdmgr has lots of commands that modify those files. For example:
>
> #fwupdmgr enable-remote lvfs-testing would cause daemon to modify a key
> in /etc/fwupd/remotes.d/lvfs-testing.conf
>
> and with fwupd 1.3.3 after performing an update you may be prompted to
> upload a report automatically in the future. Depending what you say a key
> to be modified in each remote your preference.
>
> So in an ideal scenario how should that work in your mind? Should the
> fwupdmgr tool know about this stateless configuration and clone the file
> from /usr/share/fwupd to /etc/fwupd/?
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <#1479?email_source=notifications&email_token=AAJ54FNIH65IIEOGHZIVM2LQTIETTA5CNFSM4JL47572YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDYYHDQ#issuecomment-552698766>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAJ54FO2IJIUM3K76HT5USLQTIETTANCNFSM4JL4757Q>
> .
>
|
|
More than just a few symlinks to turn on and off remotes, a bunch of individual keys in conf files get modified depending on what user selects. |
|
So I guess let me re-iterate back what I think would be ideal to you and make sure I understand and we come up with a workable idea.
@hughsie, what are your thoughts on this? |
|
as a general thing that's indeed pretty close. there's a lot of benefit from splitting what the base/defaults/etc are from what the user customizations are. CoreOS and systemd started this stateless concept, and it's... really nice in general. For most things it's pretty straightforward, there's very little upstream software that edits the config files directly, and from a sysadmin view as well as update view it's very nice (yeah there is visudo and a few similar things, but not very many) |
I don't think that's going to work with most modern daemons... It's pretty naive to assume that applications that are expecting a writable sysconfdir to store, well, their mutable system config in an immutable directory... I don't think you can just change a few compile flags and pretend that it's all going to work, especially without even testing if the service starts...
Although I don't agree that it's the applications job to copy files from /usr/etc to /etc if they are missing (as http://0pointer.de/blog/projects/stateless.html would imply) this is something we could add fairly easily.
No, I don't really want the override functionality. I think when /etc/fwupd exists then we disregard the contents of /usr/etc/fwupd completely. If we did the fallback thing, how could the user ever remove a remote (e.g. for compliance reasons) if /usr is immutable? |
I agree. I think it would be best to switch to dropping that compile flag until we have a good approach for making fwupd work with your preferred stateless approach. This current solution is broken.
Well to be fair; it does start. Fwupd can start without conffiles. Just remotes don't work because they're not configured :)
The problem with just copying files if they're missing is that you'll never get them upgraded for new keys that are introduced, changes, or removed. Regular distros handles this with conffile systems that will update conffiles if nothing changed or let you do a merge when installing an updated package.
That's true - they couldn't remove it, the best they could do was "disable" it. |
|
systems solves disable with symlink to /Dev/null
…On Tue, Nov 12, 2019, 06:40 Mario Limonciello ***@***.***> wrote:
I don't think that's going to work with most modern daemons... It's pretty
naive to assume that applications that are expecting a writable sysconfdir
to store, well, their mutable system config in an immutable directory...
I agree. I think it would be best to switch to dropping that compile flag
until we have a good approach for making fwupd work with your preferred
stateless approach. This current solution is broken.
I don't think you can just change a few compile flags and pretend that
it's all going to work, especially without even testing if the service
starts...
Well to be fair; it does start. Fwupd can start without conffiles. Just
remotes don't work because they're not configured :)
Although I don't agree that it's the applications job to copy files from
/usr/etc to /etc if they are missing (as
http://0pointer.de/blog/projects/stateless.html would imply) this is
something we could add fairly easily.
The problem with just copying files if they're missing is that you'll
never get them upgraded for new keys that are introduced, changes, or
removed. *Regular* distros handles this with conffile systems that will
update conffiles if nothing changed or let you do a merge when installing
an updated package.
If we did the fallback thing, how could the user ever remove a remote
(e.g. for compliance reasons) if /usr is immutable?
That's true - they couldn't remove it, the best they could do was
"disable" it.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1479?email_source=notifications&email_token=AAJ54FNM7XJRCF2M5OKYH53QTK56RA5CNFSM4JL47572YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOED2OVXQ#issuecomment-552921822>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAJ54FILM2OIN2HWCPNINJTQTK56RANCNFSM4JL4757Q>
.
|
|
Does systemd actually merge configuration files? I thought if you have something in Conceptually I like the idea of a remote being disabled or enabled by symlinks, but if we're changing keys in the conffile the first time you use fwupdmgr regarding sending a report what does it buy you? You'd still need to copy the file over to be modified. |
|
there's a simple way to remove, change, or add individual options by making files in |
which is why we prefer methods like |
|
I think this is raising a larger question. From your very nice explanation I think there's an undeniable benefit to the system that clear Linux is pushing. That being said, regardless of the route that fwupd ends up choosing, there's a lot of software that has become more or less standard and that won't work as described and will not be receptive to making changes due to
What is the correct way to handle such programs, presumably they should be made to be able to work under clear linux |
No, we don't allow packages to install anything in |
|
Most software in legacy space either is something users never change the config for (easy) or where we add a few small lines in a patch that checks /etc first and falls back to the distro reference if the etc file does not exist. the more complicated things like httpd have a rich configuration language that actually supports this natively |
|
@Queuecumber you raise a good point, but, let's focus on making |
|
A very simple solution is to not ship any config file at all, and have sane builtin defaults, with But I strongly prefer to keep |
Well fwupd does actually work that way for all things that have keys everything except remotes (which is the part that broke here).
I guess with the way things are built right now your --sysconfdir is completely ignored anyway due to systemd using
The state of the remotes (and cached metadata) gets put into |
|
How about, if there's no |
|
Ah then it can't modify them. |
|
There's got to be a way to solve this without pre-polluting |
|
Due to the editing commands, I think more systemd level work is required in fwupd to be stateless. Having a system config directory and admin config directory and only writing overrides/changes to the admin config directory would need to be supported I imagine. Otherwise you are left with the unfortunate scenario of a user initially installing the content and getting /etc populated with content at that time. Then if that content is updated in /usr, it is really not manageable to know if the content in /etc can or should be updated accordingly. |
Such as? Honestly, at this point I'm just fine with fwupd not working in clearlinux as it's clearly not respecting the FHS and expecting apps to do weird things... |
|
Removing it would save me work, too. Of course, the better outcome for everyone would be if |
|
This is coming back to my comment I worry that you (clear Linux) are being too idealistic, @fenrus75 makes a big assumption, you are going to run into this problem again and again as the distro matures. The only way forward is going to be to relax your constraint or reinvent things. This is fine as an experiment, but the real world requires compromise. Fwupd has become the standard for delivering firmware updates on Linux. It's not fwupd that is going to suffer from not being included in clear, it's your users. |
So just to be clear, we're not allowed to install into /etc, not allowed to copy from /usr/etc on first start, and not allowed to write to /usr. We have to start up without any etc contents, and only save changed keys to /var/lib, and also merge in user presets from /etc. If so, that sounds pretty crazy. However, /usr/etc does makes sense from a system-reset point if view. |
|
@Queuecumber we solve these issues for almost all packages. The benefit of all our users not having to wade through hundreds of files in So, we're going to solve this for every package, and, upstream projects are more than welcome to chime in and talk to us in here, as the above discussion demonstrates. So far, I am not deterred yet :) |
The copying is ... just circumventing the ideal that
It's partially made more complex because the software is architected to combine immutable repo information with user-mutable repo state. Splitting it would be a solution, and would avoid a lot of duplicate information in |
|
e.g. |
|
I assume
|
Yes |
|
great, thanks for the quick response. I'm pushing It has this default tmpfiles snipplet: On top of that the Hopefully, that puts everything in place that it "just works". |
|
That sounds like a workable consolation, thanks. |
|
|
|
It looks these promoted now, so closing this issue. |
Upstream fwupd received a bug report that on Clear they have no remotes and can't enable any.
The bug report was fwupd/fwupd#1555.
After some investigation it was noted that the debug logs showed:
FuConfig using config path of /etc/fwupdAs the package is configured with
--sysconfdir=/usr/share/fwupd/this was surprising.It turns out that it is caused by the
ConfigurationDirectorydirective in the systemd unit. This takes precedence oversysconfdir.Systemd provides this directive for unit files to use for their configuration directory in the distribution. So this leaves 3 options that we find. I'll try to compare and contrast them.
Modify the fwupd package to use
--sysconfdir=/etc.This would make it more like other distributions.
Configuration files wouldn't be overwritten by distribution updates.
I think this would fix the problem effectively.
Fix the systemd package. If the clearlinux distribution expects all packages to store data in
/usr/shareit should be configured such that ConfigurationDirectory is correct.This should fix both fwupd and any other package that encounters a similar problem.
In the clearlinux fwupd package comment out the
ConfigurationDirectorydirective in thefwupd.serviceunit. This will only fix the fwupd package, but it is the easiest and potentially least contentious option if 1 or 2 are not doable options. It was verified in the upstream bug already to solve the issue.CC original reporter @Queuecumber and @hughsie
The text was updated successfully, but these errors were encountered: