get rid of custom systemd maintainer script code (disableSystemdUnits)

[adrelanos]

Just saw, that you are using disableSystemdUnits for new developments:
https://github.com/nrgaway/qubes-mgmt-salt/blob/12fb0b6095a20ffb24051e630097697dd32baef8/debian/qubes-salt-config.postinst#L22

Please don't. At least not for new developments. I see now is a chance to prevent this getting introduced... And preventing of later discussion/testing if removal of this workaround breaks something. I would like to avoid this needless workaround from spreading "everywhere".

Yes, there is a bug in Debian.
deb-systemd-helper fails to enable systemd units when using 'WantedBy = ' with spaces:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=786418

But the solution is, "don't use spaces in systemd unit files".

There is really no need for custom systemd unit file enable/disable code. Debian jessie's debhelper auto generated code handles that just fine for loads of packages in the Debian repository. Same for Whonix packages that ship systemd unit files. We should not try to be special. If there is an issue, we should find the issue and fix it. Not apply a workaround on top.

Mid term, the custom systemd code (disableSystemdUnits...) should be removed everywhere.

@nrgaway

[nrgaway]

There is a reason to use it.

When overriding a systemd unit file, you need to first disable, then re-enable the unit file for the override to become active. Otherwise the override is not active.

[nrgaway]

There is a reason to use it.

When overriding a systemd unit file, you need to first disable, then re-enable the unit file for the override to become active. Otherwise the override is not active.

[nrgaway]

BTW, any solution needs to be compatible with Debian 7, 8 and Ubuntu Trusty, Utopic, Vivid + since they all share the same code base.

[nrgaway]

BTW, any solution needs to be compatible with Debian 7, 8 and Ubuntu Trusty, Utopic, Vivid + since they all share the same code base.

[adrelanos]

When overriding a systemd unit file, you need to first disable, then re-enable the unit file for the override to become active. Otherwise the override is not active.

What do you need to overwrite? salt-minion.service, salt-master.service are new inventions?

BTW, any solution needs to be compatible with Debian 7, 8 and Ubuntu Trusty, Utopic, Vivid + since they all share the same code base.

I guess this also works with Debian 7 (wheezy). Anyhow. Isn't it time to abolish Debian 7 (wheezy, oldstable)? Trusty, Utopic, Vivid are jessie based, the internet says. Most Ubuntu packages are imported from Debian. So that should also work.

Does someone else on the internet have a similar problem you're solving here?

[adrelanos]

When overriding a systemd unit file, you need to first disable, then re-enable the unit file for the override to become active. Otherwise the override is not active.

What do you need to overwrite? salt-minion.service, salt-master.service are new inventions?

BTW, any solution needs to be compatible with Debian 7, 8 and Ubuntu Trusty, Utopic, Vivid + since they all share the same code base.

I guess this also works with Debian 7 (wheezy). Anyhow. Isn't it time to abolish Debian 7 (wheezy, oldstable)? Trusty, Utopic, Vivid are jessie based, the internet says. Most Ubuntu packages are imported from Debian. So that should also work.

Does someone else on the internet have a similar problem you're solving here?

[nrgaway]

Debian 7 is still supported.

Take a look at the core-agent-linux debian and rpm maintainer scripts.

As far as I know, you need to disable and re-enable a unit file if overriding it. Think its always been that way. Overriding it as in including a unit file in /etc/systemd/system when one already exists in /lib/systemd/system and not using drop-ins.

When you disable the unit file, the systemd soft-links are deleted, and when you enable it again, the soft-links will then point to the new override.

Maybe deb-helper does that already? If so, would need to make sure it does for the other versions I mentioned as well :)

[nrgaway]

Debian 7 is still supported.

Take a look at the core-agent-linux debian and rpm maintainer scripts.

As far as I know, you need to disable and re-enable a unit file if overriding it. Think its always been that way. Overriding it as in including a unit file in /etc/systemd/system when one already exists in /lib/systemd/system and not using drop-ins.

When you disable the unit file, the systemd soft-links are deleted, and when you enable it again, the soft-links will then point to the new override.

Maybe deb-helper does that already? If so, would need to make sure it does for the other versions I mentioned as well :)

[adrelanos]

Maybe deb-helper does that already?

I don't know, but it's not needed, because you're using a more difficult overriding mechanism than necessary. With the easier drop-in files solution, you don't need systemctl enable / systemctl disable. More below.

As far as I know, you need to disable and re-enable a unit file if overriding it.

But in this qubes-mgmt-salt case there is nothing you need to overwrite. salt-minion.service, salt-master.service are new inventions? You're upstream? Why not just put them into the usual location /lib/systemd/system as an upstream?

Take a look at the core-agent-linux debian and rpm maintainer scripts.

Overriding it as in including a unit file in /etc/systemd/system when one already exists in /lib/systemd/system and not using drop-ins.

Let's take for example /etc/systemd/system/ModemManager.service.
(https://github.com/QubesOS/qubes-core-agent-linux/blob/fc04408c7a3c7a28c7f0d4ed4dc24e8bc3b4dc03/vm-systemd/ModemManager.service)

.include /lib/systemd/system/ModemManager.service
[Unit]
ConditionPathExists=|/var/run/qubes-service/network-manager
ConditionPathExists=|/var/run/qubes-service/modem-manager

This .include /lib/systemd/system/ModemManager.service is not needed.

Rather you would create a drop-in file /etc/systemd/system/ModemManager.service.d/40_qubes.conf with the following content:

[Unit]
ConditionPathExists=|/var/run/qubes-service/network-manager
ConditionPathExists=|/var/run/qubes-service/modem-manager

Actually, we figured out this earlier already. Using that drop-in file solution for example for qubes-whonix's /etc/systemd/system/tor.service.d/40_qubes.conf. (https://github.com/nrgaway/qubes-whonix/blob/Whonix11/etc/systemd/system/tor.service.d/40_qubes.conf)

---

drop-in files take effect after running (1) sudo systemctl daemon-reload + (2) sudo service daemon-name restart [OR (3) reboot.] --> Conclusion: For the core-agent-linux package, you don't need systemctl enable / systemctl disable code for such drop-in files in the Debian maintainer postinst script.

[adrelanos]

Maybe deb-helper does that already?

I don't know, but it's not needed, because you're using a more difficult overriding mechanism than necessary. With the easier drop-in files solution, you don't need systemctl enable / systemctl disable. More below.

As far as I know, you need to disable and re-enable a unit file if overriding it.

But in this qubes-mgmt-salt case there is nothing you need to overwrite. salt-minion.service, salt-master.service are new inventions? You're upstream? Why not just put them into the usual location /lib/systemd/system as an upstream?

Take a look at the core-agent-linux debian and rpm maintainer scripts.

Overriding it as in including a unit file in /etc/systemd/system when one already exists in /lib/systemd/system and not using drop-ins.

Let's take for example /etc/systemd/system/ModemManager.service.
(https://github.com/QubesOS/qubes-core-agent-linux/blob/fc04408c7a3c7a28c7f0d4ed4dc24e8bc3b4dc03/vm-systemd/ModemManager.service)

.include /lib/systemd/system/ModemManager.service
[Unit]
ConditionPathExists=|/var/run/qubes-service/network-manager
ConditionPathExists=|/var/run/qubes-service/modem-manager

This .include /lib/systemd/system/ModemManager.service is not needed.

Rather you would create a drop-in file /etc/systemd/system/ModemManager.service.d/40_qubes.conf with the following content:

[Unit]
ConditionPathExists=|/var/run/qubes-service/network-manager
ConditionPathExists=|/var/run/qubes-service/modem-manager

Actually, we figured out this earlier already. Using that drop-in file solution for example for qubes-whonix's /etc/systemd/system/tor.service.d/40_qubes.conf. (https://github.com/nrgaway/qubes-whonix/blob/Whonix11/etc/systemd/system/tor.service.d/40_qubes.conf)

---

drop-in files take effect after running (1) sudo systemctl daemon-reload + (2) sudo service daemon-name restart [OR (3) reboot.] --> Conclusion: For the core-agent-linux package, you don't need systemctl enable / systemctl disable code for such drop-in files in the Debian maintainer postinst script.

[nrgaway]

I do think using dropins is a great idea. I am willing to take a look at converting existing code to use drop-ins. Fedora may also benefit from this. My only point is I don't want something to break that is currently working. The current Debian code was based on Fedora code, and there are some actions I do not understand why they exist, but implemented it to mirror Fedora code as there must have been a reason to in the first place.

Lots of testing will need to be performed to make sure the drops-ins works as expected for all distribution versions (Debian 7, 8, Ubuntu Trusty, Utopic, Vivid+), since for some services it is essential the state of the unit file is activated immediately. I always build all 16+ template variations and test them when making core changes just to make sure everything still works and that takes a few days to complete; just the testing stage.

qubes-mgmt-salt service unit files are overrides. Package unit files are in /lib/systemd/system, overrides in /etc/systemd/system. These will be changing as I am moving everything around as I won't be needing them at all, only need to make sure salt-minion is disabled, dead. I expect to be able to commit the re-factoring of packages today I hope, just working on a RPM packaging glitch where a few packages are not being created for some reason.

You mention systemctl daemon-reload / restart; does deb-helper preform these actions or are they added to maintainer scripts?

And what about making a package dead, making sure its disabled?

[nrgaway]

I do think using dropins is a great idea. I am willing to take a look at converting existing code to use drop-ins. Fedora may also benefit from this. My only point is I don't want something to break that is currently working. The current Debian code was based on Fedora code, and there are some actions I do not understand why they exist, but implemented it to mirror Fedora code as there must have been a reason to in the first place.

Lots of testing will need to be performed to make sure the drops-ins works as expected for all distribution versions (Debian 7, 8, Ubuntu Trusty, Utopic, Vivid+), since for some services it is essential the state of the unit file is activated immediately. I always build all 16+ template variations and test them when making core changes just to make sure everything still works and that takes a few days to complete; just the testing stage.

qubes-mgmt-salt service unit files are overrides. Package unit files are in /lib/systemd/system, overrides in /etc/systemd/system. These will be changing as I am moving everything around as I won't be needing them at all, only need to make sure salt-minion is disabled, dead. I expect to be able to commit the re-factoring of packages today I hope, just working on a RPM packaging glitch where a few packages are not being created for some reason.

You mention systemctl daemon-reload / restart; does deb-helper preform these actions or are they added to maintainer scripts?

And what about making a package dead, making sure its disabled?

[adrelanos]

Glad you like it.

I didn't know you're not the original creator of salt-minion.service, salt-master.service.

The current Debian code was based on Fedora code, and there are some actions I do not understand why they exist

I would suggest to keep asking questions until you understand up to a point where you reach a dead. When there is no other way, then it's time for a workaround. But if you go for workarounds too early without fully understanding the issue, you create more effort/issues/confusion later one. Imho.

---

And what about making a package dead, making sure its disabled?

Can you please rephrase this question? You mean completely disabling the systemd unit? If that's your question, yes. Actually, we're are using that approach already here:
https://github.com/nrgaway/qubes-whonix/blob/Whonix11/etc/systemd/system/rads.service.d/40_qubes.conf

You mention systemctl daemon-reload / restart; does deb-helper preform these actions or are they added to maintainer scripts?

Yes and no.

case a), upstream:

If you're upstream, i.e. if you invented new systemd unit files... Then debhelper will do all this stuff just fine. No need for custom systemd code then.

FYI: This is the usual snippet that debhelper adds when using debian/rules with dh $@ --with systemd.

# Automatically added by dh_systemd_enable
# This will only remove masks created by d-s-h on package removal.
deb-systemd-helper unmask whonixcheck.service >/dev/null || true

# was-enabled defaults to true, so new installations run enable.
if deb-systemd-helper --quiet was-enabled whonixcheck.service; then
    # Enables the unit on first installation, creates new
    # symlinks on upgrades if the unit file has changed.
    deb-systemd-helper enable whonixcheck.service >/dev/null || true
else
    # Update the statefile to add new symlinks (if any), which need to be
    # cleaned up on purge. Also remove old symlinks.
    deb-systemd-helper update-state whonixcheck.service >/dev/null || true
fi
# End automatically added section
# Automatically added by dh_systemd_start
if [ -d /run/systemd/system ]; then
    systemctl --system daemon-reload >/dev/null || true
    deb-systemd-invoke start whonixcheck.service >/dev/null || true
fi
# End automatically added section

If you're upstream (original systemd units) this works reliable in all cases (install, uninstall, upgrade) because lots of eyes were on that code and they also did lots of testing. (Loads of packages from official Debian repository are doing this.) So it's most robust and simple to go their default way.

---

case b) drop-in files overrides:

I don't know that yet. Maybe(!) we need to manually run 'systemctl daemon-reload' + restart service. That would still be a lot simpler than currently. One moment please. We'll figure that out before proceeding further. Asked on the Debian systemd mailing list.
Can debhelper help making override files take effect?
Link will appear later on the archive. Will later on post the link, communicate with the list further as required and share the conclusions here.

[adrelanos]

Glad you like it.

I didn't know you're not the original creator of salt-minion.service, salt-master.service.

The current Debian code was based on Fedora code, and there are some actions I do not understand why they exist

I would suggest to keep asking questions until you understand up to a point where you reach a dead. When there is no other way, then it's time for a workaround. But if you go for workarounds too early without fully understanding the issue, you create more effort/issues/confusion later one. Imho.

---

And what about making a package dead, making sure its disabled?

Can you please rephrase this question? You mean completely disabling the systemd unit? If that's your question, yes. Actually, we're are using that approach already here:
https://github.com/nrgaway/qubes-whonix/blob/Whonix11/etc/systemd/system/rads.service.d/40_qubes.conf

You mention systemctl daemon-reload / restart; does deb-helper preform these actions or are they added to maintainer scripts?

Yes and no.

case a), upstream:

If you're upstream, i.e. if you invented new systemd unit files... Then debhelper will do all this stuff just fine. No need for custom systemd code then.

FYI: This is the usual snippet that debhelper adds when using debian/rules with dh $@ --with systemd.

# Automatically added by dh_systemd_enable
# This will only remove masks created by d-s-h on package removal.
deb-systemd-helper unmask whonixcheck.service >/dev/null || true

# was-enabled defaults to true, so new installations run enable.
if deb-systemd-helper --quiet was-enabled whonixcheck.service; then
    # Enables the unit on first installation, creates new
    # symlinks on upgrades if the unit file has changed.
    deb-systemd-helper enable whonixcheck.service >/dev/null || true
else
    # Update the statefile to add new symlinks (if any), which need to be
    # cleaned up on purge. Also remove old symlinks.
    deb-systemd-helper update-state whonixcheck.service >/dev/null || true
fi
# End automatically added section
# Automatically added by dh_systemd_start
if [ -d /run/systemd/system ]; then
    systemctl --system daemon-reload >/dev/null || true
    deb-systemd-invoke start whonixcheck.service >/dev/null || true
fi
# End automatically added section

If you're upstream (original systemd units) this works reliable in all cases (install, uninstall, upgrade) because lots of eyes were on that code and they also did lots of testing. (Loads of packages from official Debian repository are doing this.) So it's most robust and simple to go their default way.

---

case b) drop-in files overrides:

I don't know that yet. Maybe(!) we need to manually run 'systemctl daemon-reload' + restart service. That would still be a lot simpler than currently. One moment please. We'll figure that out before proceeding further. Asked on the Debian systemd mailing list.
Can debhelper help making override files take effect?
Link will appear later on the archive. Will later on post the link, communicate with the list further as required and share the conclusions here.

[adrelanos]

Can debhelper help making override files take effect?:
https://lists.alioth.debian.org/pipermail/pkg-systemd-maintainers/2015-July/007999.html

No answers yet. Will keep you posted.

[adrelanos]

Can debhelper help making override files take effect?:
https://lists.alioth.debian.org/pipermail/pkg-systemd-maintainers/2015-July/007999.html

No answers yet. Will keep you posted.

[marmarek]

I really like that drop-in approach instead of rather ugly .include
approach. Also in Fedora. Since Fedora 18 is long EOL (where such
drop-ins wasn't supported), I think we can simply convert all the VMs
packages.

Also you can ignore daemon reload - the template VM will be shut down
just after upgrade anyway.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

I really like that drop-in approach instead of rather ugly .include
approach. Also in Fedora. Since Fedora 18 is long EOL (where such
drop-ins wasn't supported), I think we can simply convert all the VMs
packages.

Also you can ignore daemon reload - the template VM will be shut down
just after upgrade anyway.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[adrelanos]

Glad you like it also!

Also you can ignore daemon reload - the template VM will be shut down just after upgrade anyway.

This is very attractive. Simplicity. Would mean no maintainer script code. But... What about support for standalone VMs that are upgraded? To support not having to reboot those after upgrading, should we care about the 'daemon reload'*?

*systemctl daemon-reload / restart service

[adrelanos]

Glad you like it also!

Also you can ignore daemon reload - the template VM will be shut down just after upgrade anyway.

This is very attractive. Simplicity. Would mean no maintainer script code. But... What about support for standalone VMs that are upgraded? To support not having to reboot those after upgrading, should we care about the 'daemon reload'*?

*systemctl daemon-reload / restart service

[marmarek]

On Mon, Jul 06, 2015 at 03:42:53PM -0700, Patrick Schleizer wrote:

Glad you like it also!

Also you can ignore daemon reload - the template VM will be shut down just after upgrade anyway.

This is very attractive. Simplicity. Would mean no maintainer script code. But... What about support for standalone VMs that are upgraded? To support not having to reboot those after upgrading, should we care about the 'daemon reload'*?

• systemctl daemon-reload / restart service

Those overrides are mainly to prevent starting the service in particular
VM type. So if the VM is already running, there is probably no point in
restarting such services.
But of course daemon-reload is a good idea.

Just one more thing - currently the real reastart-less upgrade isn't
fully possible, because qrexec-agent can't be restarted (once
terminated, dom0 part will also terminate, so no further connections
will be possible). Similar case for gui-agent - its restart means also
restart of X server in the VM - so all the windows will be closed.
But other than that, other updates theoretically are possible with VM
restart.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

On Mon, Jul 06, 2015 at 03:42:53PM -0700, Patrick Schleizer wrote:

Glad you like it also!

Also you can ignore daemon reload - the template VM will be shut down just after upgrade anyway.

This is very attractive. Simplicity. Would mean no maintainer script code. But... What about support for standalone VMs that are upgraded? To support not having to reboot those after upgrading, should we care about the 'daemon reload'*?

• systemctl daemon-reload / restart service

Those overrides are mainly to prevent starting the service in particular
VM type. So if the VM is already running, there is probably no point in
restarting such services.
But of course daemon-reload is a good idea.

Just one more thing - currently the real reastart-less upgrade isn't
fully possible, because qrexec-agent can't be restarted (once
terminated, dom0 part will also terminate, so no further connections
will be possible). Similar case for gui-agent - its restart means also
restart of X server in the VM - so all the windows will be closed.
But other than that, other updates theoretically are possible with VM
restart.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[adrelanos]

I see. So you tell me if it's either:

• a) worth implementing restart-less upgrading with respect to systemd drop-in files with this ticket or,
• b) keep this as future work, when it comes up, who knows when, if ever, or
• c) worth going for simplicity (ignore daemon reload) (no maintainer script code required for this)

I am in favor of c), because a) would not be complete anyhow and because we got enough tasks on the plate.

[adrelanos]

I see. So you tell me if it's either:

• a) worth implementing restart-less upgrading with respect to systemd drop-in files with this ticket or,
• b) keep this as future work, when it comes up, who knows when, if ever, or
• c) worth going for simplicity (ignore daemon reload) (no maintainer script code required for this)

I am in favor of c), because a) would not be complete anyhow and because we got enough tasks on the plate.

[nrgaway]

I will put this on my TODO list to take care of within the next couple of weeks. I also have some other things to take care of in core-agent as well #1051.

[nrgaway]

I will put this on my TODO list to take care of within the next couple of weeks. I also have some other things to take care of in core-agent as well #1051.

[marmarek]

On Mon, Jul 06, 2015 at 04:05:08PM -0700, Patrick Schleizer wrote:

I see. So you tell me if it's either:

• a) worth implementing restart-less upgrading with respect to systemd drop-in files with this ticket or,
• b) keep this as future work, when it comes up, who knows when, if ever, or
• c) worth going for simplicity (ignore daemon reload) (no maintainer script code required for this)

I am in favor of c), because a) would not be complete anyhow and because we got enough tasks on the plate.

Me too for c).

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

On Mon, Jul 06, 2015 at 04:05:08PM -0700, Patrick Schleizer wrote:

I see. So you tell me if it's either:

• a) worth implementing restart-less upgrading with respect to systemd drop-in files with this ticket or,
• b) keep this as future work, when it comes up, who knows when, if ever, or
• c) worth going for simplicity (ignore daemon reload) (no maintainer script code required for this)

I am in favor of c), because a) would not be complete anyhow and because we got enough tasks on the plate.

Me too for c).

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[adrelanos]

A related issue, that follows from this:
Debian Template: qubes-core-agent.postinst causes very slow package installation
#1065

[adrelanos]

A related issue, that follows from this:
Debian Template: qubes-core-agent.postinst causes very slow package installation
#1065

[adrelanos]

FYI to keep you posted.

Can debhelper help making override files take effect?:
https://lists.alioth.debian.org/pipermail/pkg-systemd-maintainers/2015-July/007999.html

No answers yet. Will keep you posted.

Answer: "no, not yet".

Posted a feature request...
help making systemd drop-in overwrite files take effect:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=793416

[adrelanos]

FYI to keep you posted.

Can debhelper help making override files take effect?:
https://lists.alioth.debian.org/pipermail/pkg-systemd-maintainers/2015-July/007999.html

No answers yet. Will keep you posted.

Answer: "no, not yet".

Posted a feature request...
help making systemd drop-in overwrite files take effect:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=793416

[nrgaway]

I pretty much have the drop-ins complete (https://github.com/marmarek/qubes-core-agent-linux/compare/master...nrgaway:r3-dropins?expand=1), just doing some final testing.

Most of the custom systemd maintainer script code is gone, although I needed to keep some of it since deb-helper does not yet support systemd preset. It appears that they will though at some point.

I fixed the speed of updating as well with another commit that manages the triggers for the xdg autostart configuration files for both Debian and Fedora (nrgaway/qubes-core-agent-linux@c616331)

[nrgaway]

I pretty much have the drop-ins complete (https://github.com/marmarek/qubes-core-agent-linux/compare/master...nrgaway:r3-dropins?expand=1), just doing some final testing.

Most of the custom systemd maintainer script code is gone, although I needed to keep some of it since deb-helper does not yet support systemd preset. It appears that they will though at some point.

I fixed the speed of updating as well with another commit that manages the triggers for the xdg autostart configuration files for both Debian and Fedora (nrgaway/qubes-core-agent-linux@c616331)

[adrelanos]

Please check lintian before/after the package changes, so no new lintian warnings are introduced.

I am not sure systemd drop-in files in this case belong into /etc/systemd/... or /lib/systemd/... I suspect the latter. lintian would complain. In any case, since drop-in files are hierarchy bases, users can overwrite those without having their settings reverted on upgrade / package manager dpkg interactive conflict resolution dialog.

Most of the custom systemd maintainer script code is gone, although I needed to keep some of it since deb-helper does not yet support systemd preset.

I don't understand this one.

Any upstream bug report / feature request / discussion? I think it would be worth adding to the function comments, so we can follow the progress, so any workarounds can be removed later on.

What would happen without function systemdPreload? Or asked the other way around, what does function systemdPreload do?

I was looking at /lib/systemd/system-preset/75-qubes-vm.preset. I think for Debian all the enable lines are superfluous. On Debian, all services are enabled and started right after installation anyhow. But most likely you need this for Fedora?

As for the disable lines, wouldn't those be better implemented as systemd drop-in files also? That way, the amount of systemd related work arounds would be reduced to zero?

[adrelanos]

Please check lintian before/after the package changes, so no new lintian warnings are introduced.

I am not sure systemd drop-in files in this case belong into /etc/systemd/... or /lib/systemd/... I suspect the latter. lintian would complain. In any case, since drop-in files are hierarchy bases, users can overwrite those without having their settings reverted on upgrade / package manager dpkg interactive conflict resolution dialog.

Most of the custom systemd maintainer script code is gone, although I needed to keep some of it since deb-helper does not yet support systemd preset.

I don't understand this one.

Any upstream bug report / feature request / discussion? I think it would be worth adding to the function comments, so we can follow the progress, so any workarounds can be removed later on.

What would happen without function systemdPreload? Or asked the other way around, what does function systemdPreload do?

I was looking at /lib/systemd/system-preset/75-qubes-vm.preset. I think for Debian all the enable lines are superfluous. On Debian, all services are enabled and started right after installation anyhow. But most likely you need this for Fedora?

As for the disable lines, wouldn't those be better implemented as systemd drop-in files also? That way, the amount of systemd related work arounds would be reduced to zero?

[marmarek]

I was looking at /lib/systemd/system-preset/75-qubes-vm.preset. I think for Debian all the enable lines are superfluous. On Debian, all services are enabled and started right after installation anyhow. But most likely you need this for Fedora?

Yes, on Fedora all services are not enabled automatically after install.

[marmarek]

I was looking at /lib/systemd/system-preset/75-qubes-vm.preset. I think for Debian all the enable lines are superfluous. On Debian, all services are enabled and started right after installation anyhow. But most likely you need this for Fedora?

Yes, on Fedora all services are not enabled automatically after install.

[nrgaway]

On 24 July 2015 at 12:47, Patrick Schleizer notifications@github.com
wrote:

Please check lintian before/after the package changes, so no new lintian
warnings are introduced.

I don't have lintian set up for Qubes. Feel free to set something up if
you wish that works with qubes-builder.

I am not sure systemd drop-in files in this case belong into
/etc/systemd/... or /lib/systemd/... I suspect the latter. lintian would
complain. In any case, since drop-in files are hierarchy bases, users can
overwrite those without having their settings reverted on upgrade / package
manager dpkg interactive conflict resolution dialog.

Drop-ins are in /etc since they are over-riding package that is not written
by us. Users can still override by placing there own .conf in same
directory.

Most of the custom systemd maintainer script code is gone, although I

needed to keep some of it since deb-helper does not yet support systemd
preset.

I don't understand this one.

The problem with Debian systemd deb-helper is that by default they are
enabling every service on install when they should be using preset. Qubes
now uses presets to list files that should be enabled / disabled in initial
installation and preset is now the recommended method of maybe enabling a
service instead of 'enable .

Any upstream bug report / feature request / discussion? I think it would
be worth adding to the function comments, so we can follow the progress, so
any workarounds can be removed later on.

Debian systemd deb-helper maintainers are aware of the issue. Have a look
at this topic:
http://comments.gmane.org/gmane.comp.sysutils.systemd.devel/25165

What would happen without function systemdPreload? Or asked the other way

around, what does function systemdPreload do?

Without it unwanted services would be enabled and would require the manual
list of systemd disables.

I was looking at /lib/systemd/system-preset/75-qubes-vm.preset
https://github.com/marmarek/qubes-core-agent-linux/blob/master/vm-systemd/75-qubes-vm.preset.
I think for Debian all the enable lines
https://github.com/marmarek/qubes-core-agent-linux/blob/285071bd59887adf4aa52e91e8c09ec81cbb962e/vm-systemd/75-qubes-vm.preset#L55-72
are superfluous. On Debian, all services are enabled and started right
after installation anyhow. But most likely you need this for Fedora?

Correct.

As for the disable lines
https://github.com/marmarek/qubes-core-agent-linux/blob/285071bd59887adf4aa52e91e8c09ec81cbb962e/vm-systemd/75-qubes-vm.preset#L19-53,
wouldn't those be better implemented as systemd drop-in files also? That
way, the amount of systemd related work around would be reduced to zero?

I don't think drop-ins are appropriate in this case; that's what preset is
for. The drop-ins are appropriate when there is a condition where it may
be enabled, but not to completely disable.

It does not take much time to issue the presets either :)

[nrgaway]

On 24 July 2015 at 12:47, Patrick Schleizer notifications@github.com
wrote:

Please check lintian before/after the package changes, so no new lintian
warnings are introduced.

I don't have lintian set up for Qubes. Feel free to set something up if
you wish that works with qubes-builder.

I am not sure systemd drop-in files in this case belong into
/etc/systemd/... or /lib/systemd/... I suspect the latter. lintian would
complain. In any case, since drop-in files are hierarchy bases, users can
overwrite those without having their settings reverted on upgrade / package
manager dpkg interactive conflict resolution dialog.

Drop-ins are in /etc since they are over-riding package that is not written
by us. Users can still override by placing there own .conf in same
directory.

Most of the custom systemd maintainer script code is gone, although I

needed to keep some of it since deb-helper does not yet support systemd
preset.

I don't understand this one.

The problem with Debian systemd deb-helper is that by default they are
enabling every service on install when they should be using preset. Qubes
now uses presets to list files that should be enabled / disabled in initial
installation and preset is now the recommended method of maybe enabling a
service instead of 'enable .

Any upstream bug report / feature request / discussion? I think it would
be worth adding to the function comments, so we can follow the progress, so
any workarounds can be removed later on.

Debian systemd deb-helper maintainers are aware of the issue. Have a look
at this topic:
http://comments.gmane.org/gmane.comp.sysutils.systemd.devel/25165

What would happen without function systemdPreload? Or asked the other way

around, what does function systemdPreload do?

Without it unwanted services would be enabled and would require the manual
list of systemd disables.

I was looking at /lib/systemd/system-preset/75-qubes-vm.preset
https://github.com/marmarek/qubes-core-agent-linux/blob/master/vm-systemd/75-qubes-vm.preset.
I think for Debian all the enable lines
https://github.com/marmarek/qubes-core-agent-linux/blob/285071bd59887adf4aa52e91e8c09ec81cbb962e/vm-systemd/75-qubes-vm.preset#L55-72
are superfluous. On Debian, all services are enabled and started right
after installation anyhow. But most likely you need this for Fedora?

Correct.

As for the disable lines
https://github.com/marmarek/qubes-core-agent-linux/blob/285071bd59887adf4aa52e91e8c09ec81cbb962e/vm-systemd/75-qubes-vm.preset#L19-53,
wouldn't those be better implemented as systemd drop-in files also? That
way, the amount of systemd related work around would be reduced to zero?

I don't think drop-ins are appropriate in this case; that's what preset is
for. The drop-ins are appropriate when there is a condition where it may
be enabled, but not to completely disable.

It does not take much time to issue the presets either :)

[adrelanos]

lintian, the Debian package sanity checker... It's just a tool to prevent messing up common mistakes that cause confusion, support requests and headaches later on. No need to automate/script it at this point.

Just for manual use. It's just sudo apt-get install lintian. Plus from the package source root folder lintian --info --display-info --pedantic. That's it. Or lintian --info --display-info --pedantic /path/to/deb. Not all have a huge impact, but why not prevent introduction of additional issues.

It does not take much time to issue the presets either :)

Not concerned about speed here. More about inventing custom workarounds that have clean/standard/policy conform solutions. I'll learn more about this.

[adrelanos]

lintian, the Debian package sanity checker... It's just a tool to prevent messing up common mistakes that cause confusion, support requests and headaches later on. No need to automate/script it at this point.

Just for manual use. It's just sudo apt-get install lintian. Plus from the package source root folder lintian --info --display-info --pedantic. That's it. Or lintian --info --display-info --pedantic /path/to/deb. Not all have a huge impact, but why not prevent introduction of additional issues.

It does not take much time to issue the presets either :)

Not concerned about speed here. More about inventing custom workarounds that have clean/standard/policy conform solutions. I'll learn more about this.

[marmarek]

On Fri, Jul 24, 2015 at 10:46:27AM -0700, nrgaway wrote:

On 24 July 2015 at 12:47, Patrick Schleizer notifications@github.com
wrote:

I am not sure systemd drop-in files in this case belong into
/etc/systemd/... or /lib/systemd/... I suspect the latter. lintian would
complain. In any case, since drop-in files are hierarchy bases, users can
overwrite those without having their settings reverted on upgrade / package
manager dpkg interactive conflict resolution dialog.

Drop-ins are in /etc since they are over-riding package that is not written
by us. Users can still override by placing there own .conf in same
directory.

Still, I think the files installed by some package should go into
(/usr?)/lib/systemd. This way we may freely modify those files in the future
without worrying about user changes. Files in /etc by definition can be
modified by the user.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

On Fri, Jul 24, 2015 at 10:46:27AM -0700, nrgaway wrote:

On 24 July 2015 at 12:47, Patrick Schleizer notifications@github.com
wrote:

I am not sure systemd drop-in files in this case belong into
/etc/systemd/... or /lib/systemd/... I suspect the latter. lintian would
complain. In any case, since drop-in files are hierarchy bases, users can
overwrite those without having their settings reverted on upgrade / package
manager dpkg interactive conflict resolution dialog.

Drop-ins are in /etc since they are over-riding package that is not written
by us. Users can still override by placing there own .conf in same
directory.

Still, I think the files installed by some package should go into
(/usr?)/lib/systemd. This way we may freely modify those files in the future
without worrying about user changes. Files in /etc by definition can be
modified by the user.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[adrelanos]

Difficult to figure out the cleanest solution here. We're clearly not upstream, but also not a system administrator. Being a derivative. Both places have pros and cons.

/etc's advantage: Users could easily delete that file and be done with it.

/lib's advantage: Easier in case of updates. Clearly unsupported location for user edits. Users can use /etc.

I also think that what the unit.service.d, i.e. /lib/systemd/system/unit.service.d/40_qubes.conf folders are for. Who else would be supposed to use /lib/systemd/system/unit.service.d/...? I don't know any precedences, unfortunately.

Most likely lintian would complain about using /etc...:
https://lintian.debian.org/tags/systemd-service-file-outside-lib.html

[adrelanos]

Difficult to figure out the cleanest solution here. We're clearly not upstream, but also not a system administrator. Being a derivative. Both places have pros and cons.

/etc's advantage: Users could easily delete that file and be done with it.

/lib's advantage: Easier in case of updates. Clearly unsupported location for user edits. Users can use /etc.

I also think that what the unit.service.d, i.e. /lib/systemd/system/unit.service.d/40_qubes.conf folders are for. Who else would be supposed to use /lib/systemd/system/unit.service.d/...? I don't know any precedences, unfortunately.

Most likely lintian would complain about using /etc...:
https://lintian.debian.org/tags/systemd-service-file-outside-lib.html

[marmarek]

On Fri, Jul 24, 2015 at 04:55:13PM -0700, Patrick Schleizer wrote:

Difficult to figure out the cleanest solution here. We're clearly not upstream, but also not a system administrator. Being a derivative. Both places have pros and cons.

/etc's advantage: Users could easily delete that file and be done with it.

The file will be restored after upgrade, wouldn't be?

/lib's advantage: Easier in case of updates. Clearly unsupported location for user edits. Users can use /etc.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

On Fri, Jul 24, 2015 at 04:55:13PM -0700, Patrick Schleizer wrote:

Difficult to figure out the cleanest solution here. We're clearly not upstream, but also not a system administrator. Being a derivative. Both places have pros and cons.

/etc's advantage: Users could easily delete that file and be done with it.

The file will be restored after upgrade, wouldn't be?

/lib's advantage: Easier in case of updates. Clearly unsupported location for user edits. Users can use /etc.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[nrgaway]

On 24 July 2015 at 19:55, Patrick Schleizer notifications@github.com
wrote:

Difficult to figure out the cleanest solution here. We're clearly not
upstream, but also not a system administrator. Being a derivative. Both
places have pros and cons.

/etc's advantage: Users could easily delete that file and be done with it.

/lib's advantage: Easier in case of updates. Clearly unsupported location
for user edits. Users can use /etc.

I also think that what the unit.service.d, i.e.
/lib/systemd/system/unit.service.d/40_qubes.conf folders are for. Who
else would be supposed to use /lib/systemd/system/unit.service.d/...? I
don't know any precedences, unfortunately.

They for the distro, but I guess we can consider the template a custom
distro, since it's not exactly 100% Debian; it's modified for use with
Qubes.

/lib/systemd/system has another advantage; it's not considered a
configuration directory and therefor we won't have conflicts on updates if
uses modify something in /etc/systemd/system.

I suppose I should also modify the postinst to disable static unit files
like colord in the lib directory as well then

Most likely lintian would complain about using /etc...:
https://lintian.debian.org/tags/systemd-service-file-outside-lib.html

—
Reply to this email directly or view it on GitHub
#1043 (comment)
.

[nrgaway]

On 24 July 2015 at 19:55, Patrick Schleizer notifications@github.com
wrote:

Difficult to figure out the cleanest solution here. We're clearly not
upstream, but also not a system administrator. Being a derivative. Both
places have pros and cons.

/etc's advantage: Users could easily delete that file and be done with it.

/lib's advantage: Easier in case of updates. Clearly unsupported location
for user edits. Users can use /etc.

I also think that what the unit.service.d, i.e.
/lib/systemd/system/unit.service.d/40_qubes.conf folders are for. Who
else would be supposed to use /lib/systemd/system/unit.service.d/...? I
don't know any precedences, unfortunately.

They for the distro, but I guess we can consider the template a custom
distro, since it's not exactly 100% Debian; it's modified for use with
Qubes.

/lib/systemd/system has another advantage; it's not considered a
configuration directory and therefor we won't have conflicts on updates if
uses modify something in /etc/systemd/system.

I suppose I should also modify the postinst to disable static unit files
like colord in the lib directory as well then

Most likely lintian would complain about using /etc...:
https://lintian.debian.org/tags/systemd-service-file-outside-lib.html

—
Reply to this email directly or view it on GitHub
#1043 (comment)
.

[nrgaway]

On 24 July 2015 at 20:11, Marek Marczykowski-Górecki <
notifications@github.com> wrote:

On Fri, Jul 24, 2015 at 04:55:13PM -0700, Patrick Schleizer wrote:

Difficult to figure out the cleanest solution here. We're clearly not
upstream, but also not a system administrator. Being a derivative. Both
places have pros and cons.

/etc's advantage: Users could easily delete that file and be done with
it.

The file will be restored after upgrade, wouldn't be?

The /etc dir is a conf dir, so most likely user would be prompted to
over-write configuration on upgrade

/lib's advantage: Easier in case of updates. Clearly unsupported
location for user edits. Users can use /etc.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

—
Reply to this email directly or view it on GitHub
#1043 (comment)
.

[nrgaway]

On 24 July 2015 at 20:11, Marek Marczykowski-Górecki <
notifications@github.com> wrote:

On Fri, Jul 24, 2015 at 04:55:13PM -0700, Patrick Schleizer wrote:

Difficult to figure out the cleanest solution here. We're clearly not
upstream, but also not a system administrator. Being a derivative. Both
places have pros and cons.

/etc's advantage: Users could easily delete that file and be done with
it.

The file will be restored after upgrade, wouldn't be?

The /etc dir is a conf dir, so most likely user would be prompted to
over-write configuration on upgrade

/lib's advantage: Easier in case of updates. Clearly unsupported
location for user edits. Users can use /etc.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

—
Reply to this email directly or view it on GitHub
#1043 (comment)
.

[adrelanos]

Marek Marczykowski-Górecki:

The file will be restored after upgrade, wouldn't be?

No. Not on Debian. Also deletion of a config file in /etc is something
that Debian package management honors. You can test this by deleting a
config file from etc, 'apt-get remove' it, 'apt-get install' it again.
Config file still gone. (To restore the config file using the package
management, you would have to force it forget about config files.
'apt-get install' it followed by 'apt-get purge' it.

[adrelanos]

Marek Marczykowski-Górecki:

The file will be restored after upgrade, wouldn't be?

No. Not on Debian. Also deletion of a config file in /etc is something
that Debian package management honors. You can test this by deleting a
config file from etc, 'apt-get remove' it, 'apt-get install' it again.
Config file still gone. (To restore the config file using the package
management, you would have to force it forget about config files.
'apt-get install' it followed by 'apt-get purge' it.

[adrelanos]

As an exception: if the file was changed, i.e. new config file defaults by upstream while the user deleted it from etc, then maybe, the user would get an interactive dpkg conflict resolution dialog. Not sure. I could test this if this was important for something.

[adrelanos]

As an exception: if the file was changed, i.e. new config file defaults by upstream while the user deleted it from etc, then maybe, the user would get an interactive dpkg conflict resolution dialog. Not sure. I could test this if this was important for something.

[marmarek]

@nrgaway what is the status of this ticket? Is the branch r3-dropins ready for merge? I really want this change before final 3.0, because it will make 3.0 maintenance much easier.

[marmarek]

@nrgaway what is the status of this ticket? Is the branch r3-dropins ready for merge? I really want this change before final 3.0, because it will make 3.0 maintenance much easier.

[nrgaway]

On 2 August 2015 at 14:39, Marek Marczykowski-Górecki <
notifications@github.com> wrote:

@nrgaway https://github.com/nrgaway what is the status of this ticket?
Is the branch r3-dropins ready for merge? I really want this change
before final 3.0, because it will make 3.0 maintenance much easier.

I have been testing for a week. Everything seems fine for jessie,
fedora-21. There is also a r3-xdg-config branch I am testing with that
adds triggers for fedora to update .desktop files and automatically create
triggers for debian / fedora based on one configuration file to simplify
maintenance,

There was an issue with wheezy and fedroa-20. Both do not support
'preset-all'; fedora-20 therefore has no qubes services enabled. I have
made changes to Wheezy to allow for that, but not the Fedora .spec file
yet. Want me to change spec so it works with fc-20? I just add a few line
to the loop where it force deletes by also setting preset for items with
enable..

Also I was noticing in Fedora (and seen other users mention this on current
version) that 'Syncing app menu' trigger hangs often, where you need to
<CTRL+C> to continue with yum update. I do not know what causes this.
Even running /usr/lib/qubes/qrexec-client-vm dom0 qubes.SyncAppMenus /bin/sh /etc/qubes-rpc/qubes.GetAppmenus will not respond at times, just
hangs. Any ideas?

I can commit my final changes by tomorrow. I will commit r2 branches once
you are satisfied :)

[nrgaway]

On 2 August 2015 at 14:39, Marek Marczykowski-Górecki <
notifications@github.com> wrote:

@nrgaway https://github.com/nrgaway what is the status of this ticket?
Is the branch r3-dropins ready for merge? I really want this change
before final 3.0, because it will make 3.0 maintenance much easier.

I have been testing for a week. Everything seems fine for jessie,
fedora-21. There is also a r3-xdg-config branch I am testing with that
adds triggers for fedora to update .desktop files and automatically create
triggers for debian / fedora based on one configuration file to simplify
maintenance,

There was an issue with wheezy and fedroa-20. Both do not support
'preset-all'; fedora-20 therefore has no qubes services enabled. I have
made changes to Wheezy to allow for that, but not the Fedora .spec file
yet. Want me to change spec so it works with fc-20? I just add a few line
to the loop where it force deletes by also setting preset for items with
enable..

Also I was noticing in Fedora (and seen other users mention this on current
version) that 'Syncing app menu' trigger hangs often, where you need to
<CTRL+C> to continue with yum update. I do not know what causes this.
Even running /usr/lib/qubes/qrexec-client-vm dom0 qubes.SyncAppMenus /bin/sh /etc/qubes-rpc/qubes.GetAppmenus will not respond at times, just
hangs. Any ideas?

I can commit my final changes by tomorrow. I will commit r2 branches once
you are satisfied :)

[nrgaway]

I am having an issue with Wheezy; route is not being set in Template (Never tested it as AppVM due to this problem). eth0 network interface is set up and qubes-misc-post.service starts without an error. I can manually restart qubes-misc-post.service or run setup-ip after Template starts and then route works.

I am going to re-build Wheezy using current master branch of core-agent-linux to see if I introduced this issue with the updates I have been working on; otherwise Jessie, fc20 and fc21 are working. I been working on finishing it up all night, and will get back to it after some rest later tonight.

[nrgaway]

I am having an issue with Wheezy; route is not being set in Template (Never tested it as AppVM due to this problem). eth0 network interface is set up and qubes-misc-post.service starts without an error. I can manually restart qubes-misc-post.service or run setup-ip after Template starts and then route works.

I am going to re-build Wheezy using current master branch of core-agent-linux to see if I introduced this issue with the updates I have been working on; otherwise Jessie, fc20 and fc21 are working. I been working on finishing it up all night, and will get back to it after some rest later tonight.

[nrgaway]

Got everything working; just doing one last round of build tests; PR should be ready within a few hours

[nrgaway]

Got everything working; just doing one last round of build tests; PR should be ready within a few hours

[marmarek]

Implemented in (merged) https://github.com/marmarek/qubes-core-agent-linux/pull/11
I think we don't want to backport this to R2, as this hardly qualifies as a bugfix.

[marmarek]

Implemented in (merged) https://github.com/marmarek/qubes-core-agent-linux/pull/11
I think we don't want to backport this to R2, as this hardly qualifies as a bugfix.

[adrelanos]

Yes. I would also advice against backport. Better making R3 release.

[adrelanos]

Yes. I would also advice against backport. Better making R3 release.