run qubes-sysinit.service and qubes-mount-dirs.service systemd services before most other systemd services

[adrelanos]

I start with one specific issue and then ask for a general solution.

This is currently affecting Qubes-Whonix with R3.2 and causing lots of issues if not fixed.

---

One specific issue...

/lib/systemd/system/crond.service.d/30_qubes.conf

Was:

# For /rw
After=qubes-misc-post.service

Was changed to:

# For /rw
After=qubes-misc-post.service qubes-sysinit.service

I do not see how qubes-misc-post.service helps with /rw. Perhaps it is outdated? It should be After=qubes-mount-dirs.service? I did not create a pull request, because I do not know if After=qubes-misc-post.service is still required?

---

The general issue...

Since lots of systemd services need to start after qubes-sysinit.service and after qubes-mount-dirs.service... I think a systemd drop-in file for each individual systemd service declaring an After=qubes-sysinit.service is non-ideal.

The same goes for the qubes-mount-dirs.service. For Qubes-Whonix, I would have to add to qubes-mount-dirs.service:

Before=tor.service
Before=tor@default.service
Before=rinetd.service
Before=control-port-filter-python.service
Before=qubes-whonix-firewall.service
Before=whonixcheck.service
Before=sdwdate.service
Before=anon-ws-disable-stacked-tor.service

Which is quite a lot. I again could use lots of systemd drop-in files declaring After=qubes-mount-dirs.service but it seems wrong to me.

Could we somehow configure qubes-sysinit.service and qubes-mount-dirs.service to run/finish before most other services run so we do not have to use drop-ins for individual, never exhaustive lists of systemd services?

[marmarek]

On Sat, Jul 23, 2016 at 10:11:36AM -0700, Patrick Schleizer wrote:

Was changed to:

Hmm, this commit looks wrong, as dependency on qubes-sysinit.service is
already there implicitly (as part of sysinit.target).

I do not see how qubes-misc-post.service helps with /rw. Perhaps it is outdated? It should be After=qubes-mount-dirs.service? I did not create a pull request, because I do not know if After=qubes-misc-post.service is still required?

I think the best solution would be to add Before=local-fs.target to
qubes-mount-dirs.service. Then all the services requiring /home and/or
/rw being mounted should depend on local-fs.target.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

On Sat, Jul 23, 2016 at 10:11:36AM -0700, Patrick Schleizer wrote:

Was changed to:

Hmm, this commit looks wrong, as dependency on qubes-sysinit.service is
already there implicitly (as part of sysinit.target).

I do not see how qubes-misc-post.service helps with /rw. Perhaps it is outdated? It should be After=qubes-mount-dirs.service? I did not create a pull request, because I do not know if After=qubes-misc-post.service is still required?

I think the best solution would be to add Before=local-fs.target to
qubes-mount-dirs.service. Then all the services requiring /home and/or
/rw being mounted should depend on local-fs.target.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[adrelanos]

Marek Marczykowski-Górecki:

On Sat, Jul 23, 2016 at 10:11:36AM -0700, Patrick Schleizer wrote:

I do not see how qubes-misc-post.service helps with /rw. Perhaps it is outdated? It should be After=qubes-mount-dirs.service? I did not create a pull request, because I do not know if After=qubes-misc-post.service is still required?

I think the best solution would be to add Before=local-fs.target to
qubes-mount-dirs.service. Then all the services requiring /home and/or
/rw being mounted should depend on local-fs.target.

This sounds good. Please lets go for this for R3.2 / next
qubes-core-agent version.

However, does not work yet. I tried qubes-mount-dirs.service with
'Before=local-fs.target' this on my local system.

qubes-db.service fails and as a follow up issue, then
qubes-mount-dirs.service also fails [cannot use qubesdb-read].

sudo journalctl -u qubes-db
-- Logs begin at Sat 2016-07-23 20:46:07 UTC, end at Sat 2016-07-23
20:49:28 UTC. --
Jul 23 20:46:07 host systemd[1]: Starting Qubes DB agent...
Jul 23 20:46:07 host qubesdb-daemon[297]: xc: error: Could not obtain
handle on privileged command interface (2 = No such file or directory):
Internal error
Jul 23 20:46:07 host qubesdb-daemon[297]: FATAL: vchan initialization failed
Jul 23 20:46:07 host systemd[1]: qubes-db.service: main process exited,
code=exited, status=1/FAILURE
Jul 23 20:46:07 host systemd[1]: Failed to start Qubes DB agent.
Jul 23 20:46:07 host systemd[1]: Unit qubes-db.service entered failed state.

'qubes-db.service' uses 'After=local-fs.target'

'qubes-mount-dirs.service' uses 'Before=local-fs.target' but needs
'qubes-db.service'

Can this contradiction be resolved?

[adrelanos]

Marek Marczykowski-Górecki:

On Sat, Jul 23, 2016 at 10:11:36AM -0700, Patrick Schleizer wrote:

I do not see how qubes-misc-post.service helps with /rw. Perhaps it is outdated? It should be After=qubes-mount-dirs.service? I did not create a pull request, because I do not know if After=qubes-misc-post.service is still required?

I think the best solution would be to add Before=local-fs.target to
qubes-mount-dirs.service. Then all the services requiring /home and/or
/rw being mounted should depend on local-fs.target.

This sounds good. Please lets go for this for R3.2 / next
qubes-core-agent version.

However, does not work yet. I tried qubes-mount-dirs.service with
'Before=local-fs.target' this on my local system.

qubes-db.service fails and as a follow up issue, then
qubes-mount-dirs.service also fails [cannot use qubesdb-read].

sudo journalctl -u qubes-db
-- Logs begin at Sat 2016-07-23 20:46:07 UTC, end at Sat 2016-07-23
20:49:28 UTC. --
Jul 23 20:46:07 host systemd[1]: Starting Qubes DB agent...
Jul 23 20:46:07 host qubesdb-daemon[297]: xc: error: Could not obtain
handle on privileged command interface (2 = No such file or directory):
Internal error
Jul 23 20:46:07 host qubesdb-daemon[297]: FATAL: vchan initialization failed
Jul 23 20:46:07 host systemd[1]: qubes-db.service: main process exited,
code=exited, status=1/FAILURE
Jul 23 20:46:07 host systemd[1]: Failed to start Qubes DB agent.
Jul 23 20:46:07 host systemd[1]: Unit qubes-db.service entered failed state.

'qubes-db.service' uses 'After=local-fs.target'

'qubes-mount-dirs.service' uses 'Before=local-fs.target' but needs
'qubes-db.service'

Can this contradiction be resolved?

[marmarek]

Try edit qubes-db.service and change After=local-fs.target to just
After=proc-xen.mount

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

Try edit qubes-db.service and change After=local-fs.target to just
After=proc-xen.mount

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[adrelanos]

This works better. No failed systemd services. However, sys-whonix stays yellow in QVMM. qrrexec broken. There is still a systemd issue:

[    1.505405] systemd[1]: Found ordering cycle on basic.target/start
[    1.505415] systemd[1]: Found dependency on paths.target/start
[    1.505422] systemd[1]: Found dependency on acpid.path/start
[    1.505430] systemd[1]: Found dependency on sysinit.target/start
[    1.505437] systemd[1]: Found dependency on local-fs.target/start
[    1.505445] systemd[1]: Found dependency on qubes-mount-dirs.service/start
[    1.505452] systemd[1]: Found dependency on basic.target/start
[    1.505459] systemd[1]: Breaking ordering cycle by deleting job paths.target/start
[    1.505470] systemd[1]: Job paths.target/start deleted to break ordering cycle starting with basic.target/start
[ SKIP ] Ordering cycle found, skipping Paths
[    1.505592] systemd[1]: Found ordering cycle on basic.target/start
[    1.505598] systemd[1]: Found dependency on sysinit.target/start
[    1.505603] systemd[1]: Found dependency on local-fs.target/start
[    1.505608] systemd[1]: Found dependency on qubes-mount-dirs.service/start
[    1.505613] systemd[1]: Found dependency on basic.target/start
[    1.505618] systemd[1]: Breaking ordering cycle by deleting job local-fs.target/start
[    1.505624] systemd[1]: Job local-fs.target/start deleted to break ordering cycle starting with basic.target/start
[ SKIP ] Ordering cycle found, skipping Local File Systems
[    1.506031] systemd[1]: Expecting device dev-hvc0.device...
         Expecting device dev-hvc0.device...
[    1.506082] systemd[1]: Starting Forward Password Requests to Wall Directory Watch.
[    1.506139] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[    1.506151] systemd[1]: Starting Remote File Systems (Pre).

[adrelanos]

This works better. No failed systemd services. However, sys-whonix stays yellow in QVMM. qrrexec broken. There is still a systemd issue:

[    1.505405] systemd[1]: Found ordering cycle on basic.target/start
[    1.505415] systemd[1]: Found dependency on paths.target/start
[    1.505422] systemd[1]: Found dependency on acpid.path/start
[    1.505430] systemd[1]: Found dependency on sysinit.target/start
[    1.505437] systemd[1]: Found dependency on local-fs.target/start
[    1.505445] systemd[1]: Found dependency on qubes-mount-dirs.service/start
[    1.505452] systemd[1]: Found dependency on basic.target/start
[    1.505459] systemd[1]: Breaking ordering cycle by deleting job paths.target/start
[    1.505470] systemd[1]: Job paths.target/start deleted to break ordering cycle starting with basic.target/start
[ SKIP ] Ordering cycle found, skipping Paths
[    1.505592] systemd[1]: Found ordering cycle on basic.target/start
[    1.505598] systemd[1]: Found dependency on sysinit.target/start
[    1.505603] systemd[1]: Found dependency on local-fs.target/start
[    1.505608] systemd[1]: Found dependency on qubes-mount-dirs.service/start
[    1.505613] systemd[1]: Found dependency on basic.target/start
[    1.505618] systemd[1]: Breaking ordering cycle by deleting job local-fs.target/start
[    1.505624] systemd[1]: Job local-fs.target/start deleted to break ordering cycle starting with basic.target/start
[ SKIP ] Ordering cycle found, skipping Local File Systems
[    1.506031] systemd[1]: Expecting device dev-hvc0.device...
         Expecting device dev-hvc0.device...
[    1.506082] systemd[1]: Starting Forward Password Requests to Wall Directory Watch.
[    1.506139] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[    1.506151] systemd[1]: Starting Remote File Systems (Pre).

[marmarek]

I'm working towards ordering qubes-mount-dirs.service before local-fs.target. So any service requiring all local filesystems being already mounted could rely on standard After=local-fs.target. I'm not sure, but I think it is implicitly part of DefaultDependencies=yes.

[marmarek]

I'm working towards ordering qubes-mount-dirs.service before local-fs.target. So any service requiring all local filesystems being already mounted could rely on standard After=local-fs.target. I'm not sure, but I think it is implicitly part of DefaultDependencies=yes.

[marmarek]

@adrelanos please review if package version dependencies in marmarek/qubes-core-qubesdb@da8b81f are correct (updated xen package will be soon)

[marmarek]

@adrelanos please review if package version dependencies in marmarek/qubes-core-qubesdb@da8b81f are correct (updated xen package will be soon)

[marmarek]

Automated announcement from builder-github

The package qubes-core-agent_3.2.9-1+deb8u1 has been pushed to the r3.2 testing repository for the Debian jessie template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing jessie-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-agent_3.2.9-1+deb8u1 has been pushed to the r3.2 testing repository for the Debian jessie template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing jessie-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-agent_3.2.9-1+deb9u1 has been pushed to the r3.2 testing repository for the Debian stretch template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing stretch-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-agent_3.2.9-1+deb9u1 has been pushed to the r3.2 testing repository for the Debian stretch template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing stretch-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.2.9-1.fc23 has been pushed to the r3.2 testing repository for the Fedora fc23 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r3.2-current-testing

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.2.9-1.fc23 has been pushed to the r3.2 testing repository for the Fedora fc23 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r3.2-current-testing

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.2.9-1.fc24 has been pushed to the r3.2 testing repository for the Fedora fc24 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r3.2-current-testing

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.2.9-1.fc24 has been pushed to the r3.2 testing repository for the Fedora fc24 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r3.2-current-testing

Changes included in this update

[adrelanos]

Marek Marczykowski-Górecki:

@adrelanos please review if package version dependencies in marmarek/qubes-core-qubesdb@da8b81f are correct (updated xen package will be soon)

I am only capable of practical testing of these Qubes internals. My
first impression is very good. This solves most if not all issues.

[adrelanos]

Marek Marczykowski-Górecki:

@adrelanos please review if package version dependencies in marmarek/qubes-core-qubesdb@da8b81f are correct (updated xen package will be soon)

I am only capable of practical testing of these Qubes internals. My
first impression is very good. This solves most if not all issues.

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.2.9-1.fc23 has been pushed to the r3.2 stable repository for the Fedora fc23 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.2.9-1.fc23 has been pushed to the r3.2 stable repository for the Fedora fc23 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.2.9-1.fc24 has been pushed to the r3.2 stable repository for the Fedora fc24 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.2.9-1.fc24 has been pushed to the r3.2 stable repository for the Fedora fc24 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-agent_3.2.10-1+deb8u1 has been pushed to the r3.2 stable repository for the Debian jessie template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-agent_3.2.10-1+deb8u1 has been pushed to the r3.2 stable repository for the Debian jessie template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.1.19-1.fc21 has been pushed to the r3.1 testing repository for the Fedora fc21 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r3.1-current-testing

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.1.19-1.fc21 has been pushed to the r3.1 testing repository for the Fedora fc21 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r3.1-current-testing

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.1.19-1.fc22 has been pushed to the r3.1 testing repository for the Fedora fc22 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r3.1-current-testing

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.1.19-1.fc22 has been pushed to the r3.1 testing repository for the Fedora fc22 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r3.1-current-testing

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.1.19-1.fc23 has been pushed to the r3.1 testing repository for the Fedora fc23 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r3.1-current-testing

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.1.19-1.fc23 has been pushed to the r3.1 testing repository for the Fedora fc23 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r3.1-current-testing

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-agent_3.1.19-1+deb8u1 has been pushed to the r3.1 testing repository for the Debian jessie template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing jessie-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-agent_3.1.19-1+deb8u1 has been pushed to the r3.1 testing repository for the Debian jessie template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing jessie-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-agent_3.1.19-1+deb9u1 has been pushed to the r3.1 testing repository for the Debian stretch template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing stretch-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-agent_3.1.19-1+deb9u1 has been pushed to the r3.1 testing repository for the Debian stretch template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing stretch-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-agent_3.1.19-1+deb7u1 has been pushed to the r3.1 testing repository for the Debian wheezy template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing wheezy-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-agent_3.1.19-1+deb7u1 has been pushed to the r3.1 testing repository for the Debian wheezy template.
To test this update, first enable the testing repository in /etc/apt/sources.list.d/qubes-*.list by uncommenting the line containing wheezy-testing, then use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.1.19-1.fc21 has been pushed to the r3.1 stable repository for the Fedora fc21 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.1.19-1.fc21 has been pushed to the r3.1 stable repository for the Fedora fc21 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.1.19-1.fc22 has been pushed to the r3.1 stable repository for the Fedora fc22 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.1.19-1.fc22 has been pushed to the r3.1 stable repository for the Fedora fc22 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.1.19-1.fc23 has been pushed to the r3.1 stable repository for the Fedora fc23 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package python2-dnf-plugins-qubes-hooks-3.1.19-1.fc23 has been pushed to the r3.1 stable repository for the Fedora fc23 template.
To install this update, please use the standard update command:

sudo yum update

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-agent_3.1.19-1+deb8u1 has been pushed to the r3.1 stable repository for the Debian jessie template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-agent_3.1.19-1+deb8u1 has been pushed to the r3.1 stable repository for the Debian jessie template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-agent_3.1.19-1+deb9u1 has been pushed to the r3.1 stable repository for the Debian stretch template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-agent_3.1.19-1+deb9u1 has been pushed to the r3.1 stable repository for the Debian stretch template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-agent_3.1.19-1+deb7u1 has been pushed to the r3.1 stable repository for the Debian wheezy template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-agent_3.1.19-1+deb7u1 has been pushed to the r3.1 stable repository for the Debian wheezy template.
To install this update, please use the standard update command:

sudo apt-get update && sudo apt-get dist-upgrade

Changes included in this update