Debian 9 runs apt update on startup, blocks user apt commands #2621

tasket · Feb 6, 2017

Qubes OS version (e.g., `R3.2`):

R3.2

Affected TemplateVMs (e.g., `fedora-23`, if applicable):

Debian 9 stretch

Expected behavior:

User can run updates immediately after starting template (or appVM)

Actual behavior:

Apt-related files are locked, and sometimes this takes more than 5 min to clear.

Steps to reproduce the behavior:

Start Debian 9 template and issue sudo apt-get update command.

General notes:

Stretch has a new systemd unit called apt-daily.timer and this activates apt-daily.service on most boots. I think systemctl disable apt-daily.timer is working (so far) to prevent activation.

It seems in Qubes' case it would be a good idea to permanently disable this timer for templates.

I caution of removing the timer. /lib/systemd/system/apt-daily.service runs /usr/lib/apt/apt.systemd.daily. I would not wonder if it does or will do in later releases of Debian other stuff than unattended upgrades (database stuff that was previously done using cron).

When you sudo apt-get purge unattended-upgrades, apt.systemd.daily will exit within a second. I recommend uninstalling it in Qubes.

Another option is to disable unattended-upgrades using apt.conf.d. See:
/etc/apt/apt.conf.d/90nounattended
(Please slightly modify that file name should you decide to use that in Qubes to prevent conflcits with the pkg-manager-no-autoupdate package by Whonix.)

adrelanos · Feb 6, 2017

I caution of removing the timer. /lib/systemd/system/apt-daily.service runs /usr/lib/apt/apt.systemd.daily. I would not wonder if it does or will do in later releases of Debian other stuff than unattended upgrades (database stuff that was previously done using cron).

When you sudo apt-get purge unattended-upgrades, apt.systemd.daily will exit within a second. I recommend uninstalling it in Qubes.

Another option is to disable unattended-upgrades using apt.conf.d. See:
/etc/apt/apt.conf.d/90nounattended
(Please slightly modify that file name should you decide to use that in Qubes to prevent conflcits with the pkg-manager-no-autoupdate package by Whonix.)

When I disabled the timer, I assumed that apt could take care of any delayed housekeeping when it is called manually. This is 'traditional', I think, and would seem to be necessary; otherwise there are side-effects like having the Qubes template update procedure fail.

The package description "unattended-upgrades" says it is for enabling full download and installation of packages... this sounds pretty specific, and I wonder if that would stop apt from running on boot (it could still do updates without download/upgrade?).

Regardless, it seems you have already addressed this in Whonix. Best thing may be to just copy your conf file approach.

tasket · Feb 6, 2017

When I disabled the timer, I assumed that apt could take care of any delayed housekeeping when it is called manually. This is 'traditional', I think, and would seem to be necessary; otherwise there are side-effects like having the Qubes template update procedure fail.

The package description "unattended-upgrades" says it is for enabling full download and installation of packages... this sounds pretty specific, and I wonder if that would stop apt from running on boot (it could still do updates without download/upgrade?).

Regardless, it seems you have already addressed this in Whonix. Best thing may be to just copy your conf file approach.

What package pulls in `unattended-upgrades`? Anyway, conf approach looks fine.

…

-- Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?

marmarek · Feb 6, 2017

What package pulls in `unattended-upgrades`? Anyway, conf approach looks fine.

…

-- Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing?

The package description "unattended-upgrades" says it is for enabling full download and installation of packages... this sounds pretty specific, and I wonder if that would stop apt from running on boot

Works for me. No more auto run of apt-get update after that package had been purged.

Anyway, conf approach looks fine.

That.

Or install the https://github.com/Whonix/pkg-manager-no-autoupdate package.
(Related to Package security-misc from Whonix to Qubes #1885)

adrelanos · Feb 6, 2017

The package description "unattended-upgrades" says it is for enabling full download and installation of packages... this sounds pretty specific, and I wonder if that would stop apt from running on boot

Works for me. No more auto run of apt-get update after that package had been purged.

Anyway, conf approach looks fine.

That.

Or install the https://github.com/Whonix/pkg-manager-no-autoupdate package.
(Related to Package security-misc from Whonix to Qubes #1885)

This isn't a bug, and certainly not Qubes specific.

What package pulls in unattended-upgrades?

It's a recommended package from (at root) gnome-packagekit. Users chose to install it on the upgrade.
Either turn off installation of recommends, or review the packages to be installed before upgrading. (Both are advisable in any case.)

@adrelanos suggests purging the package if it's been installed in error. Definitely this, rather than disabling the timer.
If Debian moves to installing and enabling this package by default then we may want to review, although having Templates automatically updating with security patches when they are started may be desirable in Qubes too.

@andrewdavidwong I'd suggest closing.

unman · Feb 8, 2017

This isn't a bug, and certainly not Qubes specific.

What package pulls in unattended-upgrades?

It's a recommended package from (at root) gnome-packagekit. Users chose to install it on the upgrade.
Either turn off installation of recommends, or review the packages to be installed before upgrading. (Both are advisable in any case.)

@adrelanos suggests purging the package if it's been installed in error. Definitely this, rather than disabling the timer.
If Debian moves to installing and enabling this package by default then we may want to review, although having Templates automatically updating with security patches when they are started may be desirable in Qubes too.

@andrewdavidwong I'd suggest closing.

This at very least needs to be documented for users who upgrade their packages. Otherwise most users will run into having that package installed and the issue described here. Enabling automatic updates by default is a separate discussion. Imo not an easy decision.

adrelanos · Feb 8, 2017

This at very least needs to be documented for users who upgrade their packages. Otherwise most users will run into having that package installed and the issue described here. Enabling automatic updates by default is a separate discussion. Imo not an easy decision.

Changed to a documentation task.

andrewdavidwong · Feb 8, 2017

Changed to a documentation task.

Suggestion: Move the new blurb about unattended-upgrades to managing-os/templates/debian/upgrade-8-to-9 under "Additional Information". If I were just starting to use Debian 9 templates (via upgrade process), that's where I'd hope to see it.

tasket · Feb 10, 2017

Suggestion: Move the new blurb about unattended-upgrades to managing-os/templates/debian/upgrade-8-to-9 under "Additional Information". If I were just starting to use Debian 9 templates (via upgrade process), that's where I'd hope to see it.

I just discovered the problem remains in one of my templates if unattended-upgrades is not installed but apt-daily.timer is enabled.

$ sudo apt-get update && sudo apt-get dist-upgrade
Reading package lists... Done
E: Could not get lock /var/lib/apt/lists/lock - open (11: Resource temporarily unavailable)
E: Unable to lock directory /var/lib/apt/lists/

$ ps aux |grep apt
root       564  0.0  0.2   4288  1504 ?        Ss   23:50   0:00 /bin/sh /usr/lib/apt/apt.systemd.daily
root       942  0.2  1.3  45476  7296 ?        S    23:50   0:00 apt-get -qq -y update
_apt       993  0.0  1.0  42144  5496 ?        S    23:50   0:00 /usr/lib/apt/methods/http
_apt       996  0.0  1.0  42144  5424 ?        S    23:50   0:00 /usr/lib/apt/methods/http
_apt       998  0.0  1.0  42144  5616 ?        S    23:50   0:00 /usr/lib/apt/methods/http
_apt      1314  0.0  1.0  41912  5476 ?        S    23:50   0:00 /usr/lib/apt/methods/gpgv
_apt      1514  0.0  1.0  42144  5576 ?        S    23:50   0:00 /usr/lib/apt/methods/http
user      1565  0.0  0.1  12720   960 pts/0    S+   23:50   0:00 grep apt

$ apt list unattended-upgrades
Listing... Done
unattended-upgrades/testing,now 0.93.1 all [residual-config]

$ sudo systemctl disable apt-daily.timer
Removed /etc/systemd/system/timers.target.wants/apt-daily.timer.

Another template has apt-daily.timer disabled and has not manifested the problem.

tasket · Feb 23, 2017

I just discovered the problem remains in one of my templates if unattended-upgrades is not installed but apt-daily.timer is enabled.

$ sudo apt-get update && sudo apt-get dist-upgrade
Reading package lists... Done
E: Could not get lock /var/lib/apt/lists/lock - open (11: Resource temporarily unavailable)
E: Unable to lock directory /var/lib/apt/lists/

$ ps aux |grep apt
root       564  0.0  0.2   4288  1504 ?        Ss   23:50   0:00 /bin/sh /usr/lib/apt/apt.systemd.daily
root       942  0.2  1.3  45476  7296 ?        S    23:50   0:00 apt-get -qq -y update
_apt       993  0.0  1.0  42144  5496 ?        S    23:50   0:00 /usr/lib/apt/methods/http
_apt       996  0.0  1.0  42144  5424 ?        S    23:50   0:00 /usr/lib/apt/methods/http
_apt       998  0.0  1.0  42144  5616 ?        S    23:50   0:00 /usr/lib/apt/methods/http
_apt      1314  0.0  1.0  41912  5476 ?        S    23:50   0:00 /usr/lib/apt/methods/gpgv
_apt      1514  0.0  1.0  42144  5576 ?        S    23:50   0:00 /usr/lib/apt/methods/http
user      1565  0.0  0.1  12720   960 pts/0    S+   23:50   0:00 grep apt

$ apt list unattended-upgrades
Listing... Done
unattended-upgrades/testing,now 0.93.1 all [residual-config]

$ sudo systemctl disable apt-daily.timer
Removed /etc/systemd/system/timers.target.wants/apt-daily.timer.

Another template has apt-daily.timer disabled and has not manifested the problem.

@tasket: Consider submitting a doc PR for that, if you think it appropriate.

andrewdavidwong · Feb 24, 2017

@tasket: Consider submitting a doc PR for that, if you think it appropriate.

Update: Debian appears to have changed apt-daily.timer to apt-daily.service, which had the side-effect of re-enabling it.

tasket · Apr 10, 2017

Update: Debian appears to have changed apt-daily.timer to apt-daily.service, which had the side-effect of re-enabling it.

Automated announcement from builder-github

The package core-agent-linux has been pushed to the r4.0 testing repository for the CentOS centos7 template.
To test this update, please install it with the following command:

sudo yum update --enablerepo=qubes-vm-r4.0-current-testing

QubesOS/qubes-issues

Join GitHub today

Debian 9 runs apt update on startup, blocks user apt commands #2621

Comments

tasket commented Feb 6, 2017

Qubes OS version (e.g., R3.2):

Affected TemplateVMs (e.g., fedora-23, if applicable):

Expected behavior:

Actual behavior:

Steps to reproduce the behavior:

General notes:

This comment has been minimized.

adrelanos commented Feb 6, 2017 • edited Edited 1 time adrelanos edited Feb 6, 2017 (most recent)

This comment has been minimized.

tasket commented Feb 6, 2017

andrewdavidwong added bug C: Debian labels Feb 6, 2017

andrewdavidwong added this to the Release 3.2 updates milestone Feb 6, 2017

This comment has been minimized.

marmarek commented Feb 6, 2017

This comment has been minimized.

adrelanos commented Feb 6, 2017

This comment has been minimized.

unman commented Feb 8, 2017

This comment has been minimized.

adrelanos commented Feb 8, 2017

andrewdavidwong added C: doc task and removed bug labels Feb 8, 2017

andrewdavidwong modified the milestones: Documentation/website, Release 3.2 updates Feb 8, 2017

This comment has been minimized.

andrewdavidwong commented Feb 8, 2017

unman referenced this issue in QubesOS/qubes-doc Feb 9, 2017

andrewdavidwong closed this in QubesOS/qubes-doc#280 Feb 10, 2017

andrewdavidwong added a commit to QubesOS/qubes-doc that referenced this issue Feb 10, 2017

This comment has been minimized.

tasket commented Feb 10, 2017

This comment has been minimized.

tasket commented Feb 23, 2017

This comment has been minimized.

andrewdavidwong commented Feb 24, 2017

This comment has been minimized.

tasket commented Apr 10, 2017

marmarek added a commit to marmarek/qubes-core-agent-linux that referenced this issue Oct 19, 2017

This comment has been minimized.

qubesos-bot commented Oct 19, 2017

qubesos-bot added the r4.0-centos7-cur-test label Oct 19, 2017

qubesos-bot referenced this issue in QubesOS/updates-status Oct 19, 2017

This comment has been minimized.

qubesos-bot commented Oct 19, 2017

qubesos-bot added the r4.0-jessie-cur-test label Oct 19, 2017

This comment has been minimized.

qubesos-bot commented Oct 19, 2017

qubesos-bot added the r4.0-fc24-cur-test label Oct 19, 2017

This comment has been minimized.

qubesos-bot commented Oct 19, 2017

qubesos-bot added the r4.0-stretch-cur-test label Oct 19, 2017

This comment has been minimized.

qubesos-bot commented Oct 19, 2017

qubesos-bot added the r4.0-fc25-cur-test label Oct 19, 2017

This comment has been minimized.

qubesos-bot commented Oct 19, 2017

qubesos-bot added the r4.0-fc26-cur-test label Oct 19, 2017

This comment has been minimized.

qubesos-bot commented Oct 30, 2017

qubesos-bot added r4.0-centos7-stable and removed r4.0-centos7-cur-test labels Oct 30, 2017

This comment has been minimized.

qubesos-bot commented Oct 30, 2017

qubesos-bot added r4.0-fc26-stable r4.0-jessie-stable and removed r4.0-jessie-cur-test labels Oct 30, 2017

This comment has been minimized.

qubesos-bot commented Oct 30, 2017

qubesos-bot added r4.0-stretch-stable and removed r4.0-stretch-cur-test labels Oct 30, 2017

marmarek added a commit to marmarek/qubes-core-agent-linux that referenced this issue Dec 13, 2017

This comment has been minimized.

qubesos-bot commented Dec 13, 2017

qubesos-bot added the r3.2-fc23-cur-test label Dec 13, 2017

qubesos-bot referenced this issue in QubesOS/updates-status Dec 13, 2017

This comment has been minimized.

qubesos-bot commented Dec 13, 2017

qubesos-bot added the r3.2-fc24-cur-test label Dec 13, 2017

This comment has been minimized.

qubesos-bot commented Dec 13, 2017

qubesos-bot added the r3.2-fc25-cur-test label Dec 13, 2017

Qubes OS version (e.g., `R3.2`):

Affected TemplateVMs (e.g., `fedora-23`, if applicable):

adrelanos commented Feb 6, 2017 •

edited

Edited 1 time

adrelanos edited Feb 6, 2017 (most recent)