New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Various dom0 update errors on 2017-04-18 #2760

Open
andrewdavidwong opened this Issue Apr 18, 2017 · 9 comments

Comments

Projects
None yet
5 participants
@andrewdavidwong
Member

andrewdavidwong commented Apr 18, 2017

Qubes OS version (e.g., R3.2):

R3.2


@marmarek: AFAIK, we don't really have a place for this sort of thing, so I'm just reporting the output here:

Qubes OS Repository for Dom0                     38 MB/s |  92 kB     00:00    

kernel.x86_64                     1000:4.4.55-11.pvops.qubes   qubes-dom0-cached
kernel-qubes-vm.x86_64            1000:4.4.55-11.pvops.qubes   qubes-dom0-cached
qubes-core-dom0.x86_64            3.2.13-1.fc23                qubes-dom0-cached
qubes-core-dom0-doc.noarch        3.2.13-1                     qubes-dom0-cached
qubes-gui-dom0.x86_64             3.2.10-1.fc23                qubes-dom0-cached
qubes-manager.x86_64              3.2.11-1.fc23                qubes-dom0-cached
qubes-mgmt-salt-dom0-qvm.noarch   3.2.1-1.fc23                 qubes-dom0-cached
qubes-utils.x86_64                3.2.4-1.fc23                 qubes-dom0-cached
qubes-utils-libs.x86_64           3.2.4-1.fc23                 qubes-dom0-cached
Redirecting to '/usr/bin/dnf --exclude= upgrade' (see 'man yum2dnf')

Qubes OS Repository for Dom0                     47 MB/s |  92 kB     00:00    
Dependencies resolved.
================================================================================
 Package              Arch   Version                    Repository         Size
================================================================================
Installing:
 kernel               x86_64 1000:4.4.55-11.pvops.qubes qubes-dom0-cached  39 M
 kernel-qubes-vm      x86_64 1000:4.4.55-11.pvops.qubes qubes-dom0-cached  43 M
Upgrading:
 qubes-core-dom0      x86_64 3.2.13-1.fc23              qubes-dom0-cached 329 k
 qubes-core-dom0-doc  noarch 3.2.13-1                   qubes-dom0-cached  26 k
 qubes-gui-dom0       x86_64 3.2.10-1.fc23              qubes-dom0-cached  67 k
 qubes-manager        x86_64 3.2.11-1.fc23              qubes-dom0-cached 1.8 M
 qubes-mgmt-salt-dom0-qvm
                      noarch 3.2.1-1.fc23               qubes-dom0-cached  36 k
 qubes-utils          x86_64 3.2.4-1.fc23               qubes-dom0-cached  25 k
 qubes-utils-libs     x86_64 3.2.4-1.fc23               qubes-dom0-cached  20 k
Removing:
 kernel               x86_64 1000:4.4.14-11.pvops.qubes @anaconda         168 M
 kernel-qubes-vm      x86_64 1000:4.4.14-11.pvops.qubes @anaconda         153 M

Transaction Summary
================================================================================
Install  2 Packages
Upgrade  7 Packages
Remove   2 Packages

Total size: 84 M
Is this ok [y/N]: y
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Upgrading   : qubes-utils-libs-3.2.4-1.fc23.x86_64                       1/18 
  Upgrading   : qubes-utils-3.2.4-1.fc23.x86_64                            2/18 
  Upgrading   : qubes-core-dom0-doc-3.2.13-1.noarch                        3/18 
  Upgrading   : qubes-core-dom0-3.2.13-1.fc23.x86_64                       4/18 
sed: can't read /etc/sysconfig/prelink: No such file or directory
Redirecting to /bin/systemctl start  xenstored.service
  Installing  : kernel-qubes-vm-1000:4.4.55-11.pvops.qubes.x86_64          5/18 
mke2fs 1.42.13 (17-May-2015)
  Upgrading   : qubes-gui-dom0-3.2.10-1.fc23.x86_64                                                                                                                                      6/18 
  Upgrading   : qubes-manager-3.2.11-1.fc23.x86_64                                                                                                                                       7/18 
  Upgrading   : qubes-mgmt-salt-dom0-qvm-3.2.1-1.fc23.noarch                                                                                                                             8/18 
  Installing  : kernel-1000:4.4.55-11.pvops.qubes.x86_64                                                                                                                                 9/18 
  Cleanup     : qubes-gui-dom0-3.2.9-1.fc23.x86_64                                                                                                                                      10/18 
Traceback (most recent call last):
  File "/bin/qvm-ls", line 287, in <module>
    main()
  File "/bin/qvm-ls", line 280, in main
    print s
IOError: [Errno 32] Broken pipe
This kernel version is used by at least one VM, cannot remove
error: %preun(kernel-qubes-vm-1000:4.4.14-11.pvops.qubes.x86_64) scriptlet failed, exit status 1
Error in PREUN scriptlet in rpm package kernel-qubes-vm
Error in PREUN scriptlet in rpm package kernel-qubes-vm
  Erasing     : kernel-qubes-vm-1000:4.4.14-11.pvops.qubes.x86_64                                                                                                                       11/18 
  Cleanup     : qubes-manager-3.2.10-1.fc23.x86_64                                                                                                                                      12/18 
  Cleanup     : qubes-core-dom0-3.2.12-1.fc23.x86_64                                                                                                                                    13/18 
  Cleanup     : qubes-core-dom0-doc-3.2.12-1.noarch                                                                                                                                     14/18 
  Cleanup     : qubes-mgmt-salt-dom0-qvm-3.2.0-1.fc23.noarch                                                                                                                            15/18 
  Cleanup     : qubes-utils-3.2.3-1.fc23.x86_64                                                                                                                                         16/18 
  Cleanup     : qubes-utils-libs-3.2.3-1.fc23.x86_64                                                                                                                                    17/18 
Generating grub configuration file ...
Found theme: /boot/grub2/themes/system/theme.txt
Found linux image: /boot/vmlinuz-4.4.55-11.pvops.qubes.x86_64
Found linux image: /boot/vmlinuz-4.4.38-11.pvops.qubes.x86_64
Found initrd image: /boot/initramfs-4.4.38-11.pvops.qubes.x86_64.img
Found linux image: /boot/vmlinuz-4.4.31-11.pvops.qubes.x86_64
Found initrd image: /boot/initramfs-4.4.31-11.pvops.qubes.x86_64.img
Found linux image: /boot/vmlinuz-4.4.14-11.pvops.qubes.x86_64
done
  Erasing     : kernel-1000:4.4.14-11.pvops.qubes.x86_64                                                                                                                                18/18 
Generating grub configuration file ...
Found theme: /boot/grub2/themes/system/theme.txt
Found linux image: /boot/vmlinuz-4.4.55-11.pvops.qubes.x86_64
Found initrd image: /boot/initramfs-4.4.55-11.pvops.qubes.x86_64.img
Found linux image: /boot/vmlinuz-4.4.38-11.pvops.qubes.x86_64
Found initrd image: /boot/initramfs-4.4.38-11.pvops.qubes.x86_64.img
Found linux image: /boot/vmlinuz-4.4.31-11.pvops.qubes.x86_64
Found initrd image: /boot/initramfs-4.4.31-11.pvops.qubes.x86_64.img
done
  Verifying   : kernel-1000:4.4.55-11.pvops.qubes.x86_64                                                                                                                                 1/18 
  Verifying   : kernel-qubes-vm-1000:4.4.55-11.pvops.qubes.x86_64                                                                                                                        2/18 
  Verifying   : qubes-core-dom0-3.2.13-1.fc23.x86_64                                                                                                                                     3/18 
  Verifying   : qubes-core-dom0-doc-3.2.13-1.noarch                                                                                                                                      4/18 
  Verifying   : qubes-gui-dom0-3.2.10-1.fc23.x86_64                                                                                                                                      5/18 
  Verifying   : qubes-manager-3.2.11-1.fc23.x86_64                                                                                                                                       6/18 
  Verifying   : qubes-mgmt-salt-dom0-qvm-3.2.1-1.fc23.noarch                                                                                                                             7/18 
  Verifying   : qubes-utils-3.2.4-1.fc23.x86_64                                                                                                                                          8/18 
  Verifying   : qubes-utils-libs-3.2.4-1.fc23.x86_64                                                                                                                                     9/18 
  Verifying   : kernel-1000:4.4.14-11.pvops.qubes.x86_64                                                                                                                                10/18 
  Verifying   : qubes-manager-3.2.10-1.fc23.x86_64                                                                                                                                      11/18 
  Verifying   : kernel-qubes-vm-1000:4.4.14-11.pvops.qubes.x86_64                                                                                                                       12/18 
  Verifying   : qubes-mgmt-salt-dom0-qvm-3.2.0-1.fc23.noarch                                                                                                                            13/18 
  Verifying   : qubes-utils-3.2.3-1.fc23.x86_64                                                                                                                                         14/18 
  Verifying   : qubes-utils-libs-3.2.3-1.fc23.x86_64                                                                                                                                    15/18 
  Verifying   : qubes-core-dom0-3.2.12-1.fc23.x86_64                                                                                                                                    16/18 
  Verifying   : qubes-core-dom0-doc-3.2.12-1.noarch                                                                                                                                     17/18 
  Verifying   : qubes-gui-dom0-3.2.9-1.fc23.x86_64                                                                                                                                      18/18 

Removed:
  kernel.x86_64 1000:4.4.14-11.pvops.qubes                                                  kernel-qubes-vm.x86_64 1000:4.4.14-11.pvops.qubes                                                 

Installed:
  kernel.x86_64 1000:4.4.55-11.pvops.qubes                                                  kernel-qubes-vm.x86_64 1000:4.4.55-11.pvops.qubes                                                 

Upgraded:
  qubes-core-dom0.x86_64 3.2.13-1.fc23                  qubes-core-dom0-doc.noarch 3.2.13-1          qubes-gui-dom0.x86_64 3.2.10-1.fc23           qubes-manager.x86_64 3.2.11-1.fc23         
  qubes-mgmt-salt-dom0-qvm.noarch 3.2.1-1.fc23          qubes-utils.x86_64 3.2.4-1.fc23              qubes-utils-libs.x86_64 3.2.4-1.fc23         

Complete!
Redirecting to '/usr/bin/dnf -q check-update' (see 'man yum2dnf')


Related issues:

Kernel bug already reported in #2757.

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Apr 18, 2017

Member

Traceback (most recent call last): File "/bin/qvm-ls", line 287, in main() File "/bin/qvm-ls", line 280, in main print s IOError: [Errno 32] Broken pipe

This one is probably safe to ignore, comes from qvm-ls -k|grep -q ... and grep -q terminates on first match.

This kernel version is used by at least one VM, cannot remove error:
%preun(kernel-qubes-vm-1000:4.4.14-11.pvops.qubes.x86_64) scriptlet failed, exit status 1
Error in PREUN scriptlet in rpm package kernel-qubes-vm
Error in PREUN scriptlet in rpm package kernel-qubes-vm
Erasing : kernel-qubes-vm-1000:4.4.14-11.pvops.qubes.x86_64

This is the real problem - even when PREUN script fails, rpm proceed to remove the package. Apparently this is fixed upstream (rpm-software-management/rpm@3eb469b), and the fix is part of rpm-4.13.0. But this package stuck in Fedora updates-testing (https://bodhi.fedoraproject.org/updates/FEDORA-2016-0906f64ec8) and fc23 hit EOL then. So we're left with broken rpm-4.13.0-rc1 :/
BTW it's interesting that "release candidate" version of critical system component was allowed, but actual stable release was hold.

Member

marmarek commented Apr 18, 2017

Traceback (most recent call last): File "/bin/qvm-ls", line 287, in main() File "/bin/qvm-ls", line 280, in main print s IOError: [Errno 32] Broken pipe

This one is probably safe to ignore, comes from qvm-ls -k|grep -q ... and grep -q terminates on first match.

This kernel version is used by at least one VM, cannot remove error:
%preun(kernel-qubes-vm-1000:4.4.14-11.pvops.qubes.x86_64) scriptlet failed, exit status 1
Error in PREUN scriptlet in rpm package kernel-qubes-vm
Error in PREUN scriptlet in rpm package kernel-qubes-vm
Erasing : kernel-qubes-vm-1000:4.4.14-11.pvops.qubes.x86_64

This is the real problem - even when PREUN script fails, rpm proceed to remove the package. Apparently this is fixed upstream (rpm-software-management/rpm@3eb469b), and the fix is part of rpm-4.13.0. But this package stuck in Fedora updates-testing (https://bodhi.fedoraproject.org/updates/FEDORA-2016-0906f64ec8) and fc23 hit EOL then. So we're left with broken rpm-4.13.0-rc1 :/
BTW it's interesting that "release candidate" version of critical system component was allowed, but actual stable release was hold.

@andrewdavidwong

This comment has been minimized.

Show comment
Hide comment
@andrewdavidwong

andrewdavidwong Apr 18, 2017

Member

What about:

sed: can't read /etc/sysconfig/prelink: No such file or directory
Redirecting to /bin/systemctl start  xenstored.service

Any problem there?

Member

andrewdavidwong commented Apr 18, 2017

What about:

sed: can't read /etc/sysconfig/prelink: No such file or directory
Redirecting to /bin/systemctl start  xenstored.service

Any problem there?

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Apr 19, 2017

Member

No, safe to ignore. I'll mute those messages.

Member

marmarek commented Apr 19, 2017

No, safe to ignore. I'll mute those messages.

@andrewdavidwong

This comment has been minimized.

Show comment
Hide comment
@andrewdavidwong

andrewdavidwong Apr 19, 2017

Member

No, safe to ignore. I'll mute those messages.

👍


Any idea how the other errors (including #2757) managed to land in stable? Do we still not have enough people testing dom0 updates?

Member

andrewdavidwong commented Apr 19, 2017

No, safe to ignore. I'll mute those messages.

👍


Any idea how the other errors (including #2757) managed to land in stable? Do we still not have enough people testing dom0 updates?

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Apr 19, 2017

Member

Any idea how the other errors (including #2757) managed to land in stable?

That problem apply only to systems where kernel version was manually set to some explicit value. By default it's "default" and follow newest installed version. That package was in testing for almost a month (QubesOS/updates-status#16).

Do we still not have enough people testing dom0 updates?

Apparently... Or maybe everyone testing updates had kernel set to "default"?
BTW what was the reason to set kernel version manually to some specific value in the first place? Some other broken update in the past?

Member

marmarek commented Apr 19, 2017

Any idea how the other errors (including #2757) managed to land in stable?

That problem apply only to systems where kernel version was manually set to some explicit value. By default it's "default" and follow newest installed version. That package was in testing for almost a month (QubesOS/updates-status#16).

Do we still not have enough people testing dom0 updates?

Apparently... Or maybe everyone testing updates had kernel set to "default"?
BTW what was the reason to set kernel version manually to some specific value in the first place? Some other broken update in the past?

@andrewdavidwong

This comment has been minimized.

Show comment
Hide comment
@andrewdavidwong

andrewdavidwong Apr 19, 2017

Member

BTW what was the reason to set kernel version manually to some specific value in the first place? Some other broken update in the past?

Could be. I don't recall ever intentionally changing it.

Member

andrewdavidwong commented Apr 19, 2017

BTW what was the reason to set kernel version manually to some specific value in the first place? Some other broken update in the past?

Could be. I don't recall ever intentionally changing it.

@mfc

This comment has been minimized.

Show comment
Hide comment
@mfc

mfc Apr 19, 2017

Member

BTW what was the reason to set kernel version manually to some specific value in the first place? Some other broken update in the past?

Could be. I don't recall ever intentionally changing it.

yeah I've never changed them myself.

Member

mfc commented Apr 19, 2017

BTW what was the reason to set kernel version manually to some specific value in the first place? Some other broken update in the past?

Could be. I don't recall ever intentionally changing it.

yeah I've never changed them myself.

@mvermaes

This comment has been minimized.

Show comment
Hide comment
@mvermaes

mvermaes Apr 19, 2017

In case it helps - I have never changed kernel from default either, but am using a custom template based on fedora-24-minimal. All AppVMs based on this fedora-24 template were not set to kernel 4.4.55-11 after rebooting for this latest dom0 update.

Using the procedure in #2757 to change them to the new default kernel corrected the issue.

The AppVMs I have that are based on the default fedora-23 template were using the new kernel already, these didn't need to be reconfigured.

In case it helps - I have never changed kernel from default either, but am using a custom template based on fedora-24-minimal. All AppVMs based on this fedora-24 template were not set to kernel 4.4.55-11 after rebooting for this latest dom0 update.

Using the procedure in #2757 to change them to the new default kernel corrected the issue.

The AppVMs I have that are based on the default fedora-23 template were using the new kernel already, these didn't need to be reconfigured.

@0spinboson

This comment has been minimized.

Show comment
Hide comment
@0spinboson

0spinboson Apr 21, 2017

Marek: "Apparently... Or maybe everyone testing updates had kernel set to "default"?"

In my case, I never noticed it before because I still had 4.4.11 installed alongside 4.8.12, so I never got any of the 4.4 updates (because they're ignored), and I was only confronted with the problem when I manually removed 4.4.11 yesterday.

0spinboson commented Apr 21, 2017

Marek: "Apparently... Or maybe everyone testing updates had kernel set to "default"?"

In my case, I never noticed it before because I still had 4.4.11 installed alongside 4.8.12, so I never got any of the 4.4 updates (because they're ignored), and I was only confronted with the problem when I manually removed 4.4.11 yesterday.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment