/qubes-issues

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

4.0 rc1 (+?) VMs fail to start #3133

Closed
holiman opened this Issue Sep 28, 2017 · 2 comments

Comments

Assignees
No one assigned
Labels
Projects
None yet
Milestone
No milestone
2 participants
@holiman @marmarek
@holiman

holiman commented Sep 28, 2017

Qubes OS version (e.g., R3.2):

  • Started from qubes 4.0 rc1
  • enabled the dom0-testing, and did dom0-upgrade

Affected TemplateVMs (e.g., fedora-23, if applicable):

All, except sys-net and sys-usb

Problem description:

  • Upgrading dom0 with current-testing enabled solved my earlier issues (was not able to shutdown properly), but now most of my VMs fail to boot.
  • The only ones it manages to start are sys-net and sys-usb

Hardware: Librem 13v2, with 2x1Tb ssd-disks.

Below are various error messages from all around the place...

Errors from journalctl

Sep 28 21:26:42 dom0 qmemman.systemstate[18757]: stat: dom '1' act=3319648407 pref=403577241.6
Sep 28 21:26:42 dom0 qmemman.systemstate[18757]: stat: xenfree=767247241 memset_reqs=[('0', 12541718365), ('1', 3784985591)]
Sep 28 21:26:42 dom0 qmemman.systemstate[18757]: mem-set domain 0 to 12541718365
Sep 28 21:26:42 dom0 qmemman.systemstate[18757]: mem-set domain 1 to 3784985591
Sep 28 21:26:42 dom0 qrexec[16539]: qubes.InputKeyboard: sys-usb -> dom0: denied: no matching rule found
Sep 28 21:31:02 dom0 qrexec[16628]: qubes.NotifyUpdates: sys-net -> dom0: allowed to dom0
Sep 28 21:31:02 dom0 qmemman.daemon.algo[18757]: balance_when_enough_memory(xen_free_memory=19786246, total_mem_pref=1785784396.8000002, total_available_memory=14560705805.199999)
Sep 28 21:31:02 dom0 qmemman.systemstate[18757]: stat: dom '0' act=12541718365 pref=1382207155.2
Sep 28 21:31:02 dom0 qmemman.systemstate[18757]: stat: dom '1' act=3784985591 pref=403577241.6
Sep 28 21:31:02 dom0 qmemman.systemstate[18757]: stat: xenfree=72215046 memset_reqs=[('1', 3690520740), ('0', 12639622971)]
Sep 28 21:31:02 dom0 qmemman.systemstate[18757]: mem-set domain 1 to 3690520740
Sep 28 21:31:02 dom0 qmemman.systemstate[18757]: mem-set domain 0 to 12639622971
Sep 28 21:31:03 dom0 qmemman.daemon.algo[18757]: balance_when_enough_memory(xen_free_memory=15615508, total_mem_pref=1741093350.4, total_available_memory=14604665868.6)
Sep 28 21:31:03 dom0 qmemman.systemstate[18757]: stat: dom '0' act=12639622971 pref=1337516108.8000002
Sep 28 21:31:03 dom0 qmemman.systemstate[18757]: stat: dom '1' act=3690520740 pref=403577241.6
Sep 28 21:31:03 dom0 qmemman.systemstate[18757]: stat: xenfree=68044308 memset_reqs=[('0', 12544332298), ('1', 3785081161)]
Sep 28 21:31:03 dom0 qmemman.systemstate[18757]: mem-set domain 0 to 12544332298
Sep 28 21:31:03 dom0 qmemman.systemstate[18757]: mem-set domain 1 to 3785081161
Sep 28 21:33:20 dom0 qubesd[18636]: Starting personal
Sep 28 21:33:20 dom0 qubesd[18636]: Starting sys-firewall
Sep 28 21:33:20 dom0 qmemman.systemstate[18757]: do_balloon(memsize=682622976)
Sep 28 21:33:20 dom0 qmemman.systemstate[18757]: xenfree=69507353
Sep 28 21:33:20 dom0 qmemman.daemon.algo[18757]: balloon: dom 0 has actual memory 12544332298
Sep 28 21:33:20 dom0 qmemman.daemon.algo[18757]: balloon: dom 1 has actual memory 3785081161
Sep 28 21:33:20 dom0 qmemman.daemon.algo[18757]: req=665544423 avail=14588320108.6 donors=[('0', 11206816189.2), ('1', 3381503919.4)]
Sep 28 21:33:20 dom0 qmemman.daemon.algo[18757]: borrow 536838077.0865299 from 0
Sep 28 21:33:20 dom0 qmemman.daemon.algo[18757]: borrow 161983567.06347007 from 1
Sep 28 21:33:20 dom0 qmemman.systemstate[18757]: memset_reqs=[('0', 12007494220), ('1', 3623097593)]
Sep 28 21:33:20 dom0 qmemman.systemstate[18757]: mem-set domain 0 to 12007494220
Sep 28 21:33:20 dom0 qmemman.systemstate[18757]: mem-set domain 1 to 3623097593
Sep 28 21:33:20 dom0 qmemman.systemstate[18757]: xenfree=767073703
Sep 28 21:33:20 dom0 dmeventd[797]: No longer monitoring thin pool qubes_dom0-pool00-tpool.
Sep 28 21:33:20 dom0 lvm[797]: Monitoring thin pool qubes_dom0-pool00-tpool.
Sep 28 21:33:20 dom0 qubesd[18636]: b'  WARNING: Sum of all thin volume sizes (2.05 TiB) exceeds the size of thin pool qubes_dom0/pool00 and the size of whole volume group (1.82 TiB)!\n'
Sep 28 21:33:21 dom0 qubesd[18636]: b'  WARNING: Sum of all thin volume sizes (2.05 TiB) exceeds the size of thin pool qubes_dom0/pool00 and the size of whole volume group (1.82 TiB)!\n'
Sep 28 21:33:21 dom0 libvirtd[18670]: 2017-09-28 19:33:21.484+0000: 18702: error : libxlDomainStart:1320 : internal error: libxenlight failed to create new domain 'sys-firewall'
Sep 28 21:33:21 dom0 qmemman.daemon.reqhandler[18757]: EOF
Sep 28 21:33:21 dom0 qubesd[18636]: QubesException("Start failed: internal error: libxenlight failed to create new domain 'sys-firewall'",) while calling src=b'dom0' meth=b'admin.vm.Start' dest=b'personal' arg=b'' len(untrusted_payload)=0
Sep 28 21:33:21 dom0 qmemman.daemon.algo[18757]: balance_when_enough_memory(xen_free_memory=714644903, total_mem_pref=1785097497.6, total_available_memory=14560139218.4)
Sep 28 21:33:21 dom0 qmemman.systemstate[18757]: stat: dom '0' act=12007494220 pref=1381520256.0
Sep 28 21:33:21 dom0 qmemman.systemstate[18757]: stat: dom '1' act=3623097593 pref=403577241.6
Sep 28 21:33:21 dom0 qmemman.systemstate[18757]: stat: xenfree=767073703 memset_reqs=[('0', 12637233746), ('1', 3691657733)]
Sep 28 21:33:21 dom0 qmemman.systemstate[18757]: mem-set domain 0 to 12637233746
Sep 28 21:33:21 dom0 qmemman.systemstate[18757]: mem-set domain 1 to 3691657733
Sep 28 21:33:23 dom0 qmemman.daemon.algo[18757]: balance_when_enough_memory(xen_free_memory=16876533, total_mem_pref=1744634342.4, total_available_memory=14601133669.6)
Sep 28 21:33:23 dom0 qmemman.systemstate[18757]: stat: dom '0' act=12637233746 pref=1341057100.8000002
Sep 28 21:33:23 dom0 qmemman.systemstate[18757]: stat: dom '1' act=3691657733 pref=403577241.6
Sep 28 21:33:23 dom0 qmemman.systemstate[18757]: stat: xenfree=69305333 memset_reqs=[('0', 12552021429), ('1', 3777400814)]
Sep 28 21:33:23 dom0 qmemman.systemstate[18757]: mem-set domain 0 to 12552021429
Sep 28 21:33:23 dom0 qmemman.systemstate[18757]: mem-set domain 1 to 3777400814

errors from libxldriver

2017-09-28 19:26:05.234+0000: libxl: libxl_create.c:1265:domcreate_rebuild_done: cannot (re-)build domain: -3
2017-09-28 19:26:19.523+0000: libxl: libxl_dom.c:892:libxl__load_hvm_firmware_module: failed to read BIOS file: No such file or directory
2017-09-28 19:26:19.523+0000: libxl: libxl_dom.c:1079:libxl__build_hvm: initializing domain firmware failed
2017-09-28 19:26:19.523+0000: libxl: libxl_create.c:1265:domcreate_rebuild_done: cannot (re-)build domain: -3
2017-09-28 19:33:21.462+0000: libxl: libxl_dom.c:892:libxl__load_hvm_firmware_module: failed to read BIOS file: No such file or directory
2017-09-28 19:33:21.462+0000: libxl: libxl_dom.c:1079:libxl__build_hvm: initializing domain firmware failed
2017-09-28 19:33:21.462+0000: libxl: libxl_create.c:1265:domcreate_rebuild_done: cannot (re-)build domain: -3

systemctl


[root@dom0 martin]# systemctl --state failed
  UNIT                          LOAD   ACTIVE SUB    DESCRIPTION
● qubes-vm@sys-firewall.service loaded failed failed Start Qubes VM sys-firewall
● systemd-modules-load.service  loaded failed failed Load Kernel Modules


[root@dom0 martin]# systemctl status systemd-modules-load
● systemd-modules-load.service - Load Kernel Modules
   Loaded: loaded (/usr/lib/systemd/system/systemd-modules-load.service; static; vendor preset: disabled)
   Active: failed (Result: exit-code) since Thu 2017-09-28 21:47:40 CEST; 11min ago
     Docs: man:systemd-modules-load.service(8)
           man:modules-load.d(5)
  Process: 1221 ExecStart=/usr/lib/systemd/systemd-modules-load (code=exited, status=1/FAILURE)
 Main PID: 1221 (code=exited, status=1/FAILURE)

Sep 28 21:47:40 dom0 systemd-modules-load[1221]: Module 'xen_evtchn' is builtin
Sep 28 21:47:40 dom0 systemd-modules-load[1221]: Module 'xen_gntdev' is builtin
Sep 28 21:47:40 dom0 systemd-modules-load[1221]: Module 'xen_gntalloc' is builtin
Sep 28 21:47:40 dom0 systemd-modules-load[1221]: Inserted module 'xen_privcmd'
Sep 28 21:47:40 dom0 systemd-modules-load[1221]: Failed to insert 'xen_acpi_processor': No such device
Sep 28 21:47:40 dom0 systemd[1]: systemd-modules-load.service: Main process exited, code=exited, status=1/FAILURE
Sep 28 21:47:40 dom0 systemd[1]: Failed to start Load Kernel Modules.
Sep 28 21:47:40 dom0 systemd[1]: systemd-modules-load.service: Unit entered failed state.
Sep 28 21:47:40 dom0 systemd[1]: systemd-modules-load.service: Failed with result 'exit-code'.


[root@dom0 martin]# systemctl status qubes-vm@sys-firewall
● qubes-vm@sys-firewall.service - Start Qubes VM sys-firewall
   Loaded: loaded (/usr/lib/systemd/system/qubes-vm@.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Thu 2017-09-28 21:48:22 CEST; 11min ago
  Process: 19913 ExecStart=/usr/bin/qvm-start --skip-if-running %i (code=exited, status=1/FAILURE)
 Main PID: 19913 (code=exited, status=1/FAILURE)

Sep 28 21:47:56 dom0 systemd[1]: Starting Start Qubes VM sys-firewall...
Sep 28 21:48:22 dom0 qvm-start[19913]: Start failed: internal error: libxenlight failed to create new domain 'sys-firewall'
Sep 28 21:48:22 dom0 systemd[1]: qubes-vm@sys-firewall.service: Main process exited, code=exited, status=1/FAILURE
Sep 28 21:48:22 dom0 systemd[1]: Failed to start Start Qubes VM sys-firewall.
Sep 28 21:48:22 dom0 systemd[1]: qubes-vm@sys-firewall.service: Unit entered failed state.
Sep 28 21:48:22 dom0 systemd[1]: qubes-vm@sys-firewall.service: Failed with result 'exit-code'.
@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Sep 28, 2017

Member

Duplicate of #3125
In short: install updates, specifically xen-hvm-stubdom-linux. Since sys-firewall do not start, you may need to temporarily switch updatevm to sys-net (see qubes-prefs tool).
Member

marmarek commented Sep 28, 2017

Duplicate of #3125
In short: install updates, specifically xen-hvm-stubdom-linux. Since sys-firewall do not start, you may need to temporarily switch updatevm to sys-net (see qubes-prefs tool).

@marmarek marmarek closed this Sep 28, 2017

@marmarek marmarek added the duplicate label Sep 28, 2017

@holiman

This comment has been minimized.

Show comment
Hide comment
@holiman

holiman Sep 29, 2017

👍 Thanks, can confirm that worked

holiman commented Sep 29, 2017

👍 Thanks, can confirm that worked
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment