VMs not starting - libxenlight failed to create new-domain in 4.0rc-1

[platschi]

Qubes OS version (e.g., R3.2):

R4.0-rc1

Affected TemplateVMs (e.g., fedora-23, if applicable):

all, except sys-net

---

Steps to reproduce the behavior:

Installed fresh R4.0-rc1 iso. Qubes / all VMs worked as expected. Then updated to current-testing with sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Expected behavior:

Domains / ServiceVMs and Templates start normally.

Actual behavior:

When using the GUI app menu to start any VM, nothing happens. The Qubes tray icon only shows that sys-net is running.

In Terminal, receive this error (with any VM, in example trying to start sys-firewall)

[platschi@dom0 qubes]$ qvm-start sys-firewall`
Start failed: internal error: libxenlight failed to create new domain 'sys-firewall'

General notes:

In /var/log/qubes.log:

Starting sys-firewall
QubesException("Start failed: internal error: libxenlight failed to create new domain 'sys-firewall'",) while calling src=b'dom0' meth=b'admin.vm.Start' dest=b'sys-firewall' arg=b'' len(untrusted_payload)=0

---

Related issues:

[marmarek]

See /var/log/libvirt/libxl/libxl-driver.log, but I guess you don't have VT-x enabled in BIOS.

[marmarek]

See /var/log/libvirt/libxl/libxl-driver.log, but I guess you don't have VT-x enabled in BIOS.

[0spinboson]

I'm also getting this since installing the latest set of (libvirt) updates, without rebooting. Downgrading "solved" the issue.

[0spinboson]

I'm also getting this since installing the latest set of (libvirt) updates, without rebooting. Downgrading "solved" the issue.

[marmarek]

Ok, I think I've found what is the problem: updated xen package, but not xen-hvm-stubdom-linux with required changes. The later was just uploaded to testing repository, try now.

[marmarek]

Ok, I think I've found what is the problem: updated xen package, but not xen-hvm-stubdom-linux with required changes. The later was just uploaded to testing repository, try now.

[0spinboson]

yes, it appears to be behaving properly again now.

[0spinboson]

yes, it appears to be behaving properly again now.

[platschi]

Exactly, the updated xen-hvm-stubdom-linux package solved it. Thanks a lot!

[platschi]

Exactly, the updated xen-hvm-stubdom-linux package solved it. Thanks a lot!

[P4z]

I have just experienced this problem with R4.0 rc2 on Dell XPS L702X.

Worth to note:

• the installer complained about issues with IO MMU (i7-2630QM)
• the installer complained about missing root.img for vm-templates/fedora-25
• the dom0 could not startup sys-net complaining about libxenlight

I'm back on R3.2 with no such issues. To me this is critical issue because I cannot even install R4

[P4z]

I have just experienced this problem with R4.0 rc2 on Dell XPS L702X.

Worth to note:

• the installer complained about issues with IO MMU (i7-2630QM)
• the installer complained about missing root.img for vm-templates/fedora-25
• the dom0 could not startup sys-net complaining about libxenlight

I'm back on R3.2 with no such issues. To me this is critical issue because I cannot even install R4

[reconmaster]

Reproduced this on x230 & t530 (i5s) with the sys-net qube using 4.0 rc3. I'll be following up with the newest rcs to verify.

[reconmaster]

Reproduced this on x230 & t530 (i5s) with the sys-net qube using 4.0 rc3. I'll be following up with the newest rcs to verify.

[hast0011]

Can be seen also on Dell Precision T5500 using 4.0 rc3 on sys-firewall

[hast0011]

Can be seen also on Dell Precision T5500 using 4.0 rc3 on sys-firewall

[marmarek]

Note that default setup of Qubes 4.0 require IOMMU (aka VT-d). If installer says you don't have it, it will be a problem. If your hardware should support IOMMU/VT-d in theory, check if you have it enabled in BIOS.

[marmarek]

Note that default setup of Qubes 4.0 require IOMMU (aka VT-d). If installer says you don't have it, it will be a problem. If your hardware should support IOMMU/VT-d in theory, check if you have it enabled in BIOS.

[hast0011]

Back after reboot:-) It was enabled. The Installer didn't show any message about it - at least I didn't see any.

[hast0011]

Back after reboot:-) It was enabled. The Installer didn't show any message about it - at least I didn't see any.

[marmarek]

Check /var/log/libvirt/libxl/libxl-driver.log in dom0, you should have some more detailed error message there.

[marmarek]

Check /var/log/libvirt/libxl/libxl-driver.log in dom0, you should have some more detailed error message there.

[reconmaster]

I'll be giving both the t530 and x230 shots with coreboot+seabios somewhere over the holiday. Other user advice on these models have mentioned firmware options not available in default lenovo firmware (17/09/27th i believe). I'll report back as soon as I get an outcome for both variables.

[reconmaster]

I'll be giving both the t530 and x230 shots with coreboot+seabios somewhere over the holiday. Other user advice on these models have mentioned firmware options not available in default lenovo firmware (17/09/27th i believe). I'll report back as soon as I get an outcome for both variables.

[na--]

When I run qvm-start -v test, the following error appears:

Start failed: internal error: libxenlight failed to create new domain 'test', see /var/log/libvirt/libxl/libxl-driver.log for details

This is the content of /var/log/libvirt/libxl/libxl-driver.log:

2018-01-03 09:07:59.561+0000: libxl: libxl.c:422:libxl__domain_rename: domain with name "test-dm" already exists.
2018-01-03 09:07:59.562+0000: libxl: libxl_dm.c:2076:stubdom_pvqemu_cb: error connecting nics devices: Function not implemented
2018-01-03 09:07:59.562+0000: libxl: libxl_create.c:1542:domcreate_devmodel_started: device model did not start: -6

I'm using R4 and I updated everything with the latest versions yesterday (even enabled the qubes-dom0-current-testing repo in dom0)

[na--]

When I run qvm-start -v test, the following error appears:

Start failed: internal error: libxenlight failed to create new domain 'test', see /var/log/libvirt/libxl/libxl-driver.log for details

This is the content of /var/log/libvirt/libxl/libxl-driver.log:

2018-01-03 09:07:59.561+0000: libxl: libxl.c:422:libxl__domain_rename: domain with name "test-dm" already exists.
2018-01-03 09:07:59.562+0000: libxl: libxl_dm.c:2076:stubdom_pvqemu_cb: error connecting nics devices: Function not implemented
2018-01-03 09:07:59.562+0000: libxl: libxl_create.c:1542:domcreate_devmodel_started: device model did not start: -6

I'm using R4 and I updated everything with the latest versions yesterday (even enabled the qubes-dom0-current-testing repo in dom0)

[marmarek]

domain with name "test-dm" already exists - some leftover from previous failed startup? Try remove it with sudo xl destroy test-dm

[marmarek]

domain with name "test-dm" already exists - some leftover from previous failed startup? Try remove it with sudo xl destroy test-dm

[na--]

I tried that, it didn't help. Trying to start the VM after destroying the test-dm failed with the same error as before. Also, I'm not sure that the test-dm was correctly destroyed, xl list showed some null vms afterwards.
Edit: I'm unable to reproduce the issue since I restarted recently, but I'll try to be more thorough in diagnosing it when it happens next.

[na--]

I tried that, it didn't help. Trying to start the VM after destroying the test-dm failed with the same error as before. Also, I'm not sure that the test-dm was correctly destroyed, xl list showed some null vms afterwards.
Edit: I'm unable to reproduce the issue since I restarted recently, but I'll try to be more thorough in diagnosing it when it happens next.

[marmarek]

@na-- did it happened again? If not, I'd assume that one of updates have fixed it (directly or indirectly).

[marmarek]

@na-- did it happened again? If not, I'd assume that one of updates have fixed it (directly or indirectly).

[na--]

It hasn't happened again, so it may be fixed. I'll write here it it occurs again.

[na--]

It hasn't happened again, so it may be fixed. I'll write here it it occurs again.

[andrewdavidwong]

@platschi, @0spinboson, @P4z, @reconmaster, @hast0011:
Please let us know whether you're still affected by this issue.

[andrewdavidwong]

@platschi, @0spinboson, @P4z, @reconmaster, @hast0011:
Please let us know whether you're still affected by this issue.

[hast0011]

I have still troubles getting Internet working on this machine, see other issue #3349. Therefore I can't install any update to see if this issue disappears. I decided to wait for rc4-iso then test again.

[hast0011]

I have still troubles getting Internet working on this machine, see other issue #3349. Therefore I can't install any update to see if this issue disappears. I decided to wait for rc4-iso then test again.

[hast0011]

Installing Q4 rc4 solved the problem for me.

[hast0011]

Installing Q4 rc4 solved the problem for me.

[P4z]

Sorry about late answer.

Unfortunately Q4 rc4 is no difference here 8-(
I am about to try final release now

[P4z]

Sorry about late answer.

Unfortunately Q4 rc4 is no difference here 8-(
I am about to try final release now