Dual-cpu computer crashes

[pascal-mignot-urca]

Qubes OS version:

R4.0 (from iso downloaded 4 april 2018)

Affected component(s):

• can't install QuBes OS
• can't boot QuBes OS

Steps to reproduce the behavior:

Try installing QuBes 4.0 on a Dell Precision T7610 workstation (2 x Xeon processors)

Expected behavior:

it should install and boot.

Actual behavior:

Xen boot is ok. It detects 8 vcpus.
vmlinuz boot crashes after the trace: "Installing Xen timer for CPU 4"

General notes:

It seems to be related to management of the vcpus on the second cpu.
Adding dom0_max_vcpus=4 as additional Xen boot option prevents the crashes.
OS can be then installed and launched if this options is set at boot time.
Unfortunately, it also uses half of the computer resources.

Related issues:

More problems after install (VMs failed to start) but can't relate them to dual cpu for now.

[marmarek]

Unfortunately, it also uses half of the computer resources.

This is fortunately not true. Dom0 is only small part of the system and actually it may be good idea to limit its resources.

vmlinuz boot crashes after the trace: "Installing Xen timer for CPU 4"

Can you provide more details about the crash?

[marmarek]

Unfortunately, it also uses half of the computer resources.

This is fortunately not true. Dom0 is only small part of the system and actually it may be good idea to limit its resources.

vmlinuz boot crashes after the trace: "Installing Xen timer for CPU 4"

Can you provide more details about the crash?

[pascal-mignot-urca]

Hi Marek,I can't see how I can provide more information, since the crash prevent me to access to the logs. If there is one, please enlight me. Limits of dom0: since this option is passed at the xen hypervisor boot, I understood it as a limitation of the hypervisor resources. But you surely know better than me.Thanks,PascalLe 4 avr. 2018 9:52 PM, Marek Marczykowski-Górecki <notifications@github.com> a écrit : Unfortunately, it also uses half of the computer resources. This is fortunately not true. Dom0 is only small part of the system and actually it may be good idea to limit its resources. vmlinuz boot crashes after the trace: "Installing Xen timer for CPU 4" Can you provide more details about the crash? —You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or mute the thread.

[pascal-mignot-urca]

Hi Marek,I can't see how I can provide more information, since the crash prevent me to access to the logs. If there is one, please enlight me. Limits of dom0: since this option is passed at the xen hypervisor boot, I understood it as a limitation of the hypervisor resources. But you surely know better than me.Thanks,PascalLe 4 avr. 2018 9:52 PM, Marek Marczykowski-Górecki <notifications@github.com> a écrit : Unfortunately, it also uses half of the computer resources. This is fortunately not true. Dom0 is only small part of the system and actually it may be good idea to limit its resources. vmlinuz boot crashes after the trace: "Installing Xen timer for CPU 4" Can you provide more details about the crash? —You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or mute the thread.

[marmarek]

Option dom0_max_vcpus=4 is only about dom0.

At this early stage, the crash message woudn't get into logs anyway. If you have serial console, you can try this (add console=com1 to Xen). If not, if you see it on screen, you can take a photo of it...

[marmarek]

Option dom0_max_vcpus=4 is only about dom0.

At this early stage, the crash message woudn't get into logs anyway. If you have serial console, you can try this (add console=com1 to Xen). If not, if you see it on screen, you can take a photo of it...

[philrdubois]

I just opened #3788 requesting to replace the kernel on the iso to 4.15; I also have a Dell Precision T (T3600) workstation (1 cpu, 8 cores) that has a similar issue and I found out last summer that it was a bug in the kernel, which has been repaired in 4.15.

Apologies if I am incorrect in believing these are related. I will be attempting to add dom0_max_vcpus=4 at boot in the morning to see if they are related.

[philrdubois]

I just opened #3788 requesting to replace the kernel on the iso to 4.15; I also have a Dell Precision T (T3600) workstation (1 cpu, 8 cores) that has a similar issue and I found out last summer that it was a bug in the kernel, which has been repaired in 4.15.

Apologies if I am incorrect in believing these are related. I will be attempting to add dom0_max_vcpus=4 at boot in the morning to see if they are related.

[pascal-mignot-urca]

Be sure to place this option just after the xen kernel: it's a xen boot option and not a linux one.

[pascal-mignot-urca]

Be sure to place this option just after the xen kernel: it's a xen boot option and not a linux one.

[pascal-mignot-urca]

Regarding the logs.
The xen kernel boot correctly signaling 8 vcpus.
Then, the linux kernel boots.

The crash occuring just after the following lines:
[...] CPU: Physical Processor ID: 0
[...] CPU: Processor Core ID: 0
[...] mce: CPU supports 2 MCE banks
[...] Last level iTLB entries: 4KB 512, 2MB 0, 4MB 0
[...] Last level dTLB entries: 4KB 512, 2MB 0, 4MB 0, 1GB 4
[...] Freeing SMP alternative memory: 32KB
[...] smpboot: Max logical package: 1
[...] VPMU disabled by hypervisor
[...] Performance events: IvyBridgeEvents, PMU not available due to virtualization, using software events only
[...] Hierarchical SRCU implementation
[...] smp: Bringing up secondary CPUs ...
[...] installing XEN timer for CPU 1
[...] NMI watchdog: disabled (cpu0): Hardware events not enabled
[...] NMI watchdog: shutting down hard lockup detector on all cpus
[...] installing XEN timer for CPU 2
[...] installing XEN timer for CPU 3
[...] installing XEN timer for CPU 4
crash here.

By the way, I don''t know if it is possible to create an iso with an additional writable partition to keep the early logs (/tmp, /var/log, ...) since lots of us are now using a usb stick to install OSes.

[pascal-mignot-urca]

Regarding the logs.
The xen kernel boot correctly signaling 8 vcpus.
Then, the linux kernel boots.

The crash occuring just after the following lines:
[...] CPU: Physical Processor ID: 0
[...] CPU: Processor Core ID: 0
[...] mce: CPU supports 2 MCE banks
[...] Last level iTLB entries: 4KB 512, 2MB 0, 4MB 0
[...] Last level dTLB entries: 4KB 512, 2MB 0, 4MB 0, 1GB 4
[...] Freeing SMP alternative memory: 32KB
[...] smpboot: Max logical package: 1
[...] VPMU disabled by hypervisor
[...] Performance events: IvyBridgeEvents, PMU not available due to virtualization, using software events only
[...] Hierarchical SRCU implementation
[...] smp: Bringing up secondary CPUs ...
[...] installing XEN timer for CPU 1
[...] NMI watchdog: disabled (cpu0): Hardware events not enabled
[...] NMI watchdog: shutting down hard lockup detector on all cpus
[...] installing XEN timer for CPU 2
[...] installing XEN timer for CPU 3
[...] installing XEN timer for CPU 4
crash here.

By the way, I don''t know if it is possible to create an iso with an additional writable partition to keep the early logs (/tmp, /var/log, ...) since lots of us are now using a usb stick to install OSes.

[philrdubois]

Strangely enough, I get different errors when booting from Legacy vs EFI. In EFI, I get errors similar to how you describe and cannot get to the linux kernel, however I cannot modify the boot commands (as is expected according to Xen documentation). In Legacy mode, I do not need to use dom0_max_vcpus=4 to get past Xen and into the kernel, where I get my kernel panic.

Due to this, our issues seem to be unrelated.

[philrdubois]

Strangely enough, I get different errors when booting from Legacy vs EFI. In EFI, I get errors similar to how you describe and cannot get to the linux kernel, however I cannot modify the boot commands (as is expected according to Xen documentation). In Legacy mode, I do not need to use dom0_max_vcpus=4 to get past Xen and into the kernel, where I get my kernel panic.

Due to this, our issues seem to be unrelated.