Skip to content
This repository has been archived by the owner on Aug 29, 2023. It is now read-only.

VM running on QEMU crashes #34

Closed
anudeep404 opened this issue May 26, 2018 · 6 comments
Closed

VM running on QEMU crashes #34

anudeep404 opened this issue May 26, 2018 · 6 comments

Comments

@anudeep404
Copy link

anudeep404 commented May 26, 2018 •
edited

Was previously running use ubuntu kernel 4.9.40 and we observe a lot of crash on big Qemu VM.
I'm trying to spin up more than 10 x (2vCPU, 8Gig) VMs, Cisco Nexus 9Ks.
#25
Saw above post and thought it is a bug on 4.9.40 to fix, I moved to :

root@eve-ng:~# uname -a
Linux eve-ng 4.14.44 #1 SMP Sat May 26 16:36:14 EEST 2018 x86_64 x86_64 x86_64 GNU/Linux

Below is the log from Qemu VM, which always goes into below boot loop.

loader > ����������������������������������������������������������������������boot nxos.7.0.3.I5.2.bin
Booting nxos.7.0.3.I5.2.bin
Trying diskboot
dev_str: bootflash: partition_str: 3 filename_str /nxos.7.0.3.I5.2.bin
command = root (hd0,3)
Filesystem type is ext2fs, partition type 0x83
Formed cmdline console=ttyS0,115200n8nn loader_ver="5.00.9" quiet debug
Formed cmdline console=ttyS0,115200n8nn loader_ver="5.00.9" quiet debug ksimg=nxos.7.0.3.I5.2.bin card_index=21099 dummy_sprom
Booting kickstart image: bootflash::3:/nxos.7.0.3.I5.2.bin....
Number of devices detected by BIOS is 1
NBI header
magic: 1b031336, len: 54, location: 94400000 (bx=0, ds=9440), exec addr: 92800000 16 16

segment header
length: 4, vendor: 11 flags: 0, loadaddr: 94000, image len: 200, memory length: 400
Reading data for kernel param. Len 512
Cmd line: console=ttyS0,115200n8nn loader_ver="5.00.9" quiet debug ksimg=nxos.7.0.3.I5.2.bin card_index=21099 dummy_sprom rw root=/dev/ram0 rdbase=0x8000000 ip=off ramdisk_size=131072 panic=5 quiet coredump_filter=0xb mtdparts=physmap-flash.0:256k(RR_LOG),512k(mtdoops),16M(plog),16M(trace) intel_idle.max_cstate=2 pcie_ports=native nopat slub_debug=- cpuidle.off=1

segment header
length: 4, vendor: 14 flags: 0, loadaddr: 100000, image len: 520e00, memory length: 800000
Loading kernel length 5377536
file_size 5377536 real_size 4456 prot_kernel_size 5361664 setupsects 30
x86_64/loader/linux.c:102: real_size = 2000, prot_size = 51d000, mmap_size = 6f0
x86_64/loader/linux.c:133: physical_start = 1000, physical_end = 9f000
x86_64/loader/linux.c:146: trying to allocate 2 pages at 9d000
Trying to allocate 1309 pages for VMLINUZ
Allocated prot_mode_mem 0xbd2d3000 real_mode_mem 0x0009d000
[Linux-EFI, setup=0x1168, size=0x51d000]

segment header
length: 14, vendor: 15 flags: 0, loadaddr: a21000, image len: 2cd18000, memory length: 10000000
Loading intird 751927296
x86_64/loader/linux.c:573: initrd_pages: 183576
x86_64/loader/linux.c:584: addr_min: 0x0 addr_max: 0x7ffff000 mmap_size: 1824
x86_64/loader/linux.c:603: desc = {type=7,ps=0x1000,vs=0x0,sz=156,attr=15}
x86_64/loader/linux.c:603: desc = {type=7,ps=0x100000,vs=0x0,sz=1792,attr=15}
x86_64/loader/linux.c:603: desc = {type=7,ps=0x808000,vs=0x0,sz=8,attr=15}
x86_64/loader/linux.c:603: desc = {type=7,ps=0x818000,vs=0x0,sz=8,attr=15}
x86_64/loader/linux.c:603: desc = {type=7,ps=0x1300000,vs=0x0,sz=518141,attr=15}
x86_64/loader/linux.c:603: desc = {type=7,ps=0x80000000,vs=0x0,sz=245054,attr=15}
x86_64/loader/linux.c:603: desc = {type=7,ps=0xbbd5e000,vs=0x0,sz=5493,attr=15}
x86_64/loader/linux.c:603: desc = {type=7,ps=0xbd86c000,vs=0x0,sz=121,attr=15}
x86_64/loader/linux.c:603: desc = {type=7,ps=0xbf8d5000,vs=0x0,sz=2,attr=15}
x86_64/loader/linux.c:603: desc = {type=7,ps=0xbfd70000,vs=0x0,sz=16,attr=15}
x86_64/loader/linux.c:603: desc = {type=7,ps=0x100000000,vs=0x0,sz=1310720,attr=15}

Loading [0M/717M]
Loading [8M/717M]
Loading [16M/717M]
Loading [24M/717M]�[2J�[01;01H�[=3h�[2J�[01;01H�[2J�[01;01H�[=3h�[2J�[01;01H�[2J�[01;01H�[=3h�[2J�[01;01H�[2J�[01;01H�[=3h�[2J�[01;01H�[2J�[01;01H�[=3h�[2J�[01;01H�[0m�[35m�[40m�[2J�[01;01H�[=3h�[2J�[01;01H�[0m�[37m�[40m�[2J�[01;01H�[=3h�[2J�[01;01H/ACPI(a0341d0,0)/PCI(0,5)/SATA(0,ffff,0)/EndEntire
/ACPI(a0341d0,0)/PCI(0,5)/SATA(0,ffff,0)/HD(1,800,8000,c48fa7a2-22f0-4149-d4bc-1993e4cf76c1)/EndEntire
/ACPI(a0341d0,0)/PCI(0,5)/SATA(0,ffff,0)/HD(2,8800,a3332,e717d076-a237-41f2-8081-f2c4fe637289)/EndEntire
/ACPI(a0341d0,0)/PCI(0,5)/SATA(0,ffff,0)/HD(3,ac000,146666,79914cb2-73d2-4fb9-1999-b15f771375a6)/EndEntire
/ACPI(a0341d0,0)/PCI(0,5)/SATA(0,ffff,0)/HD(4,1f2800,6b199c,5330e85f-6426-43dd-e986-056152ac5679)/EndEntire
/ACPI(a0341d0,0)/PCI(0,5)/SATA(0,ffff,0)/HD(5,8a4800,146666,aa000172-ffda-4794-9d8e-444ba967be56)/EndEntire
/ACPI(a0341d0,0)/PCI(0,5)/SATA(0,ffff,0)/HD(6,9eb000,146666,6fb3e8b7-501e-4b25-f791-787af6874f46)/EndEntire
/ACPI(a0341d0,0)/PCI(0,5)/SATA(0,ffff,0)/HD(7,b31800,4c8000,a9c6e5c0-20cd-4bb0-5895-154fa6daee2d)/EndEntire
/ACPI(a0341d0,0)/PCI(0,1)/ACPI(60441d0,0)/EndEntire
/ACPI(a0341d0,0)/PCI(0,1)/ACPI(60441d0,1)/EndEntire
/ACPI(a0341d0,0)/PCI(0,5)/SATA(0,ffff,0)/EndEntire
/ACPI(a0341d0,0)/PCI(0,5)/SATA(0,ffff,0)/HD(1,800,8000,c48fa7a2-22f0-4149-d4bc-1993e4cf76c1)/EndEntire
/ACPI(a0341d0,0)/PCI(0,5)/SATA(0,ffff,0)/HD(2,8800,a3332,e717d076-a237-41f2-8081-f2c4fe637289)/EndEntire
/ACPI(a0341d0,0)/PCI(0,5)/SATA(0,ffff,0)/HD(3,ac000,146666,79914cb2-73d2-4fb9-1999-b15f771375a6)/EndEntire
/ACPI(a0341d0,0)/PCI(0,5)/SATA(0,ffff,0)/HD(4,1f2800,6b199c,5330e85f-6426-43dd-e986-056152ac5679)/EndEntire
/ACPI(a0341d0,0)/PCI(0,5)/SATA(0,ffff,0)/HD(5,8a4800,146666,aa000172-ffda-4794-9d8e-444ba967be56)/EndEntire
/ACPI(a0341d0,0)/PCI(0,5)/SATA(0,ffff,0)/HD(6,9eb000,146666,6fb3e8b7-501e-4b25-f791-787af6874f46)/EndEntire
/ACPI(a0341d0,0)/PCI(0,5)/SATA(0,ffff,0)/HD(7,b31800,4c8000,a9c6e5c0-20cd-4bb0-5895-154fa6daee2d)/EndEntire
/ACPI(a0341d0,0)/PCI(0,1)/ACPI(60441d0,0)/EndEntire
/ACPI(a0341d0,0)/PCI(0,1)/ACPI(60441d0,1)/EndEntire
Total memory is 0x9f000
Total memory is 0x800000
Sysconf checksum failed. Using default values
WARNING: No BIOS Info found
Sysconf checksum failed. Using default values
serial --speed=115200 --word=8 --stop=1 --parity=no

success
Sysconf checksum failed. Using default values
Strlen bootflash: is 10
Device bootflash: is Drive 0x80 Partition 3
ATE0Q1&D2&C1S0=1

Trying to read config file /boot/grub/menu.lst.local from (hd0,4)
Filesystem type is ext2fs, partition type 0x83
Number of devices detected by BIOS is 1
Failed. Trying to read config file from (hd0,5)
Filesystem type is ext2fs, partition type 0x83
Number of devices detected by BIOS is 1
Failed.
Loader Version 5.00.9
Entering interactive mode

loader >

I understand that you need information so could you please give a set of required information needed further analysis?
@dolohow
Copy link
Owner

dolohow commented May 27, 2018 •
edited

Does it also happen without UKSM?

Is there anything in dmesg?

@anudeep404
Copy link
Author

I just tested, disabling UKSM as well, yes it fails.
In a nutshell, 10 instances of QEMU runs well, but 11th and so on are stuck in boot loop.
Looks like dmesg doesn't show anything about it.

Every 2.0s: dmesg | tail -30 Sun May 27 06:10:52 2018

[42123.099942] device vunl0_9_2 left promiscuous mode
[42123.099945] vnet0_17: port 2(vunl0_9_2) entered disabled state
[42123.099942] device vunl0_9_2 left promiscuous mode
[42123.099945] vnet0_17: port 2(vunl0_9_2) entered disabled state
[42123.147453] vnet0_17: port 2(vunl0_9_2) entered blocking state
[42123.147455] vnet0_17: port 2(vunl0_9_2) entered disabled state
[42123.147525] device vunl0_9_2 entered promiscuous mode
[42123.147541] vnet0_17: port 2(vunl0_9_2) entered blocking state
[42123.147542] vnet0_17: port 2(vunl0_9_2) entered forwarding state
[42123.157039] vnet0_9: port 2(vunl0_9_3) entered disabled state
[42123.157533] device vunl0_9_3 left promiscuous mode
[42123.157535] vnet0_9: port 2(vunl0_9_3) entered disabled state
[42123.205233] vnet0_9: port 2(vunl0_9_3) entered blocking state
[42123.205236] vnet0_9: port 2(vunl0_9_3) entered disabled state
[42123.205327] device vunl0_9_3 entered promiscuous mode
[42123.205363] vnet0_9: port 2(vunl0_9_3) entered blocking state
[42123.205365] vnet0_9: port 2(vunl0_9_3) entered forwarding state
[42123.985009] vnet0_9: port 2(vunl0_9_3) entered disabled state
[42123.985259] vnet0_17: port 2(vunl0_9_2) entered disabled state
[42123.985483] vnet0_15: port 2(vunl0_9_1) entered disabled state
[42140.237119] vnet0_18: port 1(vunl0_10_1) entered blocking state
[42140.237124] vnet0_18: port 1(vunl0_10_1) entered forwarding state
[42140.237713] vnet0_18: port 1(vunl0_10_1) entered disabled state
[42140.238129] device vunl0_10_1 left promiscuous mode
[42140.238132] vnet0_18: port 1(vunl0_10_1) entered disabled state
[42140.291915] vnet0_18: port 1(vunl0_10_1) entered blocking state
[42140.291918] vnet0_18: port 1(vunl0_10_1) entered disabled state
[42140.291989] device vunl0_10_1 entered promiscuous mode
[42140.292005] vnet0_18: port 1(vunl0_10_1) entered blocking state
[42140.292006] vnet0_18: port 1(vunl0_10_1) entered forwarding state
[42141.157352] vnet0_18: port 1(vunl0_10_1) entered disabled state
[42192.511789] kvm [5865]: vcpu0, guest rIP: 0xffffffff8102fcf8 disabled perfctr wrmsr: 0xc1 data 0xabcd

Here is the syslog:

May 27 06:15:59 eve-ng ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl del-port vunl0_16_0
May 27 06:15:59 eve-ng ovs-vsctl: ovs|00002|db_ctl_base|ERR|no port named vunl0_16_0
May 27 06:15:59 eve-ng systemd-udevd[10276]: Could not generate persistent MAC address for vunl0_16_0: No such file or directory
May 27 06:15:59 eve-ng ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl del-port vunl0_16_1
May 27 06:15:59 eve-ng ovs-vsctl: ovs|00002|db_ctl_base|ERR|no port named vunl0_16_1
May 27 06:15:59 eve-ng kernel: [42622.151360] vnet0_25: port 1(vunl0_16_1) entered blocking state
May 27 06:15:59 eve-ng kernel: [42622.151364] vnet0_25: port 1(vunl0_16_1) entered forwarding state
May 27 06:15:59 eve-ng kernel: [42622.152014] vnet0_25: port 1(vunl0_16_1) entered disabled state
May 27 06:15:59 eve-ng kernel: [42622.152451] device vunl0_16_1 left promiscuous mode
May 27 06:15:59 eve-ng kernel: [42622.152455] vnet0_25: port 1(vunl0_16_1) entered disabled state
May 27 06:15:59 eve-ng systemd-udevd[10317]: Could not generate persistent MAC address for vunl0_16_1: No such file or directory
May 27 06:15:59 eve-ng kernel: [42622.209495] vnet0_25: port 1(vunl0_16_1) entered blocking state
May 27 06:15:59 eve-ng kernel: [42622.209499] vnet0_25: port 1(vunl0_16_1) entered disabled state
May 27 06:15:59 eve-ng kernel: [42622.209630] device vunl0_16_1 entered promiscuous mode
May 27 06:15:59 eve-ng kernel: [42622.209658] vnet0_25: port 1(vunl0_16_1) entered blocking state
May 27 06:15:59 eve-ng kernel: [42622.209660] vnet0_25: port 1(vunl0_16_1) entered forwarding state
May 27 06:15:59 eve-ng ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl del-port vunl0_16_2
May 27 06:15:59 eve-ng ovs-vsctl: ovs|00002|db_ctl_base|ERR|no port named vunl0_16_2
May 27 06:15:59 eve-ng systemd-udevd[10362]: Could not generate persistent MAC address for vunl0_16_2: No such file or directory
May 27 06:15:59 eve-ng ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl del-port vunl0_16_3
May 27 06:15:59 eve-ng ovs-vsctl: ovs|00002|db_ctl_base|ERR|no port named vunl0_16_3
May 27 06:15:59 eve-ng systemd-udevd[10402]: Could not generate persistent MAC address for vunl0_16_3: No such file or directory
May 27 06:15:59 eve-ng ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl del-port vunl0_16_4
May 27 06:15:59 eve-ng ovs-vsctl: ovs|00002|db_ctl_base|ERR|no port named vunl0_16_4
May 27 06:15:59 eve-ng systemd-udevd[10455]: Could not generate persistent MAC address for vunl0_16_4: No such file or directory
May 27 06:15:59 eve-ng ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl del-port vunl0_16_5
May 27 06:15:59 eve-ng ovs-vsctl: ovs|00002|db_ctl_base|ERR|no port named vunl0_16_5
May 27 06:15:59 eve-ng systemd-udevd[10496]: Could not generate persistent MAC address for vunl0_16_5: No such file or directory
May 27 06:15:59 eve-ng ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl del-port vunl0_16_6
May 27 06:15:59 eve-ng ovs-vsctl: ovs|00002|db_ctl_base|ERR|no port named vunl0_16_6
May 27 06:15:59 eve-ng systemd-udevd[10537]: Could not generate persistent MAC address for vunl0_16_6: No such file or directory
May 27 06:15:59 eve-ng ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl del-port vunl0_16_7
May 27 06:15:59 eve-ng ovs-vsctl: ovs|00002|db_ctl_base|ERR|no port named vunl0_16_7
May 27 06:15:59 eve-ng systemd-udevd[10578]: Could not generate persistent MAC address for vunl0_16_7: No such file or directory
May 27 06:16:00 eve-ng kernel: [42623.090686] vnet0_25: port 1(vunl0_16_1) entered disabled state

@dolohow
Copy link
Owner

dolohow commented May 27, 2018

Looks like not our bug. Closing

@dolohow dolohow closed this as completed May 27, 2018
@tonydm
Copy link

tonydm commented Jan 8, 2021

Hello,

New to EVE-NG... I too am having a problem with my router node not starting and seeing this message in the syslog.
Jan 8 09:26:02 eve-ng ovs-vsctl: ovs|00002|db_ctl_base|ERR|no port named vunl0_1_6

You closed this issue stating

Looks like not our bug

The log entry shows it's generated by EVE-NG. Perhaps it's not an EVE-NG bug, but can you help your users better understand "why" it's not a bug in EVE-NG and perhaps where else to look?

Thank you very much

@gyang810
Copy link

did you guys get any answer for it? I am still struggling with this issue.

@julliopereira
Copy link

I have the same issue, when I start qemu equipments I have that lines in /var/log/syslog file of eve-ng machine:

Jan 4 00:40:32 eveng-Srv ovs-vsctl: ovs|00002|db_ctl_base|ERR|no port named vunl0_6_0
Jan 4 00:40:32 eveng-Srv kernel: [ 4300.807814] vnet0_2: port 1(vunl0_6_0) entered blocking state
Jan 4 00:40:32 eveng-Srv kernel: [ 4300.807815] vnet0_2: port 1(vunl0_6_0) entered forwarding state
Jan 4 00:40:32 eveng-Srv kernel: [ 4300.807942] vnet0_2: port 1(vunl0_6_0) entered disabled state
Jan 4 00:40:32 eveng-Srv kernel: [ 4300.808028] device vunl0_6_0 left promiscuous mode
Jan 4 00:40:32 eveng-Srv kernel: [ 4300.808031] vnet0_2: port 1(vunl0_6_0) entered disabled state
Jan 4 00:40:32 eveng-Srv systemd-udevd[32514]: Could not generate persistent MAC address for vunl0_6_0: No such file or directory
Jan 4 00:40:32 eveng-Srv ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl del-port vunl0_6_1
Jan 4 00:40:32 eveng-Srv ovs-vsctl: ovs|00002|db_ctl_base|ERR|no port named vunl0_6_1
Jan 4 00:40:32 eveng-Srv systemd-udevd[32603]: Could not generate persistent MAC address for vunl0_6_1: No such file or directory

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@tonydm @dolohow @anudeep404 @gyang810 @julliopereira