xl: correctly handle situation when cannot unbind PCI dev

[marmarek]

Reported by joanna on 12 Sep 2011 14:25 UTC

[fedora-14-x64](joanna@dom0)$ qvm-run -a wififun2 gnome-terminal
Running command on VM: 'wififun2'...
Starting the VM 'wififun2'...
--> Loading the VM (type = AppVM)...
*** glibc detected *** /usr/sbin/xl: free(): invalid next size (fast): 0x00000000008147f0 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x75746)[Memory map: ========
00400000-0041b000 r-xp 00000000 fd:01 2363397                            /usr/sbin/xl
0061a000-0061c000 rw-p 0001a000 fd:01 2363397                            /usr/sbin/xl
00814000-00816000 rw-p 00000000 00:00 0                                  [heap](0x7ff80d442746]
/usr/lib64/libxenlight.so.1.0(+0x1a5a6)[0x7ff80e5c95a6]
/usr/lib64/libxenlight.so.1.0(libxl_device_pci_list_assignable+0x1ef)[0x7ff80e5c430f]
/usr/lib64/libxenlight.so.1.0(+0x15ef9)[0x7ff80e5c4ef9]
/usr/lib64/libxenlight.so.1.0(+0x1108e)[0x7ff80e5c008e]
/usr/sbin/xl[0x41244f]
/usr/sbin/xl[0x412f0d]
/usr/sbin/xl[0x404e4a]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x7ff80d3ebc5d]
/usr/sbin/xl[0x404b29]
=======)
00816000-00817000 rw-p 00000000 00:00 0                                  [rw-p 00000000 00:00 0                                  [heap](heap]
00817000-0083c000)
0083c000-0083d000 rw-p 00000000 00:00 0                                  [rw-p 00000000 00:00 0                                  [heap](heap]
0083d000-0085c000)
7ff80c954000-7ff80c96a000 r-xp 00000000 fd:01 6815746                    /lib64/libgcc_s-4.4.4-20100503.so.1
7ff80c96a000-7ff80cb69000 ---p 00016000 fd:01 6815746                    /lib64/libgcc_s-4.4.4-20100503.so.1
7ff80cb69000-7ff80cb6a000 rw-p 00015000 fd:01 6815746                    /lib64/libgcc_s-4.4.4-20100503.so.1
7ff80cb6a000-7ff80cb7f000 r-xp 00000000 fd:01 6815802                    /lib64/libz.so.1.2.3
7ff80cb7f000-7ff80cd7e000 ---p 00015000 fd:01 6815802                    /lib64/libz.so.1.2.3
7ff80cd7e000-7ff80cd7f000 rw-p 00014000 fd:01 6815802                    /lib64/libz.so.1.2.3
7ff80cd7f000-7ff80cd9f000 r-xp 00000000 fd:01 2360417                    /usr/lib64/liblzma.so.0.0.0
7ff80cd9f000-7ff80cf9f000 ---p 00020000 fd:01 2360417                    /usr/lib64/liblzma.so.0.0.0
7ff80cf9f000-7ff80cfa0000 rw-p 00020000 fd:01 2360417                    /usr/lib64/liblzma.so.0.0.0
7ff80cfa0000-7ff80cfb0000 r-xp 00000000 fd:01 6815806                    /lib64/libbz2.so.1.0.4
7ff80cfb0000-7ff80d1af000 ---p 00010000 fd:01 6815806                    /lib64/libbz2.so.1.0.4
7ff80d1af000-7ff80d1b1000 rw-p 0000f000 fd:01 6815806                    /lib64/libbz2.so.1.0.4
7ff80d1b1000-7ff80d1c8000 r-xp 00000000 fd:01 6815782                    /lib64/libpthread-2.12.so
7ff80d1c8000-7ff80d3c7000 ---p 00017000 fd:01 6815782                    /lib64/libpthread-2.12.so
7ff80d3c7000-7ff80d3c8000 r--p 00016000 fd:01 6815782                    /lib64/libpthread-2.12.so
7ff80d3c8000-7ff80d3c9000 rw-p 00017000 fd:01 6815782                    /lib64/libpthread-2.12.so
7ff80d3c9000-7ff80d3cd000 rw-p 00000000 00:00 0 
7ff80d3cd000-7ff80d542000 r-xp 00000000 fd:01 6815758                    /lib64/libc-2.12.so
7ff80d542000-7ff80d742000 ---p 00175000 fd:01 6815758                    /lib64/libc-2.12.so
7ff80d742000-7ff80d746000 r--p 00175000 fd:01 6815758                    /lib64/libc-2.12.so
7ff80d746000-7ff80d747000 rw-p 00179000 fd:01 6815758                    /lib64/libc-2.12.so
7ff80d747000-7ff80d74c000 rw-p 00000000 00:00 0 
7ff80d74c000-7ff80d750000 r-xp 00000000 fd:01 6815836                    /lib64/libuuid.so.1.3.0
7ff80d750000-7ff80d94f000 ---p 00004000 fd:01 6815836                    /lib64/libuuid.so.1.3.0
7ff80d94f000-7ff80d950000 rw-p 00003000 fd:01 6815836                    /lib64/libuuid.so.1.3.0
7ff80d950000-7ff80d952000 r-xp 00000000 fd:01 6815790                    /lib64/libutil-2.12.so
7ff80d952000-7ff80db51000 ---p 00002000 fd:01 6815790                    /lib64/libutil-2.12.so
7ff80db51000-7ff80db52000 r--p 00001000 fd:01 6815790                    /lib64/libutil-2.12.so
7ff80db52000-7ff80db53000 rw-p 00002000 fd:01 6815790                    /lib64/libutil-2.12.so
7ff80db53000-7ff80db5a000 r-xp 00000000 fd:01 2369333                    /usr/lib64/libblktapctl.so.1.0.0
7ff80db5a000-7ff80dd59000 ---p 00007000 fd:01 2369333                    /usr/lib64/libblktapctl.so.1.0.0
7ff80dd59000-7ff80dd5a000 rw-p 00006000 fd:01 2369333                    /usr/lib64/libblktapctl.so.1.0.0
7ff80dd5a000-7ff80dd5f000 r-xp 00000000 fd:01 2369741                    /usr/lib64/libxenstore.so.3.0.0
7ff80dd5f000-7ff80df5f000 ---p 00005000 fd:01 2369741                    /usr/lib64/libxenstore.so.3.0.0
7ff80df5f000-7ff80df60000 rw-p 00005000 fd:01 2369741                    /usr/lib64/libxenstore.so.3.0.0
7ff80df60000-7ff80df63000 rw-p 00000000 00:00 0 
7ff80df63000-7ff80df87000 r-xp 00000000 fd:01 2369739                    /usr/lib64/libxenguest.so.4.0.0
7ff80df87000-7ff80e186000 ---p 00024000 fd:01 2369739                    /usr/lib64/libxenguest.so.4.0.0
7ff80e186000-7ff80e188000 rw-p 00023000 fd:01 2369739                    /usr/lib64/libxenguest.so.4.0.0
7ff80e188000-7ff80e18a000 r-xp 00000000 fd:01 6815764                    /lib64/libdl-2.12.so
7ff80e18a000-7ff80e38a000 ---p 00002000 fd:01 6815764                    /lib64/libdl-2.12.so
7ff80e38a000-7ff80e38b000 r--p 00002000 fd:01 6815764                    /lib64/libdl-2.12.so
7ff80e38b000-7ff80e38c000 rw-p 00003000 fd:01 6815764                    /lib64/libdl-2.12.so
7ff80e38c000-7ff80e3ae000 r-xp 00000000 fd:01 2360567                    /usr/lib64/libxenctrl.so.4.0.0
7ff80e3ae000-7ff80e5ae000 ---p 00022000 fd:01 2360567                    /usr/lib64/libxenctrl.so.4.0.0
7ff80e5ae000-7ff80e5af000 rw-p 00022000 fd:01 2360567                    /usr/lib64/libxenctrl.so.4.0.0
7ff80e5af000-7ff80e5d7000 r-xp 00000000 fd:01 2369740                    /usr/lib64/libxenlight.so.1.0.0
7ff80e5d7000-7ff80e7d7000 ---p 00028000 fd:01 2369740                    /usr/lib64/libxenlight.so.1.0.0
7ff80e7d7000-7ff80e7d8000 rw-p 00028000 fd:01 2369740                    /usr/lib64/libxenlight.so.1.0.0
7ff80e7d8000-7ff80e7e0000 r-xp 00000000 fd:01 2369742                    /usr/lib64/libxlutil.so.1.0.0
7ff80e7e0000-7ff80e9df000 ---p 00008000 fd:01 2369742                    /usr/lib64/libxlutil.so.1.0.0
7ff80e9df000-7ff80e9e0000 rw-p 00007000 fd:01 2369742                    /usr/lib64/libxlutil.so.1.0.0
7ff80e9e0000-7ff80e9fe000 r-xp 00000000 fd:01 6815751                    /lib64/ld-2.12.so
7ff80ebe8000-7ff80ebf0000 rw-p 00000000 00:00 0 
7ff80ebfd000-7ff80ebfe000 rw-p 00000000 00:00 0 ERROR: Failed to load VM config
[fedora-14-x64](joanna@dom0)$ 

xen 4.1.1-4, core 1.6.23, dom0 kernel 2.6.38.3-10

Migrated-From: https://wiki.qubes-os.org/ticket/351

[marmarek]

Comment by joanna on 12 Sep 2011 14:29 UTC
Ok, the above seemed to happen because there was another driver domain running that had the same PCI dev assigned. Still, xl (and qubes.py/qvm-start/qvm-run) should handle this somehow nicer than crashing on a free()...

[marmarek]

Comment by joanna on 12 Sep 2011 14:29 UTC
Ok, the above seemed to happen because there was another driver domain running that had the same PCI dev assigned. Still, xl (and qubes.py/qvm-start/qvm-run) should handle this somehow nicer than crashing on a free()...

[marmarek]

Comment by joanna on 12 Sep 2011 14:30 UTC
Once I shutdown the other VM (and waited until it disappears from xl list -- which as some 10sec!), then I could start my new driver domain fine.

[marmarek]

Comment by joanna on 12 Sep 2011 14:30 UTC
Once I shutdown the other VM (and waited until it disappears from xl list -- which as some 10sec!), then I could start my new driver domain fine.

[marmarek]

Modified by marmarek on 13 Sep 2011 14:35 UTC

[marmarek]

Modified by marmarek on 13 Sep 2011 14:35 UTC

[marmarek]

Comment by marmarek on 13 Sep 2011 17:12 UTC
http://git.qubes-os.org/gitweb/?p=marmarek/xen.git;a=commit;h=9bdf07a8b5f8d70247bf05b0c4a32d6fc43b8e01

[marmarek]

Comment by marmarek on 13 Sep 2011 17:12 UTC
http://git.qubes-os.org/gitweb/?p=marmarek/xen.git;a=commit;h=9bdf07a8b5f8d70247bf05b0c4a32d6fc43b8e01