Trying to Checkpoint MySql container - criu failed: type NOTIFY errno 0 #14

klesgidisold · 2015-08-04T16:39:36Z

Docker:

Client:
 Version:      1.8.0-dev
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   24e477b
 Built:        Sun Aug  2 11:18:41 UTC 2015
 OS/Arch:      linux/amd64
 Experimental: true

Server:
 Version:      1.8.0-dev
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   24e477b
 Built:        Sun Aug  2 11:18:41 UTC 2015
 OS/Arch:      linux/amd64
 Experimental: true

CRIU:

Version: 1.6
GitID: v1.6-147-g8ef2577

I am trying to C/R a MySQL container.

I start it

CONTAINER ID        IMAGE               COMMAND                CREATED             STATUS              PORTS               NAMES
99f990b401b0        mysql:5.7           "/entrypoint.sh mysq   2 days ago          Up 2 minutes        3306/tcp            mysql_test

and then I am trying to checkpoint it. The result is:

$ docker checkpoint mysql_test
Error response from daemon: Cannot checkpoint container mysql_test: criu failed: type NOTIFY errno 0
Error: failed to checkpoint one or more containers

dump.log

(00.000033) Added /var/lib/mysql:/var/lib/mysql ext mount mapping
(00.000058) Added /etc/resolv.conf:/etc/resolv.conf ext mount mapping
(00.000060) Added /etc/hostname:/etc/hostname ext mount mapping
(00.000061) Added /etc/hosts:/etc/hosts ext mount mapping
(00.000062) ========================================
(00.000064) Dumping processes (pid: 3076)
(00.000065) ========================================
(00.000125) Found anon-shmem device at 5
(00.000131) Reset 3176's dirty tracking
(00.000169)  ... done
(00.000193) Dirty track supported on kernel
(00.000259) irmap: Searching irmap cache in work dir
(00.000267) No irmap-cache image
(00.000270) irmap: Searching irmap cache in parent
(00.000278) irmap: No irmap cache
(00.000282) cpu: fpu:1 fxsr:1 xsave:1
(00.000363) vdso: Parsing at 7fff2d2ce000 7fff2d2d0000
(00.000366) vdso: PT_LOAD p_vaddr: 0
(00.000367) vdso: DT_HASH: 0x120
(00.000369) vdso: DT_STRTAB: 0x268
(00.000370) vdso: DT_SYMTAB: 0x160
(00.000371) vdso: DT_STRSZ: 94
(00.000372) vdso: DT_SYMENT: 24
(00.000373) vdso: nbucket 3 nchain 11 bucket 0x7fff2d2ce128 chain 0x7fff2d2ce134
(00.000376) vdso: rt [vdso] 7fff2d2ce000-7fff2d2d0000 [vvar] 7fff2d2cc000-7fff2d2ce000
(00.000425) Writing image inventory (version 1)
(00.000450) Add pid ns 1 pid 3176
(00.000461) Add net ns 2 pid 3176
(00.000469) Add ipc ns 3 pid 3176
(00.000474) Add uts ns 4 pid 3176
(00.000480) Add mnt ns 5 pid 3176
(00.000485) Add user ns 6 pid 3176
(00.000487) cg: Dumping cgroups for 3176
(00.000532) cg:  `- New css ID 1
(00.000535) cg:     `- [blkio] -> [/]
(00.000536) cg:     `- [cpu] -> [/]
(00.000537) cg:     `- [cpuacct] -> [/]
(00.000538) cg:     `- [cpuset] -> [/]
(00.000539) cg:     `- [devices] -> [/]
(00.000540) cg:     `- [freezer] -> [/]
(00.000541) cg:     `- [hugetlb] -> [/]
(00.000542) cg:     `- [memory] -> [/]
(00.000543) cg:     `- [name=systemd] -> [/]
(00.000544) cg:     `- [net_cls] -> [/]
(00.000545) cg:     `- [net_prio] -> [/]
(00.000546) cg:     `- [perf_event] -> [/]
(00.000547) cg: Set 1 is criu one
(00.000870) Seized task 3076, state 1
(00.000937)     Seizing 3076's 3108 thread
(00.000988)     Seizing 3076's 3109 thread
(00.001341)     Seizing 3076's 3110 thread
(00.001391)     Seizing 3076's 3111 thread
(00.001435)     Seizing 3076's 3112 thread
(00.001477)     Seizing 3076's 3113 thread
(00.001517)     Seizing 3076's 3114 thread
(00.001557)     Seizing 3076's 3115 thread
(00.001597)     Seizing 3076's 3116 thread
(00.001636)     Seizing 3076's 3117 thread
(00.001676)     Seizing 3076's 3118 thread
(00.001717)     Seizing 3076's 3119 thread
(00.001758)     Seizing 3076's 3126 thread
(00.001823)     Seizing 3076's 3127 thread
(00.001864)     Seizing 3076's 3128 thread
(00.001904)     Seizing 3076's 3129 thread
(00.001944)     Seizing 3076's 3130 thread
(00.001983)     Seizing 3076's 3131 thread
(00.002023)     Seizing 3076's 3132 thread
(00.002062)     Seizing 3076's 3133 thread
(00.002102)     Seizing 3076's 3134 thread
(00.002142)     Seizing 3076's 3135 thread
(00.002181)     Seizing 3076's 3136 thread
(00.002363) Collected 3076 in 1 state
(00.002376) Will take pid namespace in the image
(00.002378) Add pid ns 7 pid 3076
(00.002382) Will take net namespace in the image
(00.002383) Add net ns 8 pid 3076
(00.002387) Will take ipc namespace in the image
(00.002388) Add ipc ns 9 pid 3076
(00.002392) Will take uts namespace in the image
(00.002393) Add uts ns 10 pid 3076
(00.002397) Will take mnt namespace in the image
(00.002398) Add mnt ns 11 pid 3076
(00.002402) Lock network
(00.002404) Running network-lock scripts
(00.002413)     RPC
(00.002621) lockinfo: 1:1 1 3076 08:06:1985361 0 EOF
(00.002627) lockinfo: 2:1 1 3076 08:06:1985359 0 EOF
(00.002630) lockinfo: 3:1 1 3076 08:06:1985357 0 EOF
(00.002634) lockinfo: 4:1 1 3076 08:06:1985391 0 EOF
(00.002637) lockinfo: 5:1 1 3076 08:06:1985383 0 EOF
(00.002640) lockinfo: 6:1 1 3076 08:06:1985381 0 EOF
(00.002644) lockinfo: 7:1 1 3076 08:06:1985353 0 EOF
(00.002647) lockinfo: 8:1 1 3076 08:06:1985324 0 EOF
(00.002650) lockinfo: 9:1 1 3076 08:06:1985322 0 EOF
(00.002654) lockinfo: 10:1 0 2179 08:06:3935993 128 128
(00.002657) lockinfo: 11:1 0 2179 08:06:3935991 1073741826 1073742335
(00.002666) lockinfo: 12:1 0 2380 08:06:3957354 128 128
(00.002669) lockinfo: 13:1 0 2380 08:06:3936035 1073741826 1073742335
(00.002673) lockinfo: 14:1 0 2380 08:06:3956394 128 128
(00.002676) lockinfo: 15:1 0 2380 08:06:3936037 1073741826 1073742335
(00.002680) lockinfo: 16:1 0 2380 08:06:3954068 128 128
(00.002707) lockinfo: 17:1 0 2380 08:06:3936033 1073741826 1073742335
(00.002711) lockinfo: 18:1 0 2380 08:06:3936029 128 128
(00.002747) lockinfo: 19:1 0 2380 08:06:3936027 1073741826 1073742335
(00.002750) lockinfo: 20:1 1 2376 08:06:3935995 0 0
(00.002754) lockinfo: 21:1 0 2365 08:06:3935993 128 128
(00.002757) lockinfo: 22:1 0 2365 08:06:3935991 1073741826 1073742335
(00.002760) lockinfo: 23:1 0 2359 08:06:3935993 128 128
(00.002773) lockinfo: 24:1 0 2359 08:06:3935991 1073741826 1073742335
(00.002777) lockinfo: 25:2 1 916 00:14:12057 0 EOF
(00.002780) lockinfo: 26:1 1 948 08:06:1967392 0 EOF
(00.002783) lockinfo: 27:2 1 879 00:11:11417 0 EOF
(00.002787) lockinfo: 28:1 1 3076 08:06:1985395 0 EOF
(00.002790) lockinfo: 29:1 1 3076 08:06:1985393 0 EOF
(00.002793) lockinfo: 30:1 1 3076 08:06:1985355 0 EOF
(00.002797) lockinfo: 31:1 1 3076 08:06:1984658 0 EOF
(00.002800) lockinfo: 32:1 1 3076 08:06:1985287 0 EOF
(00.002803) lockinfo: 33:1 1 3076 08:06:1985285 0 EOF
(00.002806) lockinfo: 34:1 1 3076 08:06:1985283 0 EOF
(00.002810) lockinfo: 35:1 0 2556 08:06:3935993 128 128
(00.002813) lockinfo: 36:1 0 2556 08:06:3935991 1073741826 1073742335
(00.002822) lockinfo: 37:1 1 2380 08:06:3936016 0 EOF
(00.002826) lockinfo: 38:1 0 2354 08:06:3935993 128 128
(00.002829) lockinfo: 39:1 0 2354 08:06:3935991 1073741826 1073742335
(00.002933)     type aufs source none mnt_id 81 s_dev 0x26 / @ ./ flags 0x200000 options si=3f48c2e58a26870f,dio,dirperm1
(00.002937) Collecting AUFS branch pathnames ...
(00.003021) /sys/fs/aufs/si_3f48c2e58a26870f/br0 : /var/lib/docker/aufs/diff/99f990b401b0a4035a59dbaec80d87cebcbb1e74d863687b97049d324f93a2af
(00.003042) /sys/fs/aufs/si_3f48c2e58a26870f/br1 : /var/lib/docker/aufs/diff/99f990b401b0a4035a59dbaec80d87cebcbb1e74d863687b97049d324f93a2af-init
(00.003061) /sys/fs/aufs/si_3f48c2e58a26870f/br2 : /var/lib/docker/aufs/diff/fdb9c2185fc2d1518457e5f5d03303846d0d71de199c3bf312cfacbfd81f8c59
(00.003082) /sys/fs/aufs/si_3f48c2e58a26870f/br3 : /var/lib/docker/aufs/diff/7b4030a6b20ba9f0daec8f5b3baf6e579a087e54f2c3ed748596c899819a5083
(00.003102) /sys/fs/aufs/si_3f48c2e58a26870f/br4 : /var/lib/docker/aufs/diff/099dc92e9b15dd5a8699b105a9194615371e5e59a9ad73b706ad4b03cad644d8
(00.003121) /sys/fs/aufs/si_3f48c2e58a26870f/br5 : /var/lib/docker/aufs/diff/6e2ce31eb835b1d582ae0722e7b8b2634b1bcef84e07502f6b64f041bd646a5b
(00.003140) /sys/fs/aufs/si_3f48c2e58a26870f/br6 : /var/lib/docker/aufs/diff/5191ddfc3b018a7a88b54e602d9ebc4983c8caf3bf289594420ffd59f9ff44d1
(00.003160) /sys/fs/aufs/si_3f48c2e58a26870f/br7 : /var/lib/docker/aufs/diff/a9057a689e837969e7d9c24d0b1df21b0c171d3224266d82bcd8efdb5211179d
(00.003182) /sys/fs/aufs/si_3f48c2e58a26870f/br8 : /var/lib/docker/aufs/diff/ac0ee0367a322d51526614629cc60ec6c182624893925bea2557337dcea9c194
(00.003201) /sys/fs/aufs/si_3f48c2e58a26870f/br9 : /var/lib/docker/aufs/diff/059dbbef4dd8f26a1fc5e0e60188052d6fec14e7af889be0f530ed3bf2e6b391
(00.003220) /sys/fs/aufs/si_3f48c2e58a26870f/br10 : /var/lib/docker/aufs/diff/2d1023ecaed49e18b0a7cb473702bab72601deee129d8b7c2eb3426cc1f147f0
(00.003239) /sys/fs/aufs/si_3f48c2e58a26870f/br11 : /var/lib/docker/aufs/diff/f928e22e58b0596faa4c2e45fd12195a1e616c3b096e10e6856446c53db6dd18
(00.003258) /sys/fs/aufs/si_3f48c2e58a26870f/br12 : /var/lib/docker/aufs/diff/543ff72402d8d796cae34d4c980e247debd9bace8a7e22819dd1693541bb084e
(00.003277) /sys/fs/aufs/si_3f48c2e58a26870f/br13 : /var/lib/docker/aufs/diff/273cd71eacf0b918851435d73dcfeb067239b9bfcae37def64f4e13726299eb1
(00.003296) /sys/fs/aufs/si_3f48c2e58a26870f/br14 : /var/lib/docker/aufs/diff/c2b0136be90f7370c0a3372157f53f9b84e15a91b34e73011b47f62dd23077e7
(00.003314) /sys/fs/aufs/si_3f48c2e58a26870f/br15 : /var/lib/docker/aufs/diff/60c52dbe9d9121f0baf4d44fece2d447a0a48f4da84522f0eb7082a0ca6b465e
(00.003339) /sys/fs/aufs/si_3f48c2e58a26870f/br16 : /var/lib/docker/aufs/diff/4c8cbfd2973e1b92137486922e1af5677fced4f6902e6513b81413e22f33ceab
(00.003348)     type proc source proc mnt_id 82 s_dev 0x28 / @ ./proc flags 0x20000e options 
(00.003354)     type tmpfs source tmpfs mnt_id 83 s_dev 0x29 / @ ./dev flags 0x1000000 options mode=755
(00.003360)     type devpts source devpts mnt_id 84 s_dev 0x2a / @ ./dev/pts flags 0x20000a options gid=5,mode=620,ptmxmode=666
(00.003379)     type tmpfs source shm mnt_id 85 s_dev 0x2b / @ ./dev/shm flags 0x20000e options size=65536k
(00.003385)     type mqueue source mqueue mnt_id 86 s_dev 0x27 / @ ./dev/mqueue flags 0x20000e options 
(00.003391)     type sysfs source sysfs mnt_id 87 s_dev 0x2c / @ ./sys flags 0x20000f options 
(00.003396)     type tmpfs source tmpfs mnt_id 88 s_dev 0x2d / @ ./sys/fs/cgroup flags 0x20000e options 
(00.003401)     type cgroup source cgroup mnt_id 89 s_dev 0x18 /docker/99f990b401b0a4035a59dbaec80d87cebcbb1e74d863687b97049d324f93a2af @ ./sys/fs/cgroup/cpuset flags 0x20000f options cpuset
(00.003407)     type cgroup source cgroup mnt_id 90 s_dev 0x19 /docker/99f990b401b0a4035a59dbaec80d87cebcbb1e74d863687b97049d324f93a2af @ ./sys/fs/cgroup/cpu flags 0x20000f options cpu
(00.003443)     type cgroup source cgroup mnt_id 91 s_dev 0x1a /docker/99f990b401b0a4035a59dbaec80d87cebcbb1e74d863687b97049d324f93a2af @ ./sys/fs/cgroup/cpuacct flags 0x20000f options cpuacct
(00.003450)     type cgroup source cgroup mnt_id 92 s_dev 0x1b /docker/99f990b401b0a4035a59dbaec80d87cebcbb1e74d863687b97049d324f93a2af @ ./sys/fs/cgroup/memory flags 0x20000f options memory
(00.003459)     type cgroup source cgroup mnt_id 93 s_dev 0x1c /docker/99f990b401b0a4035a59dbaec80d87cebcbb1e74d863687b97049d324f93a2af @ ./sys/fs/cgroup/devices flags 0x20000f options devices
(00.003465)     type cgroup source cgroup mnt_id 94 s_dev 0x1d /docker/99f990b401b0a4035a59dbaec80d87cebcbb1e74d863687b97049d324f93a2af @ ./sys/fs/cgroup/freezer flags 0x20000f options freezer
(00.003471)     type cgroup source cgroup mnt_id 95 s_dev 0x1e /docker/99f990b401b0a4035a59dbaec80d87cebcbb1e74d863687b97049d324f93a2af @ ./sys/fs/cgroup/net_cls flags 0x20000f options net_cls
(00.003476)     type cgroup source cgroup mnt_id 96 s_dev 0x1f /docker/99f990b401b0a4035a59dbaec80d87cebcbb1e74d863687b97049d324f93a2af @ ./sys/fs/cgroup/blkio flags 0x20000f options blkio
(00.003512)     type cgroup source cgroup mnt_id 97 s_dev 0x20 /docker/99f990b401b0a4035a59dbaec80d87cebcbb1e74d863687b97049d324f93a2af @ ./sys/fs/cgroup/perf_event flags 0x20000f options perf_event
(00.003518)     type cgroup source cgroup mnt_id 98 s_dev 0x21 /docker/99f990b401b0a4035a59dbaec80d87cebcbb1e74d863687b97049d324f93a2af @ ./sys/fs/cgroup/net_prio flags 0x20000f options net_prio
(00.003524)     type cgroup source cgroup mnt_id 99 s_dev 0x22 /docker/99f990b401b0a4035a59dbaec80d87cebcbb1e74d863687b97049d324f93a2af @ ./sys/fs/cgroup/hugetlb flags 0x20000f options hugetlb
(00.003529)     type cgroup source systemd mnt_id 100 s_dev 0x23 / @ ./sys/fs/cgroup/systemd flags 0x20000f options name=systemd
(00.003535)     type ext4 source /dev/disk/by-uuid/b2c933d5-6fa6-4035-980a-ac7ae5fe64d2 mnt_id 101 s_dev 0x800006 /var/lib/docker/vfs/dir/e388120b0d2367b183ecf63e1a1d88efaa17fc1ea7b34441a923fe40ead740f9 @ ./var/lib/mysql flags 0x200000 options errors=remount-ro,data=ordered
(00.003545)     type ext4 source /dev/disk/by-uuid/b2c933d5-6fa6-4035-980a-ac7ae5fe64d2 mnt_id 102 s_dev 0x800006 /var/lib/docker/containers/99f990b401b0a4035a59dbaec80d87cebcbb1e74d863687b97049d324f93a2af/resolv.conf @ ./etc/resolv.conf flags 0x200000 options errors=remount-ro,data=ordered
(00.003570)     type ext4 source /dev/disk/by-uuid/b2c933d5-6fa6-4035-980a-ac7ae5fe64d2 mnt_id 103 s_dev 0x800006 /var/lib/docker/containers/99f990b401b0a4035a59dbaec80d87cebcbb1e74d863687b97049d324f93a2af/hostname @ ./etc/hostname flags 0x200000 options errors=remount-ro,data=ordered
(00.003577)     type ext4 source /dev/disk/by-uuid/b2c933d5-6fa6-4035-980a-ac7ae5fe64d2 mnt_id 104 s_dev 0x800006 /var/lib/docker/containers/99f990b401b0a4035a59dbaec80d87cebcbb1e74d863687b97049d324f93a2af/hosts @ ./etc/hosts flags 0x200000 options errors=remount-ro,data=ordered
(00.003587)     type proc source proc mnt_id 50 s_dev 0x28 /asound @ ./proc/asound flags 0x20000f options 
(00.003593)     type proc source proc mnt_id 51 s_dev 0x28 /bus @ ./proc/bus flags 0x20000f options 
(00.003598)     type proc source proc mnt_id 52 s_dev 0x28 /fs @ ./proc/fs flags 0x20000f options 
(00.003603)     type proc source proc mnt_id 53 s_dev 0x28 /irq @ ./proc/irq flags 0x20000f options 
(00.003608)     type proc source proc mnt_id 54 s_dev 0x28 /sys @ ./proc/sys flags 0x20000f options 
(00.003613)     type proc source proc mnt_id 55 s_dev 0x28 /sysrq-trigger @ ./proc/sysrq-trigger flags 0x20000f options 
(00.003618)     type tmpfs source tmpfs mnt_id 56 s_dev 0x29 /null @ ./proc/kcore flags 0x1000000 options mode=755
(00.003623)     type tmpfs source tmpfs mnt_id 57 s_dev 0x29 /null @ ./proc/timer_stats flags 0x1000000 options mode=755
(00.003634) Building mountpoints tree
(00.003636)     Building plain mount tree
(00.003637)         Working on 57->82
(00.003638)         Working on 56->82
(00.003639)         Working on 55->82
(00.003640)         Working on 54->82
(00.003641)         Working on 53->82
(00.003643)         Working on 52->82
(00.003644)         Working on 51->82
(00.003645)         Working on 50->82
(00.003646)         Working on 104->81
(00.003647)         Working on 103->81
(00.003648)         Working on 102->81
(00.003649)         Working on 101->81
(00.003650)         Working on 100->88
(00.003651)         Working on 99->88
(00.003652)         Working on 98->88
(00.003653)         Working on 97->88
(00.003654)         Working on 96->88
(00.003655)         Working on 95->88
(00.003656)         Working on 94->88
(00.003657)         Working on 93->88
(00.003658)         Working on 92->88
(00.003659)         Working on 91->88
(00.003661)         Working on 90->88
(00.003662)         Working on 89->88
(00.003663)         Working on 88->87
(00.003664)         Working on 87->81
(00.003665)         Working on 86->83
(00.003666)         Working on 85->83
(00.003667)         Working on 84->83
(00.003668)         Working on 83->81
(00.003669)         Working on 82->81
(00.003670)         Working on 81->48
(00.003672)     Resorting siblings on 81
(00.003673)     Resorting siblings on 104
(00.003674)     Resorting siblings on 103
(00.003675)     Resorting siblings on 102
(00.003676)     Resorting siblings on 101
(00.003677)     Resorting siblings on 87
(00.003678)     Resorting siblings on 88
(00.003679)     Resorting siblings on 100
(00.003680)     Resorting siblings on 99
(00.003681)     Resorting siblings on 98
(00.003682)     Resorting siblings on 97
(00.003683)     Resorting siblings on 96
(00.003684)     Resorting siblings on 95
(00.003685)     Resorting siblings on 94
(00.003686)     Resorting siblings on 93
(00.003687)     Resorting siblings on 92
(00.003688)     Resorting siblings on 91
(00.003689)     Resorting siblings on 90
(00.003690)     Resorting siblings on 89
(00.003691)     Resorting siblings on 83
(00.003692)     Resorting siblings on 86
(00.003693)     Resorting siblings on 85
(00.003694)     Resorting siblings on 84
(00.003695)     Resorting siblings on 82
(00.003696)     Resorting siblings on 57
(00.003697)     Resorting siblings on 56
(00.003698)     Resorting siblings on 55
(00.003699)     Resorting siblings on 54
(00.003700)     Resorting siblings on 53
(00.003701)     Resorting siblings on 52
(00.003702)     Resorting siblings on 51
(00.003703)     Resorting siblings on 50
(00.003704) Done:
(00.003705) [./](81->48)
(00.003707)  [./sys](87->81)
(00.003708)   [./sys/fs/cgroup](88->87)
(00.003710)    [./sys/fs/cgroup/systemd](100->88)
(00.003711)    <--
(00.003712)    [./sys/fs/cgroup/cpuset](89->88)
(00.003713)    <--
(00.003714)    [./sys/fs/cgroup/cpu](90->88)
(00.003715)    <--
(00.003716)    [./sys/fs/cgroup/cpuacct](91->88)
(00.003717)    <--
(00.003718)    [./sys/fs/cgroup/memory](92->88)
(00.003720)    <--
(00.003720)    [./sys/fs/cgroup/devices](93->88)
(00.003722)    <--
(00.003723)    [./sys/fs/cgroup/freezer](94->88)
(00.003724)    <--
(00.003725)    [./sys/fs/cgroup/net_cls](95->88)
(00.003726)    <--
(00.003727)    [./sys/fs/cgroup/blkio](96->88)
(00.003728)    <--
(00.003729)    [./sys/fs/cgroup/perf_event](97->88)
(00.003730)    <--
(00.003731)    [./sys/fs/cgroup/net_prio](98->88)
(00.003735)    <--
(00.003736)    [./sys/fs/cgroup/hugetlb](99->88)
(00.003737)    <--
(00.003738)   <--
(00.003739)  <--
(00.003740)  [./proc](82->81)
(00.003741)   [./proc/timer_stats](57->82)
(00.003742)   <--
(00.003743)   [./proc/asound](50->82)
(00.003744)   <--
(00.003745)   [./proc/bus](51->82)
(00.003746)   <--
(00.003747)   [./proc/fs](52->82)
(00.003749)   <--
(00.003750)   [./proc/irq](53->82)
(00.003751)   <--
(00.003752)   [./proc/sys](54->82)
(00.003753)   <--
(00.003754)   [./proc/sysrq-trigger](55->82)
(00.003755)   <--
(00.003756)   [./proc/kcore](56->82)
(00.003757)   <--
(00.003758)  <--
(00.003759)  [./dev](83->81)
(00.003760)   [./dev/mqueue](86->83)
(00.003761)   <--
(00.003762)   [./dev/pts](84->83)
(00.003764)   <--
(00.003765)   [./dev/shm](85->83)
(00.003766)   <--
(00.003767)  <--
(00.003768)  [./etc/hosts](104->81)
(00.003769)  <--
(00.003770)  [./var/lib/mysql](101->81)
(00.003771)  <--
(00.003772)  [./etc/resolv.conf](102->81)
(00.003773)  <--
(00.003774)  [./etc/hostname](103->81)
(00.003775)  <--
(00.003776) <--
(00.003778) Found /etc/hosts mapping for ./etc/hosts mountpoint
(00.003780) Found /etc/hostname mapping for ./etc/hostname mountpoint
(00.003781) Found /etc/resolv.conf mapping for ./etc/resolv.conf mountpoint
(00.003782) Found /var/lib/mysql mapping for ./var/lib/mysql mountpoint
(00.003790) Error (mount.c:630): 99:./sys/fs/cgroup/hugetlb doesn't have a proper root mount
(00.003792) Unlock network
(00.003794) Running network-unlock scripts
(00.003800)     RPC
(00.003842) Unfreezing tasks into 1
(00.003845)     Unseizing 3076 into 1
(00.004013) Error (cr-dump.c:1996): Dumping FAILED.

(edit: I also tried to checkpoint with apache, but the same results)

Any help?
Thanks for all the support!

The text was updated successfully, but these errors were encountered:

xemul · 2015-08-04T17:20:08Z

Error (mount.c:630): 99:./sys/fs/cgroup/hugetlb doesn't have a proper root mount

If I understand the docker+criu integration correctly, this mountpoint should be marked with --ext-moun-map option on dump.

Cc-ing @boucher and @avagin

boucher · 2015-08-04T17:22:49Z

This is still a result of the runc/docker bug that was fixed a couple of weeks ago unfortunately. I'm still waiting for this issue[1] to get fixed, so I can submit a pull request to docker to pull in the new version of libcontainer, which will fix the issue.

[1] opencontainers/runc#171

boucher · 2015-08-05T16:37:56Z

I believe if you checkout the latest version of my branch this issue should be fixed.

klesgidisold · 2015-08-05T17:37:14Z

@boucher thank you very much for the quick response! The latest version seems to fixed the previous bug, but I cannot checkpoint my container again.

Logs

(00.000033) Added /sys/fs/cgroup/cpuset:/sys/fs/cgroup/cpuset ext mount mapping
(00.000055) Added /sys/fs/cgroup/cpu:/sys/fs/cgroup/cpu ext mount mapping
(00.000057) Added /sys/fs/cgroup/cpuacct:/sys/fs/cgroup/cpuacct ext mount mapping
(00.000058) Added /sys/fs/cgroup/memory:/sys/fs/cgroup/memory ext mount mapping
(00.000059) Added /sys/fs/cgroup/devices:/sys/fs/cgroup/devices ext mount mapping
(00.000060) Added /sys/fs/cgroup/freezer:/sys/fs/cgroup/freezer ext mount mapping
(00.000062) Added /sys/fs/cgroup/net_cls:/sys/fs/cgroup/net_cls ext mount mapping
(00.000063) Added /sys/fs/cgroup/blkio:/sys/fs/cgroup/blkio ext mount mapping
(00.000064) Added /sys/fs/cgroup/perf_event:/sys/fs/cgroup/perf_event ext mount mapping
(00.000065) Added /sys/fs/cgroup/net_prio:/sys/fs/cgroup/net_prio ext mount mapping
(00.000066) Added /sys/fs/cgroup/hugetlb:/sys/fs/cgroup/hugetlb ext mount mapping
(00.000067) Added /sys/fs/cgroup/systemd:/sys/fs/cgroup/systemd ext mount mapping
(00.000069) Added /var/lib/mysql:/var/lib/mysql ext mount mapping
(00.000070) Added /etc/resolv.conf:/etc/resolv.conf ext mount mapping
(00.000071) Added /etc/hostname:/etc/hostname ext mount mapping
(00.000072) Added /etc/hosts:/etc/hosts ext mount mapping
(00.000125) ========================================
(00.000136) Dumping processes (pid: 9363)
(00.000138) ========================================
(00.000191) Found anon-shmem device at 5
(00.000198) Reset 9892's dirty tracking
(00.000236)  ... done
(00.000258) Dirty track supported on kernel
(00.000321) irmap: Searching irmap cache in work dir
(00.000329) No irmap-cache image
(00.000333) irmap: Searching irmap cache in parent
(00.000342) irmap: No irmap cache
(00.000345) cpu: fpu:1 fxsr:1 xsave:1
(00.000425) vdso: Parsing at 7ffef239c000 7ffef239e000
(00.000428) vdso: PT_LOAD p_vaddr: 0
(00.000430) vdso: DT_HASH: 0x120
(00.000431) vdso: DT_STRTAB: 0x268
(00.000432) vdso: DT_SYMTAB: 0x160
(00.000433) vdso: DT_STRSZ: 94
(00.000434) vdso: DT_SYMENT: 24
(00.000435) vdso: nbucket 3 nchain 11 bucket 0x7ffef239c128 chain 0x7ffef239c134
(00.000439) vdso: rt [vdso] 7ffef239c000-7ffef239e000 [vvar] 7ffef239a000-7ffef239c000
(00.000485) Writing image inventory (version 1)
(00.000511) Add pid ns 1 pid 9892
(00.000518) Add net ns 2 pid 9892
(00.000523) Add ipc ns 3 pid 9892
(00.000533) Add uts ns 4 pid 9892
(00.000540) Add mnt ns 5 pid 9892
(00.000546) Add user ns 6 pid 9892
(00.000548) cg: Dumping cgroups for 9892
(00.000578) cg:  `- New css ID 1
(00.000580) cg:     `- [blkio] -> [/]
(00.000581) cg:     `- [cpu] -> [/]
(00.000582) cg:     `- [cpuacct] -> [/]
(00.000583) cg:     `- [cpuset] -> [/]
(00.000585) cg:     `- [devices] -> [/]
(00.000586) cg:     `- [freezer] -> [/]
(00.000587) cg:     `- [hugetlb] -> [/]
(00.000588) cg:     `- [memory] -> [/]
(00.000589) cg:     `- [name=systemd] -> [/]
(00.000590) cg:     `- [net_cls] -> [/]
(00.000591) cg:     `- [net_prio] -> [/]
(00.000592) cg:     `- [perf_event] -> [/]
(00.000593) cg: Set 1 is criu one
(00.000788) Seized task 9363, state 1
(00.000830)     Seizing 9363's 9427 thread
(00.000872)     Seizing 9363's 9428 thread
(00.000911)     Seizing 9363's 9429 thread
(00.000949)     Seizing 9363's 9430 thread
(00.000987)     Seizing 9363's 9431 thread
(00.001023)     Seizing 9363's 9432 thread
(00.001060)     Seizing 9363's 9433 thread
(00.001097)     Seizing 9363's 9434 thread
(00.001134)     Seizing 9363's 9435 thread
(00.001171)     Seizing 9363's 9436 thread
(00.001208)     Seizing 9363's 9443 thread
(00.001247)     Seizing 9363's 9444 thread
(00.001286)     Seizing 9363's 9445 thread
(00.001325)     Seizing 9363's 9446 thread
(00.001365)     Seizing 9363's 9447 thread
(00.001503) Collected 9363 in 1 state
(00.001514) Will take pid namespace in the image
(00.001516) Add pid ns 7 pid 9363
(00.001520) Will take net namespace in the image
(00.001522) Add net ns 8 pid 9363
(00.001526) Will take ipc namespace in the image
(00.001527) Add ipc ns 9 pid 9363
(00.001531) Will take uts namespace in the image
(00.001532) Add uts ns 10 pid 9363
(00.001536) Will take mnt namespace in the image
(00.001538) Add mnt ns 11 pid 9363
(00.001546) Lock network
(00.001548) Running network-lock scripts
(00.001556)     RPC
(00.001704) lockinfo: 1:1 1 9363 08:06:1985652 0 EOF
(00.001709) lockinfo: 2:1 1 9363 08:06:1985650 0 EOF
(00.001713) lockinfo: 3:1 1 9363 08:06:1985653 0 EOF
(00.001716) lockinfo: 4:1 1 2543 08:06:3817804 1073741824 1073742335
(00.001720) lockinfo: 5:1 1 2543 08:06:3817848 0 EOF
(00.001723) lockinfo: 6:1 0 2543 08:06:3817830 1073741826 1073742335
(00.001727) lockinfo: 7:1 0 2089 08:06:3935993 128 128
(00.001730) lockinfo: 8:1 0 2089 08:06:3935991 1073741826 1073742335
(00.001733) lockinfo: 9:1 0 2286 08:06:3958158 128 128
(00.001737) lockinfo: 10:1 0 2286 08:06:3936035 1073741826 1073742335
(00.001740) lockinfo: 11:1 0 2286 08:06:3958155 128 128
(00.001743) lockinfo: 12:1 0 2286 08:06:3936037 1073741826 1073742335
(00.001747) lockinfo: 13:1 0 2286 08:06:3958146 128 128
(00.001750) lockinfo: 14:1 0 2286 08:06:3936033 1073741826 1073742335
(00.001753) lockinfo: 15:1 0 2286 08:06:3936029 128 128
(00.001757) lockinfo: 16:1 0 2286 08:06:3936027 1073741826 1073742335
(00.001760) lockinfo: 17:1 1 2427 08:06:3935995 0 0
(00.001788) lockinfo: 18:1 0 2378 08:06:3935993 128 128
(00.001791) lockinfo: 19:1 0 2378 08:06:3935991 1073741826 1073742335
(00.001795) lockinfo: 20:1 0 2365 08:06:3935993 128 128
(00.001798) lockinfo: 21:1 0 2365 08:06:3935991 1073741826 1073742335
(00.001801) lockinfo: 22:2 1 984 00:14:11967 0 EOF
(00.001805) lockinfo: 23:1 1 2543 08:06:3817826 1073741824 1073742335
(00.001808) lockinfo: 24:1 1 2543 08:06:3817901 0 EOF
(00.001812) lockinfo: 25:1 1 2543 08:06:3831264 0 EOF
(00.001815) lockinfo: 26:1 1 2543 08:06:3817808 1073741824 1073742335
(00.001819) lockinfo: 27:1 0 2543 08:06:3817852 1073741826 1073742335
(00.001822) lockinfo: 28:1 1 2543 08:06:3817812 0 EOF
(00.001825) lockinfo: 29:1 1 2543 08:06:3818013 0 EOF
(00.001829) lockinfo: 30:1 0 2386 08:06:3935993 128 128
(00.001832) lockinfo: 31:1 0 2386 08:06:3935991 1073741826 1073742335
(00.001835) lockinfo: 32:1 0 2356 08:06:3935993 128 128
(00.001839) lockinfo: 33:1 0 2356 08:06:3935991 1073741826 1073742335
(00.001842) lockinfo: 34:1 1 2286 08:06:3936016 0 EOF
(00.001845) lockinfo: 35:2 1 945 00:11:10721 0 EOF
(00.001912)     type aufs source none mnt_id 82 s_dev 0x27 / @ ./ flags 0x200000 options si=69c8af2391d1877e,dio,dirperm1
(00.001916) Collecting AUFS branch pathnames ...
(00.001978) /sys/fs/aufs/si_69c8af2391d1877e/br0 : /var/lib/docker/aufs/diff/70b2975a004c8028d8bd79f098d51bdd04f56d24b4c58bf1c5ddb0ec3dc374c7
(00.001998) /sys/fs/aufs/si_69c8af2391d1877e/br1 : /var/lib/docker/aufs/diff/70b2975a004c8028d8bd79f098d51bdd04f56d24b4c58bf1c5ddb0ec3dc374c7-init
(00.002018) /sys/fs/aufs/si_69c8af2391d1877e/br2 : /var/lib/docker/aufs/diff/4bde16f3f019ac6613c5df5124eed3eb85821559386d9a43ed481730d70496d4
(00.002037) /sys/fs/aufs/si_69c8af2391d1877e/br3 : /var/lib/docker/aufs/diff/6c4ce88e9ca6319c8313defba73c69f631051f06c79accb6f752928982f6bab8
(00.002056) /sys/fs/aufs/si_69c8af2391d1877e/br4 : /var/lib/docker/aufs/diff/9ff5db60c9c1b69f454fa8d741f4c7771869577fd08693deb3820cd8cec9414d
(00.002075) /sys/fs/aufs/si_69c8af2391d1877e/br5 : /var/lib/docker/aufs/diff/6a3ae4c0a0875d3cc99ef83862bec6b0ad5e788af4a838e98ed79535ec4fae21
(00.002094) /sys/fs/aufs/si_69c8af2391d1877e/br6 : /var/lib/docker/aufs/diff/c81a4d1c27f11e11632ec91626d3a8f00031990607ec4effdaba36be99776a98
(00.002113) /sys/fs/aufs/si_69c8af2391d1877e/br7 : /var/lib/docker/aufs/diff/a725c4d802e6c26979551a3361caa1d5f12f59cf383cfaad17b6f0c423721afd
(00.002132) /sys/fs/aufs/si_69c8af2391d1877e/br8 : /var/lib/docker/aufs/diff/630075b6f62fcc703ede459f770ceabe6c9c4d3453b07ae7030592b22dd9416d
(00.002151) /sys/fs/aufs/si_69c8af2391d1877e/br9 : /var/lib/docker/aufs/diff/fffd34efa8b772f2e147eab051efd92e6d79a478b09ab601b97561e7390c5ad9
(00.002170) /sys/fs/aufs/si_69c8af2391d1877e/br10 : /var/lib/docker/aufs/diff/ff2152de58cec8bab9bd4ffec03e61e0a4e25218045cec9bf1ac27e75a176bb5
(00.002188) /sys/fs/aufs/si_69c8af2391d1877e/br11 : /var/lib/docker/aufs/diff/07f8e8c5e66084bef8f848877857537ffe1c47edd01a93af27e7161672ad0e95
(00.002207) /sys/fs/aufs/si_69c8af2391d1877e/br12 : /var/lib/docker/aufs/diff/37bea4ee0c816e3a3fa025f36127ef8ef0817b3f8fcd7b49eb7b26064f647bb0
(00.002230) /sys/fs/aufs/si_69c8af2391d1877e/br13 : /var/lib/docker/aufs/diff/a82efea989f94b1d9fac76e26e37b0bbde11047a3afcaa47064949dfa3b3209b
(00.002249) /sys/fs/aufs/si_69c8af2391d1877e/br14 : /var/lib/docker/aufs/diff/e9e06b06e14c2f7d8df0251e3bb852c3a10a70639498163d4f180a823c18fdfc
(00.002256)     type proc source proc mnt_id 83 s_dev 0x29 / @ ./proc flags 0x20000e options 
(00.002262)     type tmpfs source tmpfs mnt_id 84 s_dev 0x2a / @ ./dev flags 0x1000000 options mode=755
(00.002267)     type devpts source devpts mnt_id 85 s_dev 0x2b / @ ./dev/pts flags 0x20000a options gid=5,mode=620,ptmxmode=666
(00.002280)     type tmpfs source shm mnt_id 86 s_dev 0x2c / @ ./dev/shm flags 0x20000e options size=65536k
(00.002286)     type mqueue source mqueue mnt_id 87 s_dev 0x28 / @ ./dev/mqueue flags 0x20000e options 
(00.002291)     type sysfs source sysfs mnt_id 88 s_dev 0x2d / @ ./sys flags 0x20000f options 
(00.002296)     type tmpfs source tmpfs mnt_id 89 s_dev 0x2e / @ ./sys/fs/cgroup flags 0x20000f options mode=755
(00.002302)     type cgroup source cgroup mnt_id 90 s_dev 0x16 /docker/70b2975a004c8028d8bd79f098d51bdd04f56d24b4c58bf1c5ddb0ec3dc374c7 @ ./sys/fs/cgroup/cpuset flags 0x20000f options cpuset
(00.002307)     type cgroup source cgroup mnt_id 91 s_dev 0x19 /docker/70b2975a004c8028d8bd79f098d51bdd04f56d24b4c58bf1c5ddb0ec3dc374c7 @ ./sys/fs/cgroup/cpu flags 0x20000f options cpu
(00.002337)     type cgroup source cgroup mnt_id 92 s_dev 0x1a /docker/70b2975a004c8028d8bd79f098d51bdd04f56d24b4c58bf1c5ddb0ec3dc374c7 @ ./sys/fs/cgroup/cpuacct flags 0x20000f options cpuacct
(00.002344)     type cgroup source cgroup mnt_id 93 s_dev 0x1b /docker/70b2975a004c8028d8bd79f098d51bdd04f56d24b4c58bf1c5ddb0ec3dc374c7 @ ./sys/fs/cgroup/memory flags 0x20000f options memory
(00.002349)     type cgroup source cgroup mnt_id 94 s_dev 0x1c /docker/70b2975a004c8028d8bd79f098d51bdd04f56d24b4c58bf1c5ddb0ec3dc374c7 @ ./sys/fs/cgroup/devices flags 0x20000f options devices
(00.002355)     type cgroup source cgroup mnt_id 95 s_dev 0x1d /docker/70b2975a004c8028d8bd79f098d51bdd04f56d24b4c58bf1c5ddb0ec3dc374c7 @ ./sys/fs/cgroup/freezer flags 0x20000f options freezer
(00.002361)     type cgroup source cgroup mnt_id 96 s_dev 0x1e /docker/70b2975a004c8028d8bd79f098d51bdd04f56d24b4c58bf1c5ddb0ec3dc374c7 @ ./sys/fs/cgroup/net_cls flags 0x20000f options net_cls
(00.002366)     type cgroup source cgroup mnt_id 97 s_dev 0x1f /docker/70b2975a004c8028d8bd79f098d51bdd04f56d24b4c58bf1c5ddb0ec3dc374c7 @ ./sys/fs/cgroup/blkio flags 0x20000f options blkio
(00.002394)     type cgroup source cgroup mnt_id 98 s_dev 0x20 /docker/70b2975a004c8028d8bd79f098d51bdd04f56d24b4c58bf1c5ddb0ec3dc374c7 @ ./sys/fs/cgroup/perf_event flags 0x20000f options perf_event
(00.002401)     type cgroup source cgroup mnt_id 99 s_dev 0x21 /docker/70b2975a004c8028d8bd79f098d51bdd04f56d24b4c58bf1c5ddb0ec3dc374c7 @ ./sys/fs/cgroup/net_prio flags 0x20000f options net_prio
(00.002409)     type cgroup source cgroup mnt_id 100 s_dev 0x22 /docker/70b2975a004c8028d8bd79f098d51bdd04f56d24b4c58bf1c5ddb0ec3dc374c7 @ ./sys/fs/cgroup/hugetlb flags 0x20000f options hugetlb
(00.002415)     type cgroup source systemd mnt_id 101 s_dev 0x24 / @ ./sys/fs/cgroup/systemd flags 0x20000f options name=systemd
(00.002421)     type ext4 source /dev/disk/by-uuid/b2c933d5-6fa6-4035-980a-ac7ae5fe64d2 mnt_id 102 s_dev 0x800006 /var/lib/docker/vfs/dir/574a5b44693a5c7cfae8450bf255369fd0515e01cb012cf318cb561992a35cc5 @ ./var/lib/mysql flags 0x200000 options errors=remount-ro,data=ordered
(00.002428)     type ext4 source /dev/disk/by-uuid/b2c933d5-6fa6-4035-980a-ac7ae5fe64d2 mnt_id 103 s_dev 0x800006 /var/lib/docker/containers/70b2975a004c8028d8bd79f098d51bdd04f56d24b4c58bf1c5ddb0ec3dc374c7/resolv.conf @ ./etc/resolv.conf flags 0x200000 options errors=remount-ro,data=ordered
(00.002452)     type ext4 source /dev/disk/by-uuid/b2c933d5-6fa6-4035-980a-ac7ae5fe64d2 mnt_id 104 s_dev 0x800006 /var/lib/docker/containers/70b2975a004c8028d8bd79f098d51bdd04f56d24b4c58bf1c5ddb0ec3dc374c7/hostname @ ./etc/hostname flags 0x200000 options errors=remount-ro,data=ordered
(00.002462)     type ext4 source /dev/disk/by-uuid/b2c933d5-6fa6-4035-980a-ac7ae5fe64d2 mnt_id 105 s_dev 0x800006 /var/lib/docker/containers/70b2975a004c8028d8bd79f098d51bdd04f56d24b4c58bf1c5ddb0ec3dc374c7/hosts @ ./etc/hosts flags 0x200000 options errors=remount-ro,data=ordered
(00.002468)     type proc source proc mnt_id 50 s_dev 0x29 /asound @ ./proc/asound flags 0x20000f options 
(00.002474)     type proc source proc mnt_id 51 s_dev 0x29 /bus @ ./proc/bus flags 0x20000f options 
(00.002479)     type proc source proc mnt_id 52 s_dev 0x29 /fs @ ./proc/fs flags 0x20000f options 
(00.002484)     type proc source proc mnt_id 53 s_dev 0x29 /irq @ ./proc/irq flags 0x20000f options 
(00.002489)     type proc source proc mnt_id 54 s_dev 0x29 /sys @ ./proc/sys flags 0x20000f options 
(00.002494)     type proc source proc mnt_id 55 s_dev 0x29 /sysrq-trigger @ ./proc/sysrq-trigger flags 0x20000f options 
(00.002500)     type tmpfs source tmpfs mnt_id 56 s_dev 0x2a /null @ ./proc/kcore flags 0x1000000 options mode=755
(00.002504)     type tmpfs source tmpfs mnt_id 57 s_dev 0x2a /null @ ./proc/timer_stats flags 0x1000000 options mode=755
(00.002512) Building mountpoints tree
(00.002514)     Building plain mount tree
(00.002515)         Working on 57->83
(00.002517)         Working on 56->83
(00.002518)         Working on 55->83
(00.002519)         Working on 54->83
(00.002520)         Working on 53->83
(00.002521)         Working on 52->83
(00.002522)         Working on 51->83
(00.002523)         Working on 50->83
(00.002524)         Working on 105->82
(00.002525)         Working on 104->82
(00.002526)         Working on 103->82
(00.002527)         Working on 102->82
(00.002529)         Working on 101->89
(00.002530)         Working on 100->89
(00.002531)         Working on 99->89
(00.002532)         Working on 98->89
(00.002533)         Working on 97->89
(00.002534)         Working on 96->89
(00.002535)         Working on 95->89
(00.002536)         Working on 94->89
(00.002537)         Working on 93->89
(00.002538)         Working on 92->89
(00.002539)         Working on 91->89
(00.002540)         Working on 90->89
(00.002541)         Working on 89->88
(00.002542)         Working on 88->82
(00.002543)         Working on 87->84
(00.002544)         Working on 86->84
(00.002545)         Working on 85->84
(00.002546)         Working on 84->82
(00.002547)         Working on 83->82
(00.002548)         Working on 82->49
(00.002550)     Resorting siblings on 82
(00.002551)     Resorting siblings on 105
(00.002552)     Resorting siblings on 104
(00.002553)     Resorting siblings on 103
(00.002554)     Resorting siblings on 102
(00.002555)     Resorting siblings on 88
(00.002556)     Resorting siblings on 89
(00.002557)     Resorting siblings on 101
(00.002558)     Resorting siblings on 100
(00.002559)     Resorting siblings on 99
(00.002560)     Resorting siblings on 98
(00.002561)     Resorting siblings on 97
(00.002563)     Resorting siblings on 96
(00.002564)     Resorting siblings on 95
(00.002565)     Resorting siblings on 94
(00.002566)     Resorting siblings on 93
(00.002567)     Resorting siblings on 92
(00.002568)     Resorting siblings on 91
(00.002569)     Resorting siblings on 90
(00.002570)     Resorting siblings on 84
(00.002571)     Resorting siblings on 87
(00.002572)     Resorting siblings on 86
(00.002573)     Resorting siblings on 85
(00.002574)     Resorting siblings on 83
(00.002575)     Resorting siblings on 57
(00.002576)     Resorting siblings on 56
(00.002577)     Resorting siblings on 55
(00.002578)     Resorting siblings on 54
(00.002579)     Resorting siblings on 53
(00.002580)     Resorting siblings on 52
(00.002581)     Resorting siblings on 51
(00.002582)     Resorting siblings on 50
(00.002583) Done:
(00.002584) [./](82->49)
(00.002586)  [./sys](88->82)
(00.002587)   [./sys/fs/cgroup](89->88)
(00.002588)    [./sys/fs/cgroup/systemd](101->89)
(00.002589)    <--
(00.002591)    [./sys/fs/cgroup/cpuset](90->89)
(00.002592)    <--
(00.002593)    [./sys/fs/cgroup/cpu](91->89)
(00.002594)    <--
(00.002595)    [./sys/fs/cgroup/cpuacct](92->89)
(00.002597)    <--
(00.002597)    [./sys/fs/cgroup/memory](93->89)
(00.002599)    <--
(00.002600)    [./sys/fs/cgroup/devices](94->89)
(00.002601)    <--
(00.002602)    [./sys/fs/cgroup/freezer](95->89)
(00.002605)    <--
(00.002606)    [./sys/fs/cgroup/net_cls](96->89)
(00.002608)    <--
(00.002609)    [./sys/fs/cgroup/blkio](97->89)
(00.002610)    <--
(00.002611)    [./sys/fs/cgroup/perf_event](98->89)
(00.002612)    <--
(00.002613)    [./sys/fs/cgroup/net_prio](99->89)
(00.002614)    <--
(00.002615)    [./sys/fs/cgroup/hugetlb](100->89)
(00.002617)    <--
(00.002618)   <--
(00.002619)  <--
(00.002620)  [./proc](83->82)
(00.002621)   [./proc/timer_stats](57->83)
(00.002622)   <--
(00.002623)   [./proc/asound](50->83)
(00.002624)   <--
(00.002625)   [./proc/bus](51->83)
(00.002627)   <--
(00.002627)   [./proc/fs](52->83)
(00.002629)   <--
(00.002630)   [./proc/irq](53->83)
(00.002631)   <--
(00.002632)   [./proc/sys](54->83)
(00.002633)   <--
(00.002634)   [./proc/sysrq-trigger](55->83)
(00.002635)   <--
(00.002636)   [./proc/kcore](56->83)
(00.002637)   <--
(00.002639)  <--
(00.002640)  [./dev](84->82)
(00.002641)   [./dev/mqueue](87->84)
(00.002642)   <--
(00.002643)   [./dev/pts](85->84)
(00.002644)   <--
(00.002645)   [./dev/shm](86->84)
(00.002646)   <--
(00.002647)  <--
(00.002648)  [./etc/hosts](105->82)
(00.002650)  <--
(00.002651)  [./var/lib/mysql](102->82)
(00.002652)  <--
(00.002653)  [./etc/resolv.conf](103->82)
(00.002654)  <--
(00.002655)  [./etc/hostname](104->82)
(00.002656)  <--
(00.002657) <--
(00.002660) Found /etc/hosts mapping for ./etc/hosts mountpoint
(00.002661) Found /etc/hostname mapping for ./etc/hostname mountpoint
(00.002662) Found /etc/resolv.conf mapping for ./etc/resolv.conf mountpoint
(00.002664) Found /var/lib/mysql mapping for ./var/lib/mysql mountpoint
(00.002665) Found /sys/fs/cgroup/systemd mapping for ./sys/fs/cgroup/systemd mountpoint
(00.002667) Found /sys/fs/cgroup/hugetlb mapping for ./sys/fs/cgroup/hugetlb mountpoint
(00.002668) Found /sys/fs/cgroup/net_prio mapping for ./sys/fs/cgroup/net_prio mountpoint
(00.002669) Found /sys/fs/cgroup/perf_event mapping for ./sys/fs/cgroup/perf_event mountpoint
(00.002671) Found /sys/fs/cgroup/blkio mapping for ./sys/fs/cgroup/blkio mountpoint
(00.002672) Found /sys/fs/cgroup/net_cls mapping for ./sys/fs/cgroup/net_cls mountpoint
(00.002673) Found /sys/fs/cgroup/freezer mapping for ./sys/fs/cgroup/freezer mountpoint
(00.002674) Found /sys/fs/cgroup/devices mapping for ./sys/fs/cgroup/devices mountpoint
(00.002675) Found /sys/fs/cgroup/memory mapping for ./sys/fs/cgroup/memory mountpoint
(00.002677) Found /sys/fs/cgroup/cpuacct mapping for ./sys/fs/cgroup/cpuacct mountpoint
(00.002678) Found /sys/fs/cgroup/cpu mapping for ./sys/fs/cgroup/cpu mountpoint
(00.002679) Found /sys/fs/cgroup/cpuset mapping for ./sys/fs/cgroup/cpuset mountpoint
(00.002690) Collecting netns 8/9363
(00.002692) Switching to 9363's net for collecting sockets
(00.002879) sk unix: unix: Dropping path /var/run/mysqld/mysqld.sock for unlinked sk 0xbab8
(00.002886) sk unix:    Collected: ino 0xbab8 peer_ino 0 family    1 type    1 state 10 name (null)
(00.002889) sk unix:    Collected: ino 0xcb1f peer_ino 0 family    1 type    5 state  7 name (null)
(00.003043)     Collected: ino 0x    bab7 family    2 type    1 port     3306 state 10 src_addr 0.0.0.0
(00.003200) Collect netlink sock 0xbcf7
(00.003202) Collect netlink sock 0xbd04
(00.003203) Collect netlink sock 0xcb1e
(00.003204) Collect netlink sock 0xbd03
(00.003206) Collect netlink sock 0xbcfa
(00.003207) Collect netlink sock 0xbcf8
(00.003208) Collect netlink sock 0xbcf9
(00.003219) ========================================
(00.003221) Dumping task (pid: 9363)
(00.003222) ========================================
(00.003223) Obtaining task stat ... (00.003247) 
(00.003249) Collecting mappings (pid: 9363)
(00.003250) ----------------------------------------
(00.003772) vma 7fec054e4000 borrows vfi from previous 7fec054d9000
(00.003778) vma 7fec056e3000 borrows vfi from previous 7fec054e4000
(00.003828) vma 7fec056e4000 borrows vfi from previous 7fec056e3000
(00.003864) vma 7fec056f0000 borrows vfi from previous 7fec056e5000
(00.003869) vma 7fec058ef000 borrows vfi from previous 7fec056f0000
(00.003874) vma 7fec058f0000 borrows vfi from previous 7fec058ef000
(00.003910) vma 7fec058fa000 borrows vfi from previous 7fec058f1000
(00.004019) vma 7fec05af9000 borrows vfi from previous 7fec058fa000
(00.004024) vma 7fec05afa000 borrows vfi from previous 7fec05af9000
(00.004068) vma 7fec06b13000 borrows vfi from previous 7fec06afd000
(00.004179) vma 7fec06d12000 borrows vfi from previous 7fec06b13000
(00.004212) vma 7fec06d2a000 borrows vfi from previous 7fec06d13000
(00.004217) vma 7fec06f29000 borrows vfi from previous 7fec06d2a000
(00.004222) vma 7fec06f2a000 borrows vfi from previous 7fec06f29000
(00.004302) vma 7fec070e8000 borrows vfi from previous 7fec06f2d000
(00.004308) vma 7fec072e7000 borrows vfi from previous 7fec070e8000
(00.004312) vma 7fec072eb000 borrows vfi from previous 7fec072e7000
(00.004353) vma 7fec073f7000 borrows vfi from previous 7fec072f2000
(00.004358) vma 7fec075f6000 borrows vfi from previous 7fec073f7000
(00.004420) vma 7fec075f7000 borrows vfi from previous 7fec075f6000
(00.004452) vma 7fec076de000 borrows vfi from previous 7fec075f8000
(00.004457) vma 7fec078dd000 borrows vfi from previous 7fec076de000
(00.004461) vma 7fec078e5000 borrows vfi from previous 7fec078dd000
(00.004532) vma 7fec07915000 borrows vfi from previous 7fec078fc000
(00.004537) vma 7fec07b14000 borrows vfi from previous 7fec07915000
(00.004542) vma 7fec07b15000 borrows vfi from previous 7fec07b14000
(00.004578) vma 7fec07b1d000 borrows vfi from previous 7fec07b1a000
(00.004583) vma 7fec07d1c000 borrows vfi from previous 7fec07b1d000
(00.004626) vma 7fec07d1d000 borrows vfi from previous 7fec07d1c000
(00.004666) vma 7fec07d27000 borrows vfi from previous 7fec07d1e000
(00.004671) vma 7fec07f27000 borrows vfi from previous 7fec07d27000
(00.004675) vma 7fec07f28000 borrows vfi from previous 7fec07f27000
(00.004756) vma 7fec07f5f000 borrows vfi from previous 7fec07f57000
(00.004761) vma 7fec0815e000 borrows vfi from previous 7fec07f5f000
(00.004766) vma 7fec0815f000 borrows vfi from previous 7fec0815e000
(00.004805) vma 7fec08179000 borrows vfi from previous 7fec08161000
(00.004811) vma 7fec08378000 borrows vfi from previous 7fec08179000
(00.004895) vma 7fec08379000 borrows vfi from previous 7fec08378000
(00.004928) vma 7fec0837b000 borrows vfi from previous 7fec0837a000
(00.004934) vma 7fec0857a000 borrows vfi from previous 7fec0837b000
(00.004938) vma 7fec0857b000 borrows vfi from previous 7fec0857a000
(00.005355) vma 7fec0879f000 borrows vfi from previous 7fec0879e000
(00.005875) vma 7fec09433000 borrows vfi from previous 7fec093c4000
(00.005956) Collected, longest area occupies 41524 pages
(00.005959) 0x7febef79c000-0x7febef79d000 (4K) prot 0 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005962) 0x7febef79d000-0x7febeff9d000 (8192K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005964) 0x7febeff9d000-0x7febeff9e000 (4K) prot 0 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005966) 0x7febeff9e000-0x7febf079e000 (8192K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005968) 0x7febf079e000-0x7febf079f000 (4K) prot 0 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005970) 0x7febf079f000-0x7febf0f9f000 (8192K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005971) 0x7febf0f9f000-0x7febf0fa0000 (4K) prot 0 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005973) 0x7febf0fa0000-0x7febf19a5000 (10260K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005975) 0x7febf1dc3000-0x7febf1dc4000 (4K) prot 0 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005977) 0x7febf1dc4000-0x7febf25c4000 (8192K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005979) 0x7febf25c4000-0x7febf25c5000 (4K) prot 0 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005981) 0x7febf25c5000-0x7febf2dc5000 (8192K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005983) 0x7febf2dc5000-0x7febf2dc6000 (4K) prot 0 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005984) 0x7febf2dc6000-0x7febf35c6000 (8192K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005986) 0x7febf35c6000-0x7febf35c7000 (4K) prot 0 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005992) 0x7febf35c7000-0x7febf3dc7000 (8192K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005994) 0x7febf3dc7000-0x7febf3dc8000 (4K) prot 0 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005996) 0x7febf3dc8000-0x7febf45c8000 (8192K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005998) 0x7febf45c8000-0x7febf45c9000 (4K) prot 0 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.005999) 0x7febf45c9000-0x7febf4dc9000 (8192K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006001) 0x7febf4dc9000-0x7febf4dca000 (4K) prot 0 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006003) 0x7febf4dca000-0x7febf55ca000 (8192K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006005) 0x7febf55ca000-0x7febf55cb000 (4K) prot 0 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006007) 0x7febf55cb000-0x7febf5dcb000 (8192K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006009) 0x7febf5dcb000-0x7febf5dcc000 (4K) prot 0 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006010) 0x7febf5dcc000-0x7fec00000000 (166096K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006012) 0x7fec00000000-0x7fec00021000 (132K) prot 0x3 flags 0x4022 st 0x201 off 0 reg ap  shmid: 0
(00.006014) 0x7fec00021000-0x7fec04000000 (65404K) prot 0 flags 0x4022 st 0x201 off 0 reg ap  shmid: 0
(00.006016) 0x7fec041f2000-0x7fec054d9000 (19356K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006018) 0x7fec054d9000-0x7fec054e4000 (44K) prot 0x5 flags 0x2 st 0x41 off 0 reg fp  shmid: 0
(00.006020) 0x7fec054e4000-0x7fec056e3000 (2044K) prot 0 flags 0x2 st 0x41 off 0xb000 reg fp  shmid: 0
(00.006022) 0x7fec056e3000-0x7fec056e4000 (4K) prot 0x1 flags 0x2 st 0x41 off 0xa000 reg fp  shmid: 0
(00.006024) 0x7fec056e4000-0x7fec056e5000 (4K) prot 0x3 flags 0x2 st 0x41 off 0xb000 reg fp  shmid: 0
(00.006025) 0x7fec056e5000-0x7fec056f0000 (44K) prot 0x5 flags 0x2 st 0x41 off 0 reg fp  shmid: 0
(00.006027) 0x7fec056f0000-0x7fec058ef000 (2044K) prot 0 flags 0x2 st 0x41 off 0xb000 reg fp  shmid: 0
(00.006029) 0x7fec058ef000-0x7fec058f0000 (4K) prot 0x1 flags 0x2 st 0x41 off 0xa000 reg fp  shmid: 0
(00.006031) 0x7fec058f0000-0x7fec058f1000 (4K) prot 0x3 flags 0x2 st 0x41 off 0xb000 reg fp  shmid: 0
(00.006033) 0x7fec058f1000-0x7fec058fa000 (36K) prot 0x5 flags 0x2 st 0x41 off 0 reg fp  shmid: 0
(00.006035) 0x7fec058fa000-0x7fec05af9000 (2044K) prot 0 flags 0x2 st 0x41 off 0x9000 reg fp  shmid: 0
(00.006037) 0x7fec05af9000-0x7fec05afa000 (4K) prot 0x1 flags 0x2 st 0x41 off 0x8000 reg fp  shmid: 0
(00.006038) 0x7fec05afa000-0x7fec05afb000 (4K) prot 0x3 flags 0x2 st 0x41 off 0x9000 reg fp  shmid: 0
(00.006040) 0x7fec05afb000-0x7fec062fc000 (8196K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006042) 0x7fec062fc000-0x7fec062fd000 (4K) prot 0 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006044) 0x7fec062fd000-0x7fec06afd000 (8192K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006046) 0x7fec06afd000-0x7fec06b13000 (88K) prot 0x5 flags 0x2 st 0x41 off 0 reg fp  shmid: 0
(00.006047) 0x7fec06b13000-0x7fec06d12000 (2044K) prot 0 flags 0x2 st 0x41 off 0x16000 reg fp  shmid: 0
(00.006049) 0x7fec06d12000-0x7fec06d13000 (4K) prot 0x3 flags 0x2 st 0x41 off 0x15000 reg fp  shmid: 0
(00.006051) 0x7fec06d13000-0x7fec06d2a000 (92K) prot 0x5 flags 0x2 st 0x41 off 0 reg fp  shmid: 0
(00.006053) 0x7fec06d2a000-0x7fec06f29000 (2044K) prot 0 flags 0x2 st 0x41 off 0x17000 reg fp  shmid: 0
(00.006055) 0x7fec06f29000-0x7fec06f2a000 (4K) prot 0x1 flags 0x2 st 0x41 off 0x16000 reg fp  shmid: 0
(00.006057) 0x7fec06f2a000-0x7fec06f2b000 (4K) prot 0x3 flags 0x2 st 0x41 off 0x17000 reg fp  shmid: 0
(00.006058) 0x7fec06f2b000-0x7fec06f2d000 (8K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006060) 0x7fec06f2d000-0x7fec070e8000 (1772K) prot 0x5 flags 0x2 st 0x41 off 0 reg fp  shmid: 0
(00.006062) 0x7fec070e8000-0x7fec072e7000 (2044K) prot 0 flags 0x2 st 0x41 off 0x1bb000 reg fp  shmid: 0
(00.006064) 0x7fec072e7000-0x7fec072eb000 (16K) prot 0x1 flags 0x2 st 0x41 off 0x1ba000 reg fp  shmid: 0
(00.006066) 0x7fec072eb000-0x7fec072ed000 (8K) prot 0x3 flags 0x2 st 0x41 off 0x1be000 reg fp  shmid: 0
(00.006072) 0x7fec072ed000-0x7fec072f2000 (20K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006074) 0x7fec072f2000-0x7fec073f7000 (1044K) prot 0x5 flags 0x2 st 0x41 off 0 reg fp  shmid: 0
(00.006076) 0x7fec073f7000-0x7fec075f6000 (2044K) prot 0 flags 0x2 st 0x41 off 0x105000 reg fp  shmid: 0
(00.006077) 0x7fec075f6000-0x7fec075f7000 (4K) prot 0x1 flags 0x2 st 0x41 off 0x104000 reg fp  shmid: 0
(00.006079) 0x7fec075f7000-0x7fec075f8000 (4K) prot 0x3 flags 0x2 st 0x41 off 0x105000 reg fp  shmid: 0
(00.006081) 0x7fec075f8000-0x7fec076de000 (920K) prot 0x5 flags 0x2 st 0x41 off 0 reg fp  shmid: 0
(00.006083) 0x7fec076de000-0x7fec078dd000 (2044K) prot 0 flags 0x2 st 0x41 off 0xe6000 reg fp  shmid: 0
(00.006085) 0x7fec078dd000-0x7fec078e5000 (32K) prot 0x1 flags 0x2 st 0x41 off 0xe5000 reg fp  shmid: 0
(00.006087) 0x7fec078e5000-0x7fec078e7000 (8K) prot 0x3 flags 0x2 st 0x41 off 0xed000 reg fp  shmid: 0
(00.006088) 0x7fec078e7000-0x7fec078fc000 (84K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006090) 0x7fec078fc000-0x7fec07915000 (100K) prot 0x5 flags 0x2 st 0x41 off 0 reg fp  shmid: 0
(00.006092) 0x7fec07915000-0x7fec07b14000 (2044K) prot 0 flags 0x2 st 0x41 off 0x19000 reg fp  shmid: 0
(00.006094) 0x7fec07b14000-0x7fec07b15000 (4K) prot 0x1 flags 0x2 st 0x41 off 0x18000 reg fp  shmid: 0
(00.006096) 0x7fec07b15000-0x7fec07b16000 (4K) prot 0x3 flags 0x2 st 0x41 off 0x19000 reg fp  shmid: 0
(00.006097) 0x7fec07b16000-0x7fec07b1a000 (16K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006099) 0x7fec07b1a000-0x7fec07b1d000 (12K) prot 0x5 flags 0x2 st 0x41 off 0 reg fp  shmid: 0
(00.006101) 0x7fec07b1d000-0x7fec07d1c000 (2044K) prot 0 flags 0x2 st 0x41 off 0x3000 reg fp  shmid: 0
(00.006103) 0x7fec07d1c000-0x7fec07d1d000 (4K) prot 0x1 flags 0x2 st 0x41 off 0x2000 reg fp  shmid: 0
(00.006105) 0x7fec07d1d000-0x7fec07d1e000 (4K) prot 0x3 flags 0x2 st 0x41 off 0x3000 reg fp  shmid: 0
(00.006107) 0x7fec07d1e000-0x7fec07d27000 (36K) prot 0x5 flags 0x2 st 0x41 off 0 reg fp  shmid: 0
(00.006108) 0x7fec07d27000-0x7fec07f27000 (2048K) prot 0 flags 0x2 st 0x41 off 0x9000 reg fp  shmid: 0
(00.006110) 0x7fec07f27000-0x7fec07f28000 (4K) prot 0x1 flags 0x2 st 0x41 off 0x9000 reg fp  shmid: 0
(00.006112) 0x7fec07f28000-0x7fec07f29000 (4K) prot 0x3 flags 0x2 st 0x41 off 0xa000 reg fp  shmid: 0
(00.006114) 0x7fec07f29000-0x7fec07f57000 (184K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006116) 0x7fec07f57000-0x7fec07f5f000 (32K) prot 0x5 flags 0x2 st 0x41 off 0 reg fp  shmid: 0
(00.006118) 0x7fec07f5f000-0x7fec0815e000 (2044K) prot 0 flags 0x2 st 0x41 off 0x8000 reg fp  shmid: 0
(00.006120) 0x7fec0815e000-0x7fec0815f000 (4K) prot 0x1 flags 0x2 st 0x41 off 0x7000 reg fp  shmid: 0
(00.006121) 0x7fec0815f000-0x7fec08160000 (4K) prot 0x3 flags 0x2 st 0x41 off 0x8000 reg fp  shmid: 0
(00.006123) 0x7fec08160000-0x7fec08161000 (4K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006125) 0x7fec08161000-0x7fec08179000 (96K) prot 0x5 flags 0x2 st 0x41 off 0 reg fp  shmid: 0
(00.006127) 0x7fec08179000-0x7fec08378000 (2044K) prot 0 flags 0x2 st 0x41 off 0x18000 reg fp  shmid: 0
(00.006129) 0x7fec08378000-0x7fec08379000 (4K) prot 0x1 flags 0x2 st 0x41 off 0x17000 reg fp  shmid: 0
(00.006131) 0x7fec08379000-0x7fec0837a000 (4K) prot 0x3 flags 0x2 st 0x41 off 0x18000 reg fp  shmid: 0
(00.006132) 0x7fec0837a000-0x7fec0837b000 (4K) prot 0x5 flags 0x2 st 0x41 off 0 reg fp  shmid: 0
(00.006134) 0x7fec0837b000-0x7fec0857a000 (2044K) prot 0 flags 0x2 st 0x41 off 0x1000 reg fp  shmid: 0
(00.006136) 0x7fec0857a000-0x7fec0857b000 (4K) prot 0x1 flags 0x2 st 0x41 off 0 reg fp  shmid: 0
(00.006138) 0x7fec0857b000-0x7fec0857c000 (4K) prot 0x3 flags 0x2 st 0x41 off 0x1000 reg fp  shmid: 0
(00.006140) 0x7fec0857c000-0x7fec0859f000 (140K) prot 0x5 flags 0x2 st 0x41 off 0 reg fp  shmid: 0
(00.006141) 0x7fec085e8000-0x7fec085e9000 (4K) prot 0 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006143) 0x7fec085e9000-0x7fec08739000 (1344K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006147) 0x7fec08739000-0x7fec0873b000 (8K) prot 0x3 flags 0x1 st 0x2000 off 0  shmid: 0
(00.006149) 0x7fec0873b000-0x7fec08740000 (20K) prot 0x3 flags 0x1 st 0x2000 off 0  shmid: 0
(00.006151) 0x7fec08740000-0x7fec0879e000 (376K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006153) 0x7fec0879e000-0x7fec0879f000 (4K) prot 0x1 flags 0x2 st 0x41 off 0x22000 reg fp  shmid: 0
(00.006155) 0x7fec0879f000-0x7fec087a0000 (4K) prot 0x3 flags 0x2 st 0x41 off 0x23000 reg fp  shmid: 0
(00.006157) 0x7fec087a0000-0x7fec087a1000 (4K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006159) 0x7fec087a1000-0x7fec091c5000 (10384K) prot 0x5 flags 0x2 st 0x41 off 0 reg fp  shmid: 0
(00.006160) 0x7fec091c5000-0x7fec091ca000 (20K) prot 0x3 flags 0x1 st 0x2000 off 0  shmid: 0
(00.006162) 0x7fec091ca000-0x7fec091cf000 (20K) prot 0x3 flags 0x1 st 0x2000 off 0  shmid: 0
(00.006164) 0x7fec091cf000-0x7fec091d4000 (20K) prot 0x3 flags 0x1 st 0x2000 off 0  shmid: 0
(00.006166) 0x7fec091d4000-0x7fec091d9000 (20K) prot 0x3 flags 0x1 st 0x2000 off 0  shmid: 0
(00.006168) 0x7fec091d9000-0x7fec091de000 (20K) prot 0x3 flags 0x1 st 0x2000 off 0  shmid: 0
(00.006169) 0x7fec091de000-0x7fec091e3000 (20K) prot 0x3 flags 0x1 st 0x2000 off 0  shmid: 0
(00.006171) 0x7fec091e3000-0x7fec091e8000 (20K) prot 0x3 flags 0x1 st 0x2000 off 0  shmid: 0
(00.006173) 0x7fec091e8000-0x7fec091ed000 (20K) prot 0x3 flags 0x1 st 0x2000 off 0  shmid: 0
(00.006175) 0x7fec091ed000-0x7fec091f2000 (20K) prot 0x3 flags 0x1 st 0x2000 off 0  shmid: 0
(00.006176) 0x7fec091f2000-0x7fec091f3000 (4K) prot 0x3 flags 0x1 st 0x2000 off 0  shmid: 0
(00.006178) 0x7fec091f3000-0x7fec093c0000 (1844K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006180) 0x7fec093c0000-0x7fec093c4000 (16K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006182) 0x7fec093c4000-0x7fec09433000 (444K) prot 0x1 flags 0x2 st 0x41 off 0xa23000 reg fp  shmid: 0
(00.006184) 0x7fec09433000-0x7fec094d8000 (660K) prot 0x3 flags 0x2 st 0x41 off 0xa92000 reg fp  shmid: 0
(00.006186) 0x7fec094d8000-0x7fec09500000 (160K) prot 0x3 flags 0x22 st 0x201 off 0 reg ap  shmid: 0
(00.006188) 0x7fec0951a000-0x7fec09ece000 (9936K) prot 0x3 flags 0x22 st 0x221 off 0 reg heap ap  shmid: 0
(00.006190) 0x7fff1b54a000-0x7fff1b56c000 (136K) prot 0x3 flags 0x122 st 0x201 off 0 reg ap  shmid: 0
(00.006191) 0x7fff1b5e8000-0x7fff1b5ea000 (8K) prot 0x1 flags 0x22 st 0x1201 off 0 reg vvar ap  shmid: 0
(00.006193) 0x7fff1b5ea000-0x7fff1b5ec000 (8K) prot 0x5 flags 0x22 st 0x209 off 0 reg vdso ap  shmid: 0
(00.006195) 0xffffffffff600000-0xffffffffff601000 (4K) prot 0x5 flags 0x22 st 0x204 off 0 vsys ap  shmid: 0
(00.006197) ----------------------------------------
(00.006200) 
(00.006202) Collecting fds (pid: 9363)
(00.006203) ----------------------------------------
(00.006237) Found 31 file descriptors
(00.006239) ----------------------------------------
(00.006251) Dump private signals of 9363
(00.006257) Dump private signals of 9427
(00.006262) Dump private signals of 9428
(00.006265) Dump private signals of 9429
(00.006268) Dump private signals of 9430
(00.006273) Dump private signals of 9431
(00.006277) Dump private signals of 9432
(00.006281) Dump private signals of 9433
(00.006285) Dump private signals of 9434
(00.006289) Dump private signals of 9435
(00.006292) Dump private signals of 9436
(00.006296) Dump private signals of 9443
(00.006301) Dump private signals of 9444
(00.006305) Dump private signals of 9445
(00.006307) Dump private signals of 9446
(00.006311) Dump private signals of 9447
(00.006314) Dump shared signals of 9363
(00.006325) Parasite syscall_ip at 0x7fec054d9000
(00.006385) Putting parasite blob into 0x7fb5fb376000->0x7fec04042000
(00.006414) Dumping GP/FPU registers for 9363
(00.006416) Warn  (arch/x86/crtools.c:132): Will restore 9363 with interrupted system call
(00.006425) xsave runtime structure
(00.006426) -----------------------
(00.006427) cwd:37f swd:0 twd:0 fop:0 mxcsr:1fa0 mxcsr_mask:ffff
(00.006429) magic1:46505853 extended_size:344 xstate_bv:7 xstate_size:340
(00.006431) xstate_bv: 7
(00.006435) -----------------------
(00.006437) Putting tsock into pid 9363
(00.006507) Wait for parasite being daemonized...
(00.006509) Wait for ack 2 on daemon socket
pie: Running daemon thread leader
pie: __sent ack msg: 2 2 0
pie: Daemon waits for command
(00.006524) Fetched ack: 2 2 0
(00.006526) Parasite 9363 has been switched to daemon mode
(00.006531) Sent msg to daemon 15 0 0
pie: __fetched msg: 15 0 0
pie: __sent ack msg: 15 15 0
pie: Daemon waits for command
(00.006552) Wait for ack 15 on daemon socket
(00.006556) Fetched ack: 15 15 0
(00.006566) Sent msg to daemon 17 0 0
pie: __fetched msg: 17 0 0
(00.006568) Wait for ack 17 on daemon socket
pie: __sent ack msg: 17 17 0
pie: Daemon waits for command
(00.006574) Fetched ack: 17 17 0
(00.006581) Checking AIO rings
(00.006583)  `- Ring #0 @7fec08739000
(00.006584)  `- Ring #1 @7fec0873b000
(00.006585)  `- Ring #2 @7fec091c5000
(00.006586)  `- Ring #3 @7fec091ca000
(00.006587)  `- Ring #4 @7fec091cf000
(00.006588)  `- Ring #5 @7fec091d4000
(00.006589)  `- Ring #6 @7fec091d9000
(00.006591)  `- Ring #7 @7fec091de000
(00.006592)  `- Ring #8 @7fec091e3000
(00.006593)  `- Ring #9 @7fec091e8000
(00.006594)  `- Ring #10 @7fec091ed000
(00.006595)  `- Ring #11 @7fec091f2000
(00.006599) Sent msg to daemon 18 0 0
(00.006600) Wait for ack 18 on daemon socket
pie: __fetched msg: 18 0 0
pie: __sent ack msg: 18 18 0
pie: Daemon waits for command
(00.006607) Fetched ack: 18 18 0
(00.006609)  `- Ring #0 has 255 reqs, estimated to 126
(00.006611)  `- Ring #1 has 639 reqs, estimated to 318
(00.006612)  `- Ring #2 has 639 reqs, estimated to 318
(00.006613)  `- Ring #3 has 639 reqs, estimated to 318
(00.006614)  `- Ring #4 has 639 reqs, estimated to 318
(00.006615)  `- Ring #5 has 639 reqs, estimated to 318
(00.006616)  `- Ring #6 has 639 reqs, estimated to 318
(00.006618)  `- Ring #7 has 639 reqs, estimated to 318
(00.006619)  `- Ring #8 has 639 reqs, estimated to 318
(00.006620)  `- Ring #9 has 639 reqs, estimated to 318
(00.006621)  `- Ring #10 has 639 reqs, estimated to 318
(00.006622)  `- Ring #11 has 127 reqs, estimated to 62
(00.006626) Sent msg to daemon 12 0 0
(00.006627) Wait for ack 12 on daemon socket
pie: __fetched msg: 12 0 0
pie: __sent ack msg: 12 12 0
pie: Daemon waits for command
(00.006636) Fetched ack: 12 12 0
(00.006638) sid=1 pgid=1 pid=1
(00.006656) 
(00.006657) Dumping opened files (pid: 9363)
(00.006659) ----------------------------------------
(00.006663) Sent msg to daemon 14 0 0
pie: __fetched msg: 14 0 0
pie: __sent ack msg: 14 14 0
pie: Daemon waits for command
(00.006698) Wait for ack 14 on daemon socket
(00.006701) Fetched ack: 14 14 0
(00.006717) 9363 fdinfo 0: pos: 0x               0 flags:           100000/0
(00.006737) Dumping path for 0 fd via self 28 [/dev/null]
(00.006753) fdinfo: type: 0x 1 flags: 0100000/0 pos: 0x       0 fd: 0
(00.006776) 9363 fdinfo 1: pos: 0x               0 flags:           100002/0
(00.006779) Error (file-lock.c:225): Some file locks are hold by dumping tasks!You can try --file-locks to dump them.
(00.006786) ----------------------------------------
(00.006788) Error (cr-dump.c:1633): Dump files (pid: 9363) failed with -1
(00.006804) Waiting for 9363 to trap
(00.006807) Daemon 9363 exited trapping
(00.006811) Sent msg to daemon 6 0 0
pie: __fetched msg: 6 0 0
pie: 1: new_sp=0x7fec041e9008 ip 0x7fec0701a12d
(00.006841) 9363 was trapped
(00.006843) `- Expecting exit
(00.006852) 9363 was trapped
(00.006854) 9363 is going to execute the syscall f
(00.006863) 9363 was stopped
(00.006873) 9363 was trapped
(00.006876) 9363 is going to execute the syscall 1
(00.006883) 9363 was trapped
(00.006885) `- Expecting exit
(00.006892) 9363 was trapped
(00.006894) 9363 is going to execute the syscall b
(00.006908) 9363 was stopped
(00.006991) Unlock network
(00.006993) Running network-unlock scripts
(00.006999)     RPC
(00.007058) Unfreezing tasks into 1
(00.007061)     Unseizing 9363 into 1
(00.007157) Error (cr-dump.c:1996): Dumping FAILED.

I can see something about --file-locks but I cannot specify the problem.

Any ideas?

Thank you for your help..

boucher · 2015-08-05T18:01:49Z

Criu wants you to pass the file locks command line flag. In the future, we'll do this by default in Docker, but currently its not supported at all.

If you want to get something working for yourself and don't need to worry about long term maintenance of it or upstreaming it, you can just edit container_linux.go in libcontainer's vendor folder and just change this line to always pass true:

FileLocks:      proto.Bool(criuOpts.FileLocks),

klesgidisold · 2015-08-05T18:18:50Z

@boucher and @xemul you are great! Problem solved. Thank you very much for the help. Keep up the good work..!

xemul · 2015-08-05T20:06:58Z

@klesgidis you're welcome :) @boucher , thank you VERY MUCH!

klesgidisold · 2015-08-05T23:21:34Z

Sorry for re-opening.. Trying to checkpoint the same container to another pc I have this dump

(00.002239) Added /sys/fs/cgroup/cpuset:/sys/fs/cgroup/cpuset ext mount mapping
(00.002261) Added /sys/fs/cgroup/cpu:/sys/fs/cgroup/cpu ext mount mapping
(00.002263) Added /sys/fs/cgroup/cpuacct:/sys/fs/cgroup/cpuacct ext mount mapping
(00.002265) Added /sys/fs/cgroup/memory:/sys/fs/cgroup/memory ext mount mapping
(00.002266) Added /sys/fs/cgroup/devices:/sys/fs/cgroup/devices ext mount mapping
(00.002267) Added /sys/fs/cgroup/freezer:/sys/fs/cgroup/freezer ext mount mapping
(00.002269) Added /sys/fs/cgroup/blkio:/sys/fs/cgroup/blkio ext mount mapping
(00.002270) Added /sys/fs/cgroup/perf_event:/sys/fs/cgroup/perf_event ext mount mapping
(00.002271) Added /sys/fs/cgroup/hugetlb:/sys/fs/cgroup/hugetlb ext mount mapping
(00.002272) Added /sys/fs/cgroup/systemd:/sys/fs/cgroup/systemd ext mount mapping
(00.002274) Added /var/lib/mysql:/var/lib/mysql ext mount mapping
(00.002275) Added /etc/resolv.conf:/etc/resolv.conf ext mount mapping
(00.002276) Added /etc/hostname:/etc/hostname ext mount mapping
(00.002278) Added /etc/hosts:/etc/hosts ext mount mapping
(00.002280) ========================================
(00.002283) Dumping processes (pid: 14232)
(00.002289) ========================================
(00.002351) Found anon-shmem device at 4
(00.002650) Reset 14291's dirty tracking
(00.002693)  ... done
(00.002717) Dirty track supported on kernel
(00.002774) irmap: Searching irmap cache in work dir
(00.002783) No irmap-cache image
(00.002786) irmap: Searching irmap cache in parent
(00.002790) irmap: No irmap cache
(00.002798) cpu: fpu:1 fxsr:1 xsave:1
(00.002869) vdso: Parsing at 7fff0f7e3000 7fff0f7e5000
(00.002873) vdso: PT_LOAD p_vaddr: ffffffffff700000
(00.002874) vdso: DT_HASH: 0xffffffffff700120
(00.002876) vdso: DT_STRTAB: 0xffffffffff700268
(00.002877) vdso: DT_SYMTAB: 0xffffffffff700160
(00.002878) vdso: DT_STRSZ: 94
(00.002879) vdso: DT_SYMENT: 24
(00.002881) vdso: nbucket 3 nchain 11 bucket 0x7fff0f7e3128 chain 0x7fff0f7e3134
(00.002884) vdso: rt [vdso] 7fff0f7e3000-7fff0f7e5000 [vvar] ffffffffffffffff-ffffffffffffffff
(00.002933) Writing image inventory (version 1)
(00.002962) Add pid ns 1 pid 14291
(00.002967) Add net ns 2 pid 14291
(00.002971) Add ipc ns 3 pid 14291
(00.002974) Add uts ns 4 pid 14291
(00.002979) Add mnt ns 5 pid 14291
(00.002983) Add user ns 6 pid 14291
(00.002984) cg: Dumping cgroups for 14291
(00.003006) cg:  `- New css ID 1
(00.003008) cg:     `- [blkio] -> [/]
(00.003009) cg:     `- [cpu] -> [/]
(00.003010) cg:     `- [cpuacct] -> [/]
(00.003011) cg:     `- [cpuset] -> [/]
(00.003013) cg:     `- [devices] -> [/]
(00.003014) cg:     `- [freezer] -> [/]
(00.003015) cg:     `- [hugetlb] -> [/]
(00.003016) cg:     `- [memory] -> [/]
(00.003018) cg:     `- [name=systemd] -> [/]
(00.003019) cg:     `- [perf_event] -> [/]
(00.003020) cg: Set 1 is criu one
(00.003126) Seized task 14232, state 1
(00.003178)     Seizing 14232's 14256 thread
(00.003231)     Seizing 14232's 14257 thread
(00.003276)     Seizing 14232's 14258 thread
(00.003319)     Seizing 14232's 14259 thread
(00.003362)     Seizing 14232's 14260 thread
(00.003404)     Seizing 14232's 14261 thread
(00.003445)     Seizing 14232's 14262 thread
(00.003483)     Seizing 14232's 14263 thread
(00.003522)     Seizing 14232's 14264 thread
(00.003560)     Seizing 14232's 14265 thread
(00.003597)     Seizing 14232's 14266 thread
(00.003634)     Seizing 14232's 14267 thread
(00.003672)     Seizing 14232's 14269 thread
(00.003709)     Seizing 14232's 14270 thread
(00.003768)     Seizing 14232's 14271 thread
(00.003809)     Seizing 14232's 14272 thread
(00.003849)     Seizing 14232's 14273 thread
(00.003886)     Seizing 14232's 14274 thread
(00.003923)     Seizing 14232's 14275 thread
(00.003961)     Seizing 14232's 14276 thread
(00.003998)     Seizing 14232's 14277 thread
(00.004034)     Seizing 14232's 14278 thread
(00.004077)     Seizing 14232's 14279 thread
(00.004115)     Seizing 14232's 14280 thread
(00.004152)     Seizing 14232's 14282 thread
(00.004188)     Seizing 14232's 14283 thread
(00.004411) Collected 14232 in 1 state
(00.004424) Will take pid namespace in the image
(00.004426) Add pid ns 7 pid 14232
(00.004431) Will take net namespace in the image
(00.004434) Add net ns 8 pid 14232
(00.004438) Will take ipc namespace in the image
(00.004439) Add ipc ns 9 pid 14232
(00.004443) Will take uts namespace in the image
(00.004444) Add uts ns 10 pid 14232
(00.004449) Will take mnt namespace in the image
(00.004450) Add mnt ns 11 pid 14232
(00.004455) Lock network
(00.004456) Running network-lock scripts
(00.004465)     RPC
(00.004655) lockinfo: 1:1 1 14232 08:06:6835380 0 EOF
(00.004664) lockinfo: 2:1 1 14232 08:06:6835378 0 EOF
(00.004667) lockinfo: 3:1 1 14232 08:06:6835346 0 EOF
(00.004670) lockinfo: 4:1 1 14232 08:06:6835344 0 EOF
(00.004673) lockinfo: 5:1 1 14232 08:06:6835342 0 EOF
(00.004675) lockinfo: 6:1 1 14232 08:06:6835340 0 EOF
(00.004678) lockinfo: 7:1 1 14232 08:06:6835338 0 EOF
(00.004680) lockinfo: 8:1 1 14232 08:06:6835322 0 EOF
(00.004683) lockinfo: 9:1 1 14232 08:06:6835320 0 EOF
(00.004685) lockinfo: 10:1 1 14232 08:06:6834251 0 EOF
(00.004688) lockinfo: 11:1 1 14232 08:06:6835304 0 EOF
(00.004691) lockinfo: 12:1 1 14232 08:06:6835303 0 EOF
(00.004695) lockinfo: 13:1 1 14232 08:06:6835302 0 EOF
(00.004697) lockinfo: 14:1 1 2776 08:06:9306508 0 EOF
(00.004700) lockinfo: 15:1 1 2776 08:06:9306463 0 EOF
(00.004703) lockinfo: 16:1 1 2776 08:06:9306501 0 EOF
(00.004705) lockinfo: 17:1 1 2776 08:06:9306498 0 EOF
(00.004708) lockinfo: 18:1 1 14232 08:06:6835376 0 EOF
(00.004710) lockinfo: 19:1 1 14232 08:06:6835368 0 EOF
(00.004735) lockinfo: 20:1 1 14232 08:06:6835366 0 EOF
(00.004740) lockinfo: 21:1 0 3738 08:06:9315259 128 128
(00.004744) lockinfo: 22:1 0 3738 08:06:9306671 1073741826 1073742335
(00.004747) lockinfo: 23:1 0 3491 00:10:12112 4 4
(00.004750) lockinfo: 24:1 0 3491 00:10:12099 4 4
(00.004753) lockinfo: 25:1 1 3491 00:10:20637 0 0
(00.004755) lockinfo: 26:1 1 3000 08:06:9306674 0 0
(00.004758) lockinfo: 27:1 1 2995 08:06:9306575 0 0
(00.004760) lockinfo: 28:1 1 2990 08:06:9306560 0 0
(00.004763) lockinfo: 29:1 1 2982 08:06:9306532 0 0
(00.004765) lockinfo: 30:1 1 2776 08:06:9306510 0 EOF
(00.004768) lockinfo: 31:1 1 2776 08:06:9306483 0 EOF
(00.004770) lockinfo: 32:1 1 2776 08:06:9306509 0 EOF
(00.004773) lockinfo: 33:1 1 2776 08:06:9306488 0 EOF
(00.004775) lockinfo: 34:1 1 2776 08:06:9306460 0 EOF
(00.004778) lockinfo: 35:1 1 2776 08:06:9306472 0 EOF
(00.004781) lockinfo: 36:1 0 2752 08:06:9314983 128 128
(00.004783) lockinfo: 37:1 0 2752 08:06:9306321 1073741826 1073742335
(00.004786) lockinfo: 38:2 1 1977 08:06:6832002 0 EOF
(00.004808) lockinfo: 39:2 1 1977 08:06:6689034 0 EOF
(00.004812) lockinfo: 40:1 1 1516 08:06:6832560 0 EOF
(00.004817) lockinfo: 41:1 1 1516 08:06:6832559 0 EOF
(00.004822) lockinfo: 42:1 1 1516 08:06:6832558 0 EOF
(00.004824) lockinfo: 43:1 1 1336 00:10:12249 0 EOF
(00.004827) lockinfo: 44:2 1 1272 00:10:12235 0 EOF
(00.004829) lockinfo: 45:1 0 657 00:10:12119 4 4
(00.004832) lockinfo: 46:1 0 657 00:10:12116 4 4
(00.004834) lockinfo: 47:1 0 657 00:10:12115 4 4
(00.004837) lockinfo: 48:1 0 657 00:10:12112 4 4
(00.004839) lockinfo: 49:1 0 657 00:10:12111 4 4
(00.004842) lockinfo: 50:1 0 657 00:10:12110 4 4
(00.004844) lockinfo: 51:1 0 657 00:10:12109 4 4
(00.004847) lockinfo: 52:1 0 657 00:10:12108 4 4
(00.004849) lockinfo: 53:1 0 657 00:10:12107 4 4
(00.004852) lockinfo: 54:1 0 657 00:10:12106 4 4
(00.004854) lockinfo: 55:1 0 657 00:10:12105 4 4
(00.004856) lockinfo: 56:1 0 657 00:10:12099 4 4
(00.004859) lockinfo: 57:1 1 657 00:10:12102 0 0
(00.004862) lockinfo: 58:1 0 3738 08:06:9315250 128 128
(00.004864) lockinfo: 59:1 0 3738 08:06:9306646 1073741826 1073742335
(00.004881) lockinfo: 60:1 1 2998 08:06:9306588 0 0
(00.004883) lockinfo: 61:1 1 2776 08:06:9306504 0 EOF
(00.004886) lockinfo: 62:1 1 2776 08:06:9306516 0 EOF
(00.004888) lockinfo: 63:1 1 2776 08:06:9306476 0 EOF
(00.004891) lockinfo: 64:1 1 2776 08:06:9306456 0 EOF
(00.004893) lockinfo: 65:1 1 2776 08:06:9306455 0 EOF
(00.004896) lockinfo: 66:1 1 2776 08:06:9306454 0 EOF
(00.004898) lockinfo: 67:1 0 3738 08:06:9315261 128 128
(00.004901) lockinfo: 68:1 0 3738 08:06:9306803 1073741826 1073742335
(00.004904) lockinfo: 69:1 0 3738 08:06:9310179 128 128
(00.004910) lockinfo: 70:1 0 3738 08:06:9306185 1073741826 1073742335
(00.004912) lockinfo: 71:1 1 3738 08:06:9306187 0 EOF
(00.004915) lockinfo: 72:1 1 2776 08:06:9306520 0 EOF
(00.004917) lockinfo: 73:1 1 2776 08:06:9306470 0 EOF
(00.004920) lockinfo: 74:1 1 2776 08:06:9306513 0 EOF
(00.004982)     type aufs source none mnt_id 83 s_dev 0x25 / @ ./ flags 0x200000 options si=9dfeb884d3af12,dio
(00.004985) Collecting AUFS branch pathnames ...
(00.005037) /sys/fs/aufs/si_9dfeb884d3af12/br0 : /var/lib/docker/aufs/diff/b8ea7fb3f4a0007a769804a101ba551356d6a3b1909ab242562310874fd9b573
(00.005056) /sys/fs/aufs/si_9dfeb884d3af12/br1 : /var/lib/docker/aufs/diff/b8ea7fb3f4a0007a769804a101ba551356d6a3b1909ab242562310874fd9b573-init
(00.005079) /sys/fs/aufs/si_9dfeb884d3af12/br2 : /var/lib/docker/aufs/diff/becf32719e1a5fc799fa623bf2e693c0ea8893681ff737d17f262da54e5f9e4d
(00.005103) /sys/fs/aufs/si_9dfeb884d3af12/br3 : /var/lib/docker/aufs/diff/45bbaac91df67317e99901baf9315d375d96a2c38ac1d5c6660e596ca5b196cc
(00.005123) /sys/fs/aufs/si_9dfeb884d3af12/br4 : /var/lib/docker/aufs/diff/d46ed293170c46c2bc2130fff99586ce413eceebf92c75a8c1c1f65480450b89
(00.005138) /sys/fs/aufs/si_9dfeb884d3af12/br5 : /var/lib/docker/aufs/diff/2f8f0498fc4a7a005a0a0cd0843214f7c8812f7eb269e72bc6674c331f9ddea9
(00.005154) /sys/fs/aufs/si_9dfeb884d3af12/br6 : /var/lib/docker/aufs/diff/763e1254cc81b1b9f9564ac868dd3ad2cdb259198992cd775b3603e43ab0c35a
(00.005169) /sys/fs/aufs/si_9dfeb884d3af12/br7 : /var/lib/docker/aufs/diff/63c62fd45dfef0b2dd8a75ce53ef260baa0453c76504b59d789c92f3731f2f0b
(00.005184) /sys/fs/aufs/si_9dfeb884d3af12/br8 : /var/lib/docker/aufs/diff/5241f54f8c1fca5ec0c12074284d5c5f463f068d3ba08aa94798268e72575cde
(00.005200) /sys/fs/aufs/si_9dfeb884d3af12/br9 : /var/lib/docker/aufs/diff/a6f2d086ca40208f25a7ad7c1d5bf04986f912b210766358487c44b6f8696c23
(00.005216) /sys/fs/aufs/si_9dfeb884d3af12/br10 : /var/lib/docker/aufs/diff/7cf2bb0df4f1c6606e1685ed4da0a8aa6dc4a0d0ee1340e2b79c6696d54944b2
(00.005233) /sys/fs/aufs/si_9dfeb884d3af12/br11 : /var/lib/docker/aufs/diff/f928e22e58b0596faa4c2e45fd12195a1e616c3b096e10e6856446c53db6dd18
(00.005248) /sys/fs/aufs/si_9dfeb884d3af12/br12 : /var/lib/docker/aufs/diff/543ff72402d8d796cae34d4c980e247debd9bace8a7e22819dd1693541bb084e
(00.005263) /sys/fs/aufs/si_9dfeb884d3af12/br13 : /var/lib/docker/aufs/diff/273cd71eacf0b918851435d73dcfeb067239b9bfcae37def64f4e13726299eb1
(00.005278) /sys/fs/aufs/si_9dfeb884d3af12/br14 : /var/lib/docker/aufs/diff/c2b0136be90f7370c0a3372157f53f9b84e15a91b34e73011b47f62dd23077e7
(00.005293) /sys/fs/aufs/si_9dfeb884d3af12/br15 : /var/lib/docker/aufs/diff/60c52dbe9d9121f0baf4d44fece2d447a0a48f4da84522f0eb7082a0ca6b465e
(00.005309) /sys/fs/aufs/si_9dfeb884d3af12/br16 : /var/lib/docker/aufs/diff/4c8cbfd2973e1b92137486922e1af5677fced4f6902e6513b81413e22f33ceab
(00.005317)     type proc source proc mnt_id 84 s_dev 0x27 / @ ./proc flags 0x20000e options 
(00.005323)     type tmpfs source tmpfs mnt_id 85 s_dev 0x28 / @ ./dev flags 0x1000000 options mode=755
(00.005328)     type devpts source devpts mnt_id 86 s_dev 0x29 / @ ./dev/pts flags 0x20000a options gid=5,mode=620,ptmxmode=666
(00.005340)     type tmpfs source shm mnt_id 87 s_dev 0x2a / @ ./dev/shm flags 0x20000e options size=65536k
(00.005344)     type mqueue source mqueue mnt_id 88 s_dev 0x26 / @ ./dev/mqueue flags 0x20000e options 
(00.005348)     type sysfs source sysfs mnt_id 89 s_dev 0x2b / @ ./sys flags 0x20000f options 
(00.005352)     type tmpfs source tmpfs mnt_id 90 s_dev 0x2c / @ ./sys/fs/cgroup flags 0x20000f options mode=755
(00.005357)     type cgroup source cgroup mnt_id 91 s_dev 0x14 /docker/b8ea7fb3f4a0007a769804a101ba551356d6a3b1909ab242562310874fd9b573 @ ./sys/fs/cgroup/cpuset flags 0x20000f options cpuset
(00.005361)     type cgroup source cgroup mnt_id 92 s_dev 0x15 /docker/b8ea7fb3f4a0007a769804a101ba551356d6a3b1909ab242562310874fd9b573 @ ./sys/fs/cgroup/cpu flags 0x20000f options cpu
(00.005384)     type cgroup source cgroup mnt_id 93 s_dev 0x16 /docker/b8ea7fb3f4a0007a769804a101ba551356d6a3b1909ab242562310874fd9b573 @ ./sys/fs/cgroup/cpuacct flags 0x20000f options cpuacct
(00.005389)     type cgroup source cgroup mnt_id 94 s_dev 0x17 /docker/b8ea7fb3f4a0007a769804a101ba551356d6a3b1909ab242562310874fd9b573 @ ./sys/fs/cgroup/memory flags 0x20000f options memory
(00.005394)     type cgroup source cgroup mnt_id 95 s_dev 0x18 /docker/b8ea7fb3f4a0007a769804a101ba551356d6a3b1909ab242562310874fd9b573 @ ./sys/fs/cgroup/devices flags 0x20000f options devices
(00.005398)     type cgroup source cgroup mnt_id 96 s_dev 0x19 /docker/b8ea7fb3f4a0007a769804a101ba551356d6a3b1909ab242562310874fd9b573 @ ./sys/fs/cgroup/freezer flags 0x20000f options freezer
(00.005402)     type cgroup source cgroup mnt_id 97 s_dev 0x1a /docker/b8ea7fb3f4a0007a769804a101ba551356d6a3b1909ab242562310874fd9b573 @ ./sys/fs/cgroup/blkio flags 0x20000f options blkio
(00.005406)     type cgroup source cgroup mnt_id 98 s_dev 0x1b /docker/b8ea7fb3f4a0007a769804a101ba551356d6a3b1909ab242562310874fd9b573 @ ./sys/fs/cgroup/perf_event flags 0x20000f options perf_event
(00.005424)     type cgroup source cgroup mnt_id 99 s_dev 0x1c /docker/b8ea7fb3f4a0007a769804a101ba551356d6a3b1909ab242562310874fd9b573 @ ./sys/fs/cgroup/hugetlb flags 0x20000f options hugetlb
(00.005429)     type cgroup source systemd mnt_id 100 s_dev 0x22 / @ ./sys/fs/cgroup/systemd flags 0x20000f options name=systemd
(00.005435)     type ext4 source /dev/disk/by-uuid/43310852-abc4-4b8b-81bd-9c4b444e3a5f mnt_id 101 s_dev 0x800006 /var/lib/docker/volumes/344d98eb8e8e7d4d419941111032826b86db1bdb507e103e41638d39f55f403d/_data @ ./var/lib/mysql flags 0x200000 options errors=remount-ro,data=ordered
(00.005440)     type ext4 source /dev/disk/by-uuid/43310852-abc4-4b8b-81bd-9c4b444e3a5f mnt_id 102 s_dev 0x800006 /var/lib/docker/containers/b8ea7fb3f4a0007a769804a101ba551356d6a3b1909ab242562310874fd9b573/resolv.conf @ ./etc/resolv.conf flags 0x200000 options errors=remount-ro,data=ordered
(00.005447)     type ext4 source /dev/disk/by-uuid/43310852-abc4-4b8b-81bd-9c4b444e3a5f mnt_id 104 s_dev 0x800006 /var/lib/docker/containers/b8ea7fb3f4a0007a769804a101ba551356d6a3b1909ab242562310874fd9b573/hostname @ ./etc/hostname flags 0x200000 options errors=remount-ro,data=ordered
(00.005461)     type ext4 source /dev/disk/by-uuid/43310852-abc4-4b8b-81bd-9c4b444e3a5f mnt_id 105 s_dev 0x800006 /var/lib/docker/containers/b8ea7fb3f4a0007a769804a101ba551356d6a3b1909ab242562310874fd9b573/hosts @ ./etc/hosts flags 0x200000 options errors=remount-ro,data=ordered
(00.005466)     type proc source proc mnt_id 50 s_dev 0x27 /asound @ ./proc/asound flags 0x20000f options 
(00.005470)     type proc source proc mnt_id 52 s_dev 0x27 /bus @ ./proc/bus flags 0x20000f options 
(00.005474)     type proc source proc mnt_id 53 s_dev 0x27 /fs @ ./proc/fs flags 0x20000f options 
(00.005477)     type proc source proc mnt_id 54 s_dev 0x27 /irq @ ./proc/irq flags 0x20000f options 
(00.005481)     type proc source proc mnt_id 55 s_dev 0x27 /sys @ ./proc/sys flags 0x20000f options 
(00.005485)     type proc source proc mnt_id 56 s_dev 0x27 /sysrq-trigger @ ./proc/sysrq-trigger flags 0x20000f options 
(00.005489)     type tmpfs source tmpfs mnt_id 57 s_dev 0x28 /null @ ./proc/kcore flags 0x1000000 options mode=755
(00.005493)     type tmpfs source tmpfs mnt_id 58 s_dev 0x28 /null @ ./proc/latency_stats flags 0x1000000 options mode=755
(00.005497)     type tmpfs source tmpfs mnt_id 59 s_dev 0x28 /null @ ./proc/timer_stats flags 0x1000000 options mode=755
(00.005505) Building mountpoints tree
(00.005507)     Building plain mount tree
(00.005508)         Working on 59->84
(00.005509)         Working on 58->84
(00.005511)         Working on 57->84
(00.005512)         Working on 56->84
(00.005513)         Working on 55->84
(00.005514)         Working on 54->84
(00.005516)         Working on 53->84
(00.005517)         Working on 52->84
(00.005518)         Working on 50->84
(00.005519)         Working on 105->83
(00.005520)         Working on 104->83
(00.005522)         Working on 102->83
(00.005523)         Working on 101->83
(00.005524)         Working on 100->90
(00.005525)         Working on 99->90
(00.005527)         Working on 98->90
(00.005528)         Working on 97->90
(00.005529)         Working on 96->90
(00.005530)         Working on 95->90
(00.005533)         Working on 94->90
(00.005534)         Working on 93->90
(00.005535)         Working on 92->90
(00.005536)         Working on 91->90
(00.005537)         Working on 90->89
(00.005539)         Working on 89->83
(00.005540)         Working on 88->85
(00.005541)         Working on 87->85
(00.005542)         Working on 86->85
(00.005544)         Working on 85->83
(00.005545)         Working on 84->83
(00.005546)         Working on 83->49
(00.005547)     Resorting siblings on 83
(00.005549)     Resorting siblings on 105
(00.005550)     Resorting siblings on 104
(00.005551)     Resorting siblings on 102
(00.005552)     Resorting siblings on 101
(00.005554)     Resorting siblings on 89
(00.005555)     Resorting siblings on 90
(00.005556)     Resorting siblings on 100
(00.005557)     Resorting siblings on 99
(00.005559)     Resorting siblings on 98
(00.005560)     Resorting siblings on 97
(00.005561)     Resorting siblings on 96
(00.005562)     Resorting siblings on 95
(00.005564)     Resorting siblings on 94
(00.005565)     Resorting siblings on 93
(00.005566)     Resorting siblings on 92
(00.005567)     Resorting siblings on 91
(00.005568)     Resorting siblings on 85
(00.005570)     Resorting siblings on 88
(00.005571)     Resorting siblings on 87
(00.005572)     Resorting siblings on 86
(00.005573)     Resorting siblings on 84
(00.005574)     Resorting siblings on 59
(00.005576)     Resorting siblings on 58
(00.005577)     Resorting siblings on 57
(00.005578)     Resorting siblings on 56
(00.005579)     Resorting siblings on 55
(00.005580)     Resorting siblings on 54
(00.005582)     Resorting siblings on 53
(00.005583)     Resorting siblings on 52
(00.005584)     Resorting siblings on 50
(00.005585) Done:
(00.005586) [./](83->49)
(00.005588)  [./sys](89->83)
(00.005589)   [./sys/fs/cgroup](90->89)
(00.005591)    [./sys/fs/cgroup/systemd](100->90)
(00.005592)    <--
(00.005593)    [./sys/fs/cgroup/cpuset](91->90)
(00.005595)    <--
(00.005596)    [./sys/fs/cgroup/cpu](92->90)
(00.005597)    <--
(00.005598)    [./sys/fs/cgroup/cpuacct](93->90)
(00.005600)    <--
(00.005601)    [./sys/fs/cgroup/memory](94->90)
(00.005602)    <--
(00.005603)    [./sys/fs/cgroup/devices](95->90)
(00.005605)    <--
(00.005606)    [./sys/fs/cgroup/freezer](96->90)
(00.005607)    <--
(00.005608)    [./sys/fs/cgroup/blkio](97->90)
(00.005610)    <--
(00.005611)    [./sys/fs/cgroup/perf_event](98->90)
(00.005612)    <--
(00.005613)    [./sys/fs/cgroup/hugetlb](99->90)
(00.005615)    <--
(00.005616)   <--
(00.005617)  <--
(00.005618)  [./proc](84->83)
(00.005620)   [./proc/timer_stats](59->84)
(00.005621)   <--
(00.005622)   [./proc/asound](50->84)
(00.005623)   <--
(00.005625)   [./proc/bus](52->84)
(00.005626)   <--
(00.005627)   [./proc/fs](53->84)
(00.005628)   <--
(00.005629)   [./proc/irq](54->84)
(00.005631)   <--
(00.005632)   [./proc/sys](55->84)
(00.005633)   <--
(00.005634)   [./proc/sysrq-trigger](56->84)
(00.005636)   <--
(00.005637)   [./proc/kcore](57->84)
(00.005638)   <--
(00.005639)   [./proc/latency_stats](58->84)
(00.005641)   <--
(00.005642)  <--
(00.005643)  [./dev](85->83)
(00.005645)   [./dev/mqueue](88->85)
(00.005646)   <--
(00.005647)   [./dev/pts](86->85)
(00.005648)   <--
(00.005649)   [./dev/shm](87->85)
(00.005651)   <--
(00.005652)  <--
(00.005653)  [./etc/hosts](105->83)
(00.005655)  <--
(00.005656)  [./var/lib/mysql](101->83)
(00.005657)  <--
(00.005658)  [./etc/resolv.conf](102->83)
(00.005660)  <--
(00.005661)  [./etc/hostname](104->83)
(00.005662)  <--
(00.005663) <--
(00.005666) Found /etc/hosts mapping for ./etc/hosts mountpoint
(00.005668) Found /etc/hostname mapping for ./etc/hostname mountpoint
(00.005669) Found /etc/resolv.conf mapping for ./etc/resolv.conf mountpoint
(00.005671) Found /var/lib/mysql mapping for ./var/lib/mysql mountpoint
(00.005672) Found /sys/fs/cgroup/systemd mapping for ./sys/fs/cgroup/systemd mountpoint
(00.005674) Found /sys/fs/cgroup/hugetlb mapping for ./sys/fs/cgroup/hugetlb mountpoint
(00.005675) Found /sys/fs/cgroup/perf_event mapping for ./sys/fs/cgroup/perf_event mountpoint
(00.005676) Found /sys/fs/cgroup/blkio mapping for ./sys/fs/cgroup/blkio mountpoint
(00.005678) Found /sys/fs/cgroup/freezer mapping for ./sys/fs/cgroup/freezer mountpoint
(00.005680) Found /sys/fs/cgroup/devices mapping for ./sys/fs/cgroup/devices mountpoint
(00.005681) Found /sys/fs/cgroup/memory mapping for ./sys/fs/cgroup/memory mountpoint
(00.005683) Found /sys/fs/cgroup/cpuacct mapping for ./sys/fs/cgroup/cpuacct mountpoint
(00.005684) Found /sys/fs/cgroup/cpu mapping for ./sys/fs/cgroup/cpu mountpoint
(00.005685) Found /sys/fs/cgroup/cpuset mapping for ./sys/fs/cgroup/cpuset mountpoint
(00.005695) Collecting netns 8/14232
(00.005697) Switching to 14232's net for collecting sockets
(00.023896) unix: Dropping path /var/run/mysqld/mysqld.sock for unlinked bound sk 0x10.0x2b78 real 0x25.0x69
(00.023911)     Collected: ino 0x18e2a peer_ino 0 family    1 type    1 state 10 name (null)
(00.023916)     Collected: ino 0x18a4e peer_ino 0 family    1 type    5 state  7 name (null)
(00.024138)     Collected: ino 0x   18665 family   10 type    1 port     3306 state 10 src_addr ::
(00.024179) Collect netlink sock 0x18d36
(00.024182) Collect netlink sock 0x18a4d
(00.024183) Collect netlink sock 0x18d42
(00.024184) Collect netlink sock 0x18d39
(00.024186) Collect netlink sock 0x18d37
(00.024187) Collect netlink sock 0x18d38
(00.024206) ========================================
(00.024208) Dumping task (pid: 14232)
(00.024209) ========================================
(00.024210) Obtaining task stat ... (00.024247) 
(00.024249) Collecting mappings (pid: 14232)
(00.024250) ----------------------------------------
(00.024530) Replacing /var/lib/docker/aufs/diff/5241f54f8c1fca5ec0c12074284d5c5f463f068d3ba08aa94798268e72575cde/usr/sbin/mysqld with /usr/sbin/mysqld
(00.024536) Saved AUFS paths ./usr/sbin/mysqld and /var/lib/docker/aufs/mnt/b8ea7fb3f4a0007a769804a101ba551356d6a3b1909ab242562310874fd9b573/usr/sbin/mysqld
(00.024555) vma 1cf7000 borrows vfi from previous 400000
(00.024560) vma 1de8000 borrows vfi from previous 1cf7000
(00.025459) Error (proc_parse.c:404): Can't handle non-regular mapping on 14232's map 7f6ba0c22000
(00.025467) Error (cr-dump.c:1487): Collect mappings (pid: 14232) failed with -1
(00.025509) Unlock network
(00.025513) Running network-unlock scripts
(00.025519)     RPC
(00.025620) Unfreezing tasks into 1
(00.025625)     Unseizing 14232 into 1
(00.025783) Error (cr-dump.c:1947): Dumping FAILED.

I think, the only difference is that I haven't apply the patches of torvalds (http://criu.org/Docker)..

Do I have apply them too or is something else?

Thank you

xemul · 2015-08-06T06:59:27Z

Error (proc_parse.c:404): Can't handle non-regular mapping on 14232's map 7f6ba0c22000

Most likely this is AIO ring, to dump it we need the kernel patches mentioned on the page.

klesgidisold · 2015-08-06T09:40:32Z

Thank you @xemul.. I 'll apply the fixes and inform here about the results..

fl0yd · 2015-08-06T12:25:13Z

Running on Ubuntu Server 15.04 you don't need to apply any kernel patches. What distro and kernel are you running?

klesgidisold · 2015-08-06T17:39:49Z

@fl0yd , actually I realized that I hadn't update my kernel on this PC. I use Linux Mint 17 with kernel 3.13.0-24-generic.. :P I'll upgrade it and inform for the results.. Thanks

klesgidisold · 2015-08-06T17:50:56Z

It worked like a charm! :D Guys I appreciate your help!

Before: =[log]=> dump/zdtm/transition/shmem/142/1/dump.log ------------------------ grep Error ------------------------ pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> ------------------------ ERROR OVER ------------------------ After: =[log]=> dump/zdtm/transition/shmem/62/1/dump.log ------------------------ grep Error ------------------------ pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 1: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#13: -4 pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 1: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#14: -4 pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#13: -4 pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#14: -4 pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#13: -4 pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#14: -4 ------------------------ ERROR OVER ------------------------ Signed-off-by: Dmitry Safonov <dsafonov@virtuozzo.com>

Before: =[log]=> dump/zdtm/transition/shmem/142/1/dump.log ------------------------ grep Error ------------------------ pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> ------------------------ ERROR OVER ------------------------ After: =[log]=> dump/zdtm/transition/shmem/62/1/dump.log ------------------------ grep Error ------------------------ pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 1: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#13: -4 pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 1: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#14: -4 pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#13: -4 pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#14: -4 pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#13: -4 pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#14: -4 ------------------------ ERROR OVER ------------------------ Cc: Andrei Vagin <avagin@virtuozzo.com> Signed-off-by: Dmitry Safonov <dsafonov@virtuozzo.com>

Before: =[log]=> dump/zdtm/transition/shmem/142/1/dump.log ------------------------ grep Error ------------------------ pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> ------------------------ ERROR OVER ------------------------ After: =[log]=> dump/zdtm/transition/shmem/62/1/dump.log ------------------------ grep Error ------------------------ pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 1: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#13: -4 pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 1: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#14: -4 pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#13: -4 pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#14: -4 pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#13: -4 pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#14: -4 ------------------------ ERROR OVER ------------------------ Signed-off-by: Dmitry Safonov <dsafonov@virtuozzo.com>

Before: =[log]=> dump/zdtm/transition/shmem/142/1/dump.log ------------------------ grep Error ------------------------ pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> ------------------------ ERROR OVER ------------------------ After: =[log]=> dump/zdtm/transition/shmem/62/1/dump.log ------------------------ grep Error ------------------------ pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 1: e/asm/parasite.h:45): Failed to dump TLS descriptor #13: -4 pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 1: e/asm/parasite.h:45): Failed to dump TLS descriptor #14: -4 pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: e/asm/parasite.h:45): Failed to dump TLS descriptor #13: -4 pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: e/asm/parasite.h:45): Failed to dump TLS descriptor #14: -4 pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: e/asm/parasite.h:45): Failed to dump TLS descriptor #13: -4 pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: e/asm/parasite.h:45): Failed to dump TLS descriptor #14: -4 ------------------------ ERROR OVER ------------------------ Signed-off-by: Dmitry Safonov <dsafonov@virtuozzo.com> Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>

Before: =[log]=> dump/zdtm/transition/shmem/142/1/dump.log ------------------------ grep Error ------------------------ pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> ------------------------ ERROR OVER ------------------------ After: =[log]=> dump/zdtm/transition/shmem/62/1/dump.log ------------------------ grep Error ------------------------ pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 1: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#13: -4 pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 1: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#14: -4 pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#13: -4 pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#14: -4 pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#13: -4 pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: e/asm/parasite.h:45): Failed to dump TLS descriptor checkpoint-restore#14: -4 ------------------------ ERROR OVER ------------------------ Signed-off-by: Dmitry Safonov <dsafonov@virtuozzo.com>

Before: =[log]=> dump/zdtm/transition/shmem/142/1/dump.log ------------------------ grep Error ------------------------ pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> ------------------------ ERROR OVER ------------------------ After: =[log]=> dump/zdtm/transition/shmem/62/1/dump.log ------------------------ grep Error ------------------------ pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 1: e/asm/parasite.h:45): Failed to dump TLS descriptor #13: -4 pie: 1: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 1: e/asm/parasite.h:45): Failed to dump TLS descriptor #14: -4 pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: e/asm/parasite.h:45): Failed to dump TLS descriptor #13: -4 pie: 5: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 5: e/asm/parasite.h:45): Failed to dump TLS descriptor #14: -4 pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: e/asm/parasite.h:45): Failed to dump TLS descriptor #13: -4 pie: 6: Error (/home/travis/build/0x7f454c46/criu/criu/arch/x86/includ> pie: 6: e/asm/parasite.h:45): Failed to dump TLS descriptor #14: -4 ------------------------ ERROR OVER ------------------------ Signed-off-by: Dmitry Safonov <dsafonov@virtuozzo.com> Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>

'info' array is off-by-one, nla_parse_nested() requires destination array (i.e. 'info') to have maxtype+1 (i.e. IFLA_INFO_MAX+1) elements: ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffef823e3f8 WRITE of size 48 at 0x7ffef823e3f8 thread T0 #0 0x7f9ab7a3915b in __asan_memset (/usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0/libasan.so.2+0x8d15b) #1 0x7f9ab6d4e553 in nla_parse (/usr/lib64/libnl-3.so.200+0xa553) #2 0x4acfb7 in dump_one_netdev criu/net.c:445 checkpoint-restore#3 0x4adb60 in dump_one_ethernet criu/net.c:594 checkpoint-restore#4 0x4adb60 in dump_one_link criu/net.c:665 checkpoint-restore#5 0x48af69 in nlmsg_receive criu/libnetlink.c:45 checkpoint-restore#6 0x48af69 in do_rtnl_req criu/libnetlink.c:119 checkpoint-restore#7 0x4b0e86 in dump_links criu/net.c:878 checkpoint-restore#8 0x4b0e86 in dump_net_ns criu/net.c:1651 checkpoint-restore#9 0x4a760d in do_dump_namespaces criu/namespaces.c:985 checkpoint-restore#10 0x4a760d in dump_namespaces criu/namespaces.c:1045 checkpoint-restore#11 0x451ef7 in cr_dump_tasks criu/cr-dump.c:1799 checkpoint-restore#12 0x424588 in main criu/crtools.c:736 checkpoint-restore#13 0x7f9ab67b171f in __libc_start_main (/lib64/libc.so.6+0x2071f) checkpoint-restore#14 0x4253d8 in _start (/criu/criu/criu+0x4253d8) Address 0x7ffef823e3f8 is located in stack of thread T0 at offset 264 in frame #0 0x4ac9ef in dump_one_netdev criu/net.c:364 This frame has 5 object(s): [32, 168) 'netdev' [224, 264) 'info' <== Memory access at offset 264 overflows this variable [320, 1040) 'req' [1088, 3368) 'path' [3424, 3625) 'stable_secret' Increase 'info' size to fix this. Fixes: b705dcc ("net: pass the struct nlattrs to dump() functions") Signed-off-by: Andrey Ryabinin <aryabinin@virtuozzo.com>

'info' array is off-by-one, nla_parse_nested() requires destination array (i.e. 'info') to have maxtype+1 (i.e. IFLA_INFO_MAX+1) elements: ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffef823e3f8 WRITE of size 48 at 0x7ffef823e3f8 thread T0 #0 0x7f9ab7a3915b in __asan_memset (/usr/lib/gcc/x86_64-pc-linux-gnu/5.4.0/libasan.so.2+0x8d15b) #1 0x7f9ab6d4e553 in nla_parse (/usr/lib64/libnl-3.so.200+0xa553) #2 0x4acfb7 in dump_one_netdev criu/net.c:445 #3 0x4adb60 in dump_one_ethernet criu/net.c:594 #4 0x4adb60 in dump_one_link criu/net.c:665 #5 0x48af69 in nlmsg_receive criu/libnetlink.c:45 #6 0x48af69 in do_rtnl_req criu/libnetlink.c:119 #7 0x4b0e86 in dump_links criu/net.c:878 #8 0x4b0e86 in dump_net_ns criu/net.c:1651 #9 0x4a760d in do_dump_namespaces criu/namespaces.c:985 #10 0x4a760d in dump_namespaces criu/namespaces.c:1045 #11 0x451ef7 in cr_dump_tasks criu/cr-dump.c:1799 #12 0x424588 in main criu/crtools.c:736 #13 0x7f9ab67b171f in __libc_start_main (/lib64/libc.so.6+0x2071f) #14 0x4253d8 in _start (/criu/criu/criu+0x4253d8) Address 0x7ffef823e3f8 is located in stack of thread T0 at offset 264 in frame #0 0x4ac9ef in dump_one_netdev criu/net.c:364 This frame has 5 object(s): [32, 168) 'netdev' [224, 264) 'info' <== Memory access at offset 264 overflows this variable [320, 1040) 'req' [1088, 3368) 'path' [3424, 3625) 'stable_secret' Increase 'info' size to fix this. Fixes: b705dcc ("net: pass the struct nlattrs to dump() functions") travis-ci: success for net: fix stack out-of-bounds access in dump_one_netdev() Signed-off-by: Andrey Ryabinin <aryabinin@virtuozzo.com> Acked-by: Cyrill Gorcunov <gorcunov@openvz.org> Signed-off-by: Pavel Emelyanov <xemul@virtuozzo.com>

xemul closed this as completed Aug 5, 2015

throwbear mentioned this issue May 15, 2021

[Python Supervisord] criu restore is successful, But the supervisord program coredump #1477

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Trying to Checkpoint MySql container - criu failed: type NOTIFY errno 0 #14

Trying to Checkpoint MySql container - criu failed: type NOTIFY errno 0 #14

klesgidisold commented Aug 4, 2015

xemul commented Aug 4, 2015

boucher commented Aug 4, 2015

boucher commented Aug 5, 2015

klesgidisold commented Aug 5, 2015

boucher commented Aug 5, 2015

klesgidisold commented Aug 5, 2015

xemul commented Aug 5, 2015

klesgidisold commented Aug 5, 2015

xemul commented Aug 6, 2015

klesgidisold commented Aug 6, 2015

fl0yd commented Aug 6, 2015

klesgidisold commented Aug 6, 2015

klesgidisold commented Aug 6, 2015

Trying to Checkpoint MySql container - criu failed: type NOTIFY errno 0 #14

Trying to Checkpoint MySql container - criu failed: type NOTIFY errno 0 #14

Comments

klesgidisold commented Aug 4, 2015

xemul commented Aug 4, 2015

boucher commented Aug 4, 2015

boucher commented Aug 5, 2015

klesgidisold commented Aug 5, 2015

boucher commented Aug 5, 2015

klesgidisold commented Aug 5, 2015

xemul commented Aug 5, 2015

klesgidisold commented Aug 5, 2015

xemul commented Aug 6, 2015

klesgidisold commented Aug 6, 2015

fl0yd commented Aug 6, 2015

klesgidisold commented Aug 6, 2015

klesgidisold commented Aug 6, 2015