beta4: containers don't obtain ip address #1666

Closed
skyxx opened this Issue Feb 26, 2016 · 6 comments

Comments

Projects
None yet
5 participants
@skyxx

skyxx commented Feb 26, 2016

The template below is mostly useful for bug reports and support questions.
Feel free to remove anything which doesn't apply to you and add more information where it makes sense.

Required information

  • Distribution: Ubuntu Xenial server amd64
  • Distribution version: 16.04
  • The output of "lxc info" or if that fails:
    • Kernel version: 4.4.0-7-generic #22-Ubuntu
    • LXC version: 2.0.0.beta4
    • LXD version: 2.0.0.beta4
    • Storage backend in use: BTRFS
    • driverversion: 2.0.0.rc3
    • storageversion: "4.4"

Issue description

Hi.
When I upgrade from beta3 to beta4:

  • networking stop working, containers don't obtain ip address.
  • also i cannot stop container (it hangs)

Thanks.

Information to attach

  • Syslog looks clean.

  • main daemon log (/var/log/lxd.log) .. looks clean

  • container lxc.log contains ERRORs and WARNs

        lxc 20160226173103.237 ERROR    lxc_utils - utils.c:safe_mount:1691 - Device or resource busy - Failed to mount mqueue onto /usr/lib/x86_64-linux-gnu/lxc/dev/mqueue
        lxc 20160226173103.238 ERROR    lxc_utils - utils.c:open_without_symlink:1625 - No such file or directory - Error examining efi in /usr/lib/x86_64-linux-gnu/lxc/sys/firmware/efi/efivars
    
        lxc 20160226173103.063 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1121 - Not attaching to cgroup name=systemd unknown to /var/lib/lxd/containers test
        lxc 20160226173103.063 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1121 - Not attaching to cgroup net_cls unknown to /var/lib/lxd/containers test
        lxc 20160226173103.063 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1121 - Not attaching to cgroup pids unknown to /var/lib/lxd/containers test
        lxc 20160226173103.063 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1121 - Not attaching to cgroup freezer unknown to /var/lib/lxd/containers test
        lxc 20160226173103.063 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1121 - Not attaching to cgroup blkio unknown to /var/lib/lxd/containers test
        lxc 20160226173103.063 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1121 - Not attaching to cgroup hugetlb unknown to /var/lib/lxd/containers test
        lxc 20160226173103.063 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1121 - Not attaching to cgroup cpu unknown to /var/lib/lxd/containers test
        lxc 20160226173103.063 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1121 - Not attaching to cgroup memory unknown to /var/lib/lxd/containers test
        lxc 20160226173103.063 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1121 - Not attaching to cgroup perf_event unknown to /var/lib/lxd/containers test
        lxc 20160226173103.063 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1121 - Not attaching to cgroup devices unknown to /var/lib/lxd/containers test
        lxc 20160226173103.063 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1121 - Not attaching to cgroup cpuset unknown to /var/lib/lxd/containers test
        lxc 20160226173103.064 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1121 - Not attaching to cgroup name=systemd unknown to /var/lib/lxd/containers test
        lxc 20160226173103.064 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1121 - Not attaching to cgroup net_cls unknown to /var/lib/lxd/containers test
        lxc 20160226173103.064 WARN     lxc_cgfs - cgfs.c:lxc_cgroup_get_container_info:1121 - Not attaching to cgroup pids unknown to /var/lib/lxd/containers test
    

    ...
    ...

@stgraber

This comment has been minimized.

Show comment
Hide comment
@stgraber

stgraber Feb 26, 2016

Member

The 4.4.0-7-generic kernel is unfortunately broken. Please reboot on 4.4.0-6-generic or wait for 4.4.0-8-generic.

The -7 kernel fails to mount cgroupfs in the container with EBUSY which causes systemd or anything else which attempts to mount cgroups to fail.

The fixed kernel is currently going through testing and should be released soon.

Member

stgraber commented Feb 26, 2016

The 4.4.0-7-generic kernel is unfortunately broken. Please reboot on 4.4.0-6-generic or wait for 4.4.0-8-generic.

The -7 kernel fails to mount cgroupfs in the container with EBUSY which causes systemd or anything else which attempts to mount cgroups to fail.

The fixed kernel is currently going through testing and should be released soon.

@stgraber stgraber closed this Feb 26, 2016

@skyxx

This comment has been minimized.

Show comment
Hide comment
@skyxx

skyxx Feb 26, 2016

Thank you. I fallback to 4.4.0.6 and now it work okay.

skyxx commented Feb 26, 2016

Thank you. I fallback to 4.4.0.6 and now it work okay.

@vince06fr

This comment has been minimized.

Show comment
Hide comment
@vince06fr

vince06fr Feb 27, 2016

4.4.0.8 is out but the issue always here

4.4.0.8 is out but the issue always here

@ivan

This comment has been minimized.

Show comment
Hide comment
@ivan

ivan Apr 21, 2016

I still see Failed to mount mqueue onto /usr/lib/x86_64-linux-gnu/lxc/dev/mqueue failures about half the time (using just LXC 2.0.0-0ubuntu2) on Ubuntu 16.04 / Linux 4.4.0-21-generic.

After that happens, nothing can kill the containers. If you try to kill their inits, they go into uninterruptable sleep. systemd can't shut down the host machine even with 7x ctrl-alt-del.

I strongly recommend LXC users avoid Ubuntu 16.04 until this is fixed.

Update: problems seem to go away when downgrading to linux-image 4.2.0-35-generic. Will try to make this reproducible later.

ivan commented Apr 21, 2016

I still see Failed to mount mqueue onto /usr/lib/x86_64-linux-gnu/lxc/dev/mqueue failures about half the time (using just LXC 2.0.0-0ubuntu2) on Ubuntu 16.04 / Linux 4.4.0-21-generic.

After that happens, nothing can kill the containers. If you try to kill their inits, they go into uninterruptable sleep. systemd can't shut down the host machine even with 7x ctrl-alt-del.

I strongly recommend LXC users avoid Ubuntu 16.04 until this is fixed.

Update: problems seem to go away when downgrading to linux-image 4.2.0-35-generic. Will try to make this reproducible later.

@hallyn

This comment has been minimized.

Show comment
Hide comment
@hallyn

hallyn Apr 21, 2016

Member

@ivan please open a new bug showing exactly how you create and start the containers, so we can reproduce.

Member

hallyn commented Apr 21, 2016

@ivan please open a new bug showing exactly how you create and start the containers, so we can reproduce.

@stgraber

This comment has been minimized.

Show comment
Hide comment
@stgraber

stgraber Apr 21, 2016

Member

Also note that the mqueue error isn't your problem at all, the mqueue mount is marked as optional, so look for other errors because that's not it.

Member

stgraber commented Apr 21, 2016

Also note that the mqueue error isn't your problem at all, the mqueue mount is marked as optional, so look for other errors because that's not it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment