Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running docker on RHEL #172

Closed
davidreuss opened this issue Mar 26, 2013 · 140 comments

Comments

@davidreuss
Copy link

commented Mar 26, 2013

This issue is meant for tracking dockers status on RHEL (Red Hat Enterprise Linux)

The one main pain-point i believe is AUFS, which isn't directly available on RHEL.

I haven't myself had as much time to play around with it, but i wanted a central location for storing progress/feedback from people trying to get docker running.

@shykes

This comment has been minimized.

Copy link
Collaborator

commented Mar 26, 2013

@jpetazzo

This comment has been minimized.

Copy link
Contributor

commented Mar 27, 2013

You might want to try this kernel:
http://get.docker.io/kernels/kernel-3.2.40_grsec_dotcloud-4.x86_64.rpm

Warning: the package is 700MB! I'm sorry, that's what I got when running make rpm on our kernel tree.

This kernel features:

  • grsec (for hardened security)
  • AUFS
  • setns support

I don't have any RHEL/Fedora/CentOS machine around, so I can't test it, but it's compiled with exactly the same .config as our kernels. It is configured to run on bare metal, KVM virtual machines, Xen virtual machines (and therefore supports EC2). It might work on HyperV but I never tested it there.

@shykes

This comment has been minimized.

Copy link
Collaborator

commented Mar 27, 2013

cc @brianm

@shykes

This comment has been minimized.

Copy link
Collaborator

commented Mar 27, 2013

Anyone else interested in Red Hat support?

@bfulton

This comment has been minimized.

Copy link

commented Mar 27, 2013

+1

@scrothers

This comment has been minimized.

Copy link

commented Mar 27, 2013

I'm interested in building a proper AUFS RPM against the existing sources, honestly I think having it released to EPEL or something would be best. Future maintanability and development ecosystem and so on.

I'll start researching what this effort will take and open a case and link it to this one.

@mrallen1

This comment has been minimized.

Copy link
Contributor

commented Mar 28, 2013

+1

Sent from my iPhone

On Mar 27, 2013, at 5:51 PM, Solomon Hykes notifications@github.com wrote:

Anyone else interested in Red Hat support?


Reply to this email directly or view it on GitHub.

@ryan-mars

This comment has been minimized.

Copy link

commented Mar 28, 2013

+1

@josephglanville

This comment has been minimized.

Copy link

commented Mar 28, 2013

Ideally a DKMS RPM should be built. I am pretty sure RHEL 6 uses DKMS and it will keep AUFS working across kernel upgrades.

@jpetazzo

This comment has been minimized.

Copy link
Contributor

commented Mar 28, 2013

I could be wrong, but it seems that AUFS doesn't support building through
DKMS.
Unless there is a generic way to build any kind of module with DKMS?
I would be quite surprised though; because AUFS hooks itself within some
very specific places within the VFS layer.

On Wed, Mar 27, 2013 at 11:51 PM, Joseph Glanville <notifications@github.com

wrote:

Ideally a DKMS RPM should be build. I am pretty sure RHEL 6 uses DKMS and
it will keep AUFS working across kernel upgrades.


Reply to this email directly or view it on GitHubhttps://github.com//issues/172#issuecomment-15571153
.

@josephglanville

This comment has been minimized.

Copy link

commented Mar 28, 2013

DKMS can build any module. It's the same as doing make, make install in the module source directory. (or doing make modules; make modules_install in the kernel source tree).

The only thing that DKMS does for you is automatically rebuild the module against the newly installed kernel so that it has the correct symbols.

@jpetazzo

This comment has been minimized.

Copy link
Contributor

commented Mar 28, 2013

It's worth trying, but even when it's built as a module, AUFS patches the
VFS layer, AFAIR.

@josephglanville

This comment has been minimized.

Copy link

commented Mar 28, 2013

Ahh yes you are correct, my mistake. :(

Shame about that, I hope one day AUFS becomes mainline so this doesn't happen sigh.

@timbunce

This comment has been minimized.

Copy link

commented Mar 28, 2013

+1 for RHEL/CentOS

@scrothers

This comment has been minimized.

Copy link

commented Mar 28, 2013

It appears that DKMS will not work for this, as we cannot logically separate the components of AUFS from the Kernel image 100%. It still requires the Kernel image be patched with specific AUFS code.

The only realistic way to include this is to patch it directly into the EL6 upstream Kernel and release that code as buildable. Which is what I'm going to pursue right now. Once it's complete I'll be releasing it on my Github, however a longer term goal may be to operate a specific Kernel for Docker anyway. Include all the new fancy cgroup utils and other filesystems.

Thoughts?

@jpetazzo

This comment has been minimized.

Copy link
Contributor

commented Mar 29, 2013

If you can point me to the EL6 source tree, I can add it to our kernel build farm (with some luck, our AUFS patch queue will apply neatly; or at least, require only minor work to merge).

FWIW, I recompiled our production kernels without debugging symbols (so now the RPM is just 50MB instead of 700MB):

http://get.docker.io/kernels/kernel-3.2.40_grsec_dotcloud-4.x86_64.rpm

If someone could give it a try, that would be awesome.

@scrothers

This comment has been minimized.

Copy link

commented Mar 29, 2013

Depends, are you using AUFS 3.x? If so the RHEL Kernel won't work, since it's a 2.6.32 Kernel. It'll be missing most of the API/ABI that I'm assuming AUFS 3 requires since it specifically states Kernel 3.0 requirement.

However though, as requested, the current SRPM:
http://ftp.redhat.com/pub/redhat/linux/enterprise/6Server/en/os/SRPMS/kernel-2.6.32-358.2.1.el6.src.rpm

Obviously you can unpack the source using rpm, in the case you're on debian or osx (with brew) or something...

rpm2cpio kernel-blablabla.rpm | cpio -idmv

Also, going to give that Kernel a try now on an EL6 machine.

@scrothers

This comment has been minimized.

Copy link

commented Mar 29, 2013

Yea, there's a lot of depsolving and conflicts in this Kernel you have posted...

Where is the srpm at? I'll be happy to work the spec out.

Ideally, this Kernel wouldn't upgrade or replace the existing RHEL Kernel (so it'll remain updated) it should be supplemental to the RHEL Kernel and updated separately.

Errors here:
https://gist.github.com/stevencrothers/5269833/raw/a42f40f6feb2813fb6a78b4e80cfcb08c1cc525e/gistfile1.txt

I'm going to open a case for this so it can be properly tracked.

@scrothers

This comment has been minimized.

Copy link

commented Mar 29, 2013

For the record, new case is here to track the specific Kernel changes:
#253

This case should be made dependent on 253, not sure if that's possible on Github.

@stahnma

This comment has been minimized.

Copy link

commented May 2, 2013

It seems like targeting Fedora 18/19 (and thus what will likely become EL7) would be easier/better than working on the 2.6 kernels in EL6.

@shykes

This comment has been minimized.

Copy link
Collaborator

commented May 2, 2013

FYI Ubuntu is discontinuing support for aufs in their 3.9 kernels, so we
might be forced to extend support to another filesystem sooner rather than
later. I started playing with btrfs, and will continue to tinker during my
upcoming vacation :)

On Wed, May 1, 2013 at 10:47 PM, Michael Stahnke
notifications@github.comwrote:

It seems like targeting Fedora 18/19 (and thus what will likely become
EL7) would be easier/better than working on the 2.6 kernels in EL6.


Reply to this email directly or view it on GitHubhttps://github.com//issues/172#issuecomment-17321530
.

@pdf

This comment has been minimized.

Copy link

commented May 15, 2013

OverlayFS is attempting (again) acceptance for merge into mainline 3.10, but it still seems unlikely at this point, since Al Viro (the VFS maintainer) doesn't like some chunks of the design (primarily use of xattrs for opaques), which seems to be causing a little friction since Miklos is somewhat attached to the idea for portability (FWIW, I kind of agree with Miklos here).

That said, it's already included in the standard kernel patchsets for at least Ubuntu and SuSE, maybe others. and as @shykes points out, aufs is soon to be dropped for Ubuntu so some decision here will have to be made.

@jpetazzo

This comment has been minimized.

Copy link
Contributor

commented May 15, 2013

I'm actively working on btrfs support for docker [1].
It's not ready yet, but it should offer a pretty solid alternative for
systems without aufs.

[1] jpetazzo/docker@master...btrfs

@dmacvicar

This comment has been minimized.

Copy link

commented May 18, 2013

I just hacked support for overlayfs, which is is already on the SUSE and Ubuntu kernels, and probably the only one that will make it to upstream. I can't get the tests to pass because an unmount fails in the test. However I can run the container and docker diff seems to be working.
https://github.com/dmacvicar/docker/tree/feature_overlayfs

//cc @flavio @silviomoioli

@kstaken

This comment has been minimized.

Copy link
Contributor

commented May 27, 2013

@dmacvicar Did you change FILESYSTEM_MAX_STACK_DEPTH in your kernel to allow overlayfs to stack more than 2 layers?

edit: just noticed this limit may not apply on opensuse if they don't have that check in their version of overlayfs. On ubuntu FILESYSTEM_MAX_STACK_DEPTH is 2 so mounting a stack a b c d will fail when you go to mount d.

@dmacvicar

This comment has been minimized.

Copy link

commented May 28, 2013

@kstaken Mmmm... I only tried with 3 layers so you may be right base + 1 + 1 is still in the max range.

I need to check whether our kernel patches also have that limit.

@dmacvicar

This comment has been minimized.

Copy link

commented May 28, 2013

@kstaken you are completely right

May 28 10:21:47 piscola kernel: overlayfs: maximum fs stacking depth exceeded

I oversaw that. Documentation says "The lower filesystem can even be another overlayfs", but did not mention any limits. ARGH.

I would guess for docker the main usecase would be to have an image and keep the changes of the image separate, and then commit them to another image. Are more layers at the same time useful?
Because then we could introduce the limit to docker itself, when using overlayfs.

@mztriz

This comment has been minimized.

Copy link

commented Sep 17, 2013

+1 I would like to see some RHEL support as well.

@gamefiend

This comment has been minimized.

Copy link

commented Sep 17, 2013

+1, yes please.

@vannenc

This comment has been minimized.

Copy link

commented Sep 19, 2013

+1, for CentOS/RedHat

@kencochrane

This comment has been minimized.

@bflad

This comment has been minimized.

Copy link
Contributor

commented Sep 19, 2013

That is awesome news.

@jefferai

This comment has been minimized.

Copy link

commented Sep 25, 2013

Is there any information about where this plugin API is being designed? I don't see any relevant branches in the Github repository. I'm very interested in the possibility of a ZoL plugin as an alternative to btrfs and thinly-provisioned device-mapper, but if a btrfs plugin is already in development, it would be nice to be able to see that code to see what it would look like to make a ZoL version.

@doublerebel

This comment has been minimized.

Copy link

commented Sep 28, 2013

👍

@pungoyal

This comment has been minimized.

Copy link

commented Oct 10, 2013

+1 for Red Hat support

@leifwalsh

This comment has been minimized.

Copy link

commented Oct 22, 2013

+1 centos 6, and 5 if you can get it

@goldmann

This comment has been minimized.

Copy link
Contributor

commented Nov 8, 2013

A blog post describing the details about RHEL/Fedora support for Docker: http://community.redhat.com/adventures-in-dockerland/

@sciurus

This comment has been minimized.

Copy link

commented Nov 26, 2013

Now that

  • Docker 0.7 is released and supports devicemapper thin provisioning as an alternative to aufs
  • Docker has been packaged in Fedora and EPEL

I think this issue can be closed.

@goldmann

This comment has been minimized.

Copy link
Contributor

commented Nov 26, 2013

It's packaged, but not pushed to the repos yet. Please hold on, we'll be there soon!

@blalor

This comment has been minimized.

Copy link

commented Nov 26, 2013

Will Docker 0.7 work on CentOS 6? I think that's still on kernel 2.6, but the Docker docks say 3.8 (?) is required.

@NuxRo

This comment has been minimized.

Copy link

commented Nov 26, 2013

It works nicely:
yum --enablerepo=epel-testing install docker-io libcgroup
service cgconfig start
service docker start

@blalor

This comment has been minimized.

Copy link

commented Nov 26, 2013

Excellent. yum resolutely refused to find docker-io in epel-testing, but I used “yum localinstall” with the URL to the package IN THE EPEL-TESTING REPO and it works a treat. Yum even found lxc and lxc-libs in epel-testing, just not docker-io. I just started the centos container on CentOS.

MY GOD, IT’S FULL OF STARS!

@blalor

This comment has been minimized.

Copy link

commented Nov 27, 2013

Er, not quite. The docker-io service in epel-testing starts with -b none, so networking isn't available. Attempting to start docker without that flag causes it to fail.

[root@vagrant-centos63 tmp]# /usr/bin/docker -D -r=false -d
2013/11/26 19:25:07 WARNING: You are running linux kernel version 2.6.32-279.19.1.el6.x86_64, which might be unstable running docker. Please upgrade your kernel to 3.8.0.
[/var/lib/docker|f4b03b56] +job initapi()
[/var/lib/docker|f4b03b56.initapi()] Creating server
[debug] driver.go:84 Error loading driver aufs: AUFS was not found in /proc/filesystems
[debug] deviceset.go:475 Generated prefix: docker-253:0-404404
[debug] deviceset.go:478 Checking for existence of the pool 'docker-253:0-404404-pool'
[debug] deviceset.go:237 loadMetadata()
[debug] deviceset.go:242 loadMetadata END
[debug] runtime.go:642 Using graph driver devicemapper
[debug] network.go:150 Creating bridge docker0 with network 172.17.42.1/16
[/var/lib/docker|f4b03b56] -job initapi() = ERR (Error creating bridge: operation not supported)
2013/11/26 19:25:07 initapi: Error creating bridge: operation not supported
@bflad

This comment has been minimized.

Copy link
Contributor

commented Nov 27, 2013

@NuxRo

This comment has been minimized.

Copy link

commented Nov 27, 2013

Right, so you either need to create a bridge (docker0) by yourself and specify it in /etc/sysconfig/docker or go for a newer kernel.

Kernel-lt from Elrepo works fine at a first glance.

@NuxRo

This comment has been minimized.

Copy link

commented Nov 27, 2013

Until docker-io makes a comeback in epel-testing it can be downloaded from http://koji.fedoraproject.org/koji/packageinfo?packageID=17053

@blalor

This comment has been minimized.

Copy link

commented Nov 27, 2013

I'm working on a sample Vagrant box and bootstrapping script for CentOS. https://github.com/blalor/vagrant-centos64-docker It's not working yet, but there has been some good discussion on docker-dev today. You don't really need an RPM as you can just grab the docker binary. My Vagrant config still doesn't work (lxc incompatibility?) but I hope to get back to it later tonight to get it working.

@bflad

This comment has been minimized.

Copy link
Contributor

commented Dec 3, 2013

Just to follow up with this thread, the bridging and networking on CentOS 6 should be fixed along with the libcgroup/cgconfig fixes in docker-io packages as of last few days. According to Red Hat Bugzilla 1000662 and Fedora build system, the latest docker-io package (docker-io-0.7.0-14) is being pushed to EPEL 6 stable right now. Fedora build system is showing FC19/FC20 being pushed to updates stable repositories shortly as well.

Great work everyone!

@blalor

This comment has been minimized.

Copy link

commented Dec 3, 2013

Great news!! Last I looked the docker-io package didn't set up the docker0 bridge. Does it now do that, or is that an additional step required to get Docker containers to have network access?

@bflad

This comment has been minimized.

Copy link
Contributor

commented Dec 3, 2013

All handled without effort with latest package, at least from what I can tell. :)

[root@docker-centos-6 ~]# rpm -ihv http://kojipkgs.fedoraproject.org//packages/docker-io/0.7.0/14.el6/x86_64/docker-io-0.7.0-14.el6.x86_64.rpm
Retrieving http://kojipkgs.fedoraproject.org//packages/docker-io/0.7.0/14.el6/x86_64/docker-io-0.7.0-14.el6.x86_64.rpm
Preparing...                ########################################### [100%]
   1:docker-io              ########################################### [100%]
[root@docker-centos-6 ~]# cat /etc/sysconfig/docker
other_args=""
[root@docker-centos-6 ~]# service docker start
Starting cgconfig service:                                 [  OK  ]
Starting docker:                                     [  OK  ]
[root@docker-centos-6 ~]# ps aux | grep docker
root      5111  1.0  1.2 542220 12416 pts/0    Sl   03:21   0:00 /usr/bin/docker -d
[root@docker-centos-6 ~]# docker run -d -p 9999:9999 busybox /bin/nc -l -l -p 9999
Unable to find image 'busybox' (tag: latest) locally
Pulling repository busybox
e9aa60c60128: Download complete
661e7e01438e581b0acce27ea15856b824461cc64914e46b8d374e954013188b
[root@docker-centos-6 ~]# netstat -ntlp | grep 9999
tcp        0      0 :::9999                     :::*                        LISTEN      5111/docker
[root@docker-centos-6 ~]# telnet ::1 9999
Trying ::1...
Connected to ::1.
[root@docker-centos-6 ~]# brctl show
bridge name bridge id   STP enabled interfaces
docker0   8000.000000000000 no
@goldmann

This comment has been minimized.

Copy link
Contributor

commented Dec 3, 2013

@blalor It's fixed in the latest update.

All--

From now on please report all issues regarding docker-io EPEL package in the Red Hat Bugzilla. Thanks!

I'll update the docs with instruction on how to install Docker on CentOS/RHEL.

I guess we can close this issue now.

@jamtur01

This comment has been minimized.

Copy link
Contributor

commented Dec 26, 2013

Let's close it! :) Thanks @goldmann!

@jamtur01 jamtur01 closed this Dec 26, 2013

mrunalp pushed a commit to mrunalp/docker that referenced this issue Jul 20, 2016
seemethere pushed a commit to seemethere/moby that referenced this issue Feb 6, 2019
Merge pull request moby#172 from thaJeztah/18.03_backport_fix-dm-errmsg
[18.03-ee] backport gd/dm: fix error message
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.