Join GitHub today
GitHub is home to over 20 million developers working together to host and review code, manage projects, and build software together.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
Already on GitHub? Sign in to your account
Running docker on RHEL #172
Comments
|
You might want to try this kernel: Warning: the package is 700MB! I'm sorry, that's what I got when running This kernel features:
I don't have any RHEL/Fedora/CentOS machine around, so I can't test it, but it's compiled with exactly the same |
|
cc @brianm |
|
Anyone else interested in Red Hat support? |
bfulton
commented
Mar 27, 2013
|
+1 |
scrothers
commented
Mar 27, 2013
|
I'm interested in building a proper AUFS RPM against the existing sources, honestly I think having it released to EPEL or something would be best. Future maintanability and development ecosystem and so on. I'll start researching what this effort will take and open a case and link it to this one. |
|
+1 Sent from my iPhone On Mar 27, 2013, at 5:51 PM, Solomon Hykes notifications@github.com wrote:
|
ryanwmarsh
commented
Mar 28, 2013
|
+1 |
josephglanville
commented
Mar 28, 2013
|
Ideally a DKMS RPM should be built. I am pretty sure RHEL 6 uses DKMS and it will keep AUFS working across kernel upgrades. |
|
I could be wrong, but it seems that AUFS doesn't support building through On Wed, Mar 27, 2013 at 11:51 PM, Joseph Glanville <notifications@github.com
|
josephglanville
commented
Mar 28, 2013
|
DKMS can build any module. It's the same as doing make, make install in the module source directory. (or doing make modules; make modules_install in the kernel source tree). The only thing that DKMS does for you is automatically rebuild the module against the newly installed kernel so that it has the correct symbols. |
|
It's worth trying, but even when it's built as a module, AUFS patches the |
josephglanville
commented
Mar 28, 2013
|
Ahh yes you are correct, my mistake. :( Shame about that, I hope one day AUFS becomes mainline so this doesn't happen sigh. |
timbunce
commented
Mar 28, 2013
|
+1 for RHEL/CentOS |
scrothers
commented
Mar 28, 2013
|
It appears that DKMS will not work for this, as we cannot logically separate the components of AUFS from the Kernel image 100%. It still requires the Kernel image be patched with specific AUFS code. The only realistic way to include this is to patch it directly into the EL6 upstream Kernel and release that code as buildable. Which is what I'm going to pursue right now. Once it's complete I'll be releasing it on my Github, however a longer term goal may be to operate a specific Kernel for Docker anyway. Include all the new fancy cgroup utils and other filesystems. Thoughts? |
|
If you can point me to the EL6 source tree, I can add it to our kernel build farm (with some luck, our AUFS patch queue will apply neatly; or at least, require only minor work to merge). FWIW, I recompiled our production kernels without debugging symbols (so now the RPM is just 50MB instead of 700MB): http://get.docker.io/kernels/kernel-3.2.40_grsec_dotcloud-4.x86_64.rpm If someone could give it a try, that would be awesome. |
scrothers
commented
Mar 29, 2013
|
Depends, are you using AUFS 3.x? If so the RHEL Kernel won't work, since it's a 2.6.32 Kernel. It'll be missing most of the API/ABI that I'm assuming AUFS 3 requires since it specifically states Kernel 3.0 requirement. However though, as requested, the current SRPM: Obviously you can unpack the source using rpm, in the case you're on debian or osx (with brew) or something... rpm2cpio kernel-blablabla.rpm | cpio -idmv Also, going to give that Kernel a try now on an EL6 machine. |
scrothers
commented
Mar 29, 2013
|
Yea, there's a lot of depsolving and conflicts in this Kernel you have posted... Where is the srpm at? I'll be happy to work the spec out. Ideally, this Kernel wouldn't upgrade or replace the existing RHEL Kernel (so it'll remain updated) it should be supplemental to the RHEL Kernel and updated separately. Errors here: I'm going to open a case for this so it can be properly tracked. |
scrothers
commented
Mar 29, 2013
|
For the record, new case is here to track the specific Kernel changes: This case should be made dependent on 253, not sure if that's possible on Github. |
stahnma
commented
May 2, 2013
|
It seems like targeting Fedora 18/19 (and thus what will likely become EL7) would be easier/better than working on the 2.6 kernels in EL6. |
|
FYI Ubuntu is discontinuing support for aufs in their 3.9 kernels, so we On Wed, May 1, 2013 at 10:47 PM, Michael Stahnke
|
pdf
commented
May 15, 2013
|
OverlayFS is attempting (again) acceptance for merge into mainline 3.10, but it still seems unlikely at this point, since Al Viro (the VFS maintainer) doesn't like some chunks of the design (primarily use of xattrs for opaques), which seems to be causing a little friction since Miklos is somewhat attached to the idea for portability (FWIW, I kind of agree with Miklos here). That said, it's already included in the standard kernel patchsets for at least Ubuntu and SuSE, maybe others. and as @shykes points out, aufs is soon to be dropped for Ubuntu so some decision here will have to be made. |
|
I'm actively working on btrfs support for docker [1]. |
dmacvicar
commented
May 18, 2013
|
I just hacked support for overlayfs, which is is already on the SUSE and Ubuntu kernels, and probably the only one that will make it to upstream. I can't get the tests to pass because an unmount fails in the test. However I can run the container and docker diff seems to be working. //cc @flavio @silviomoioli |
|
@dmacvicar Did you change FILESYSTEM_MAX_STACK_DEPTH in your kernel to allow overlayfs to stack more than 2 layers? edit: just noticed this limit may not apply on opensuse if they don't have that check in their version of overlayfs. On ubuntu FILESYSTEM_MAX_STACK_DEPTH is 2 so mounting a stack a b c d will fail when you go to mount d. |
dmacvicar
commented
May 28, 2013
|
@kstaken Mmmm... I only tried with 3 layers so you may be right base + 1 + 1 is still in the max range. I need to check whether our kernel patches also have that limit. |
dmacvicar
commented
May 28, 2013
|
@kstaken you are completely right May 28 10:21:47 piscola kernel: overlayfs: maximum fs stacking depth exceeded I oversaw that. Documentation says "The lower filesystem can even be another overlayfs", but did not mention any limits. ARGH. I would guess for docker the main usecase would be to have an image and keep the changes of the image separate, and then commit them to another image. Are more layers at the same time useful? |
dmacvicar
commented
May 28, 2013
|
@kstaken I talked to Miklos (overlayfs author) as he work for us, and he confirmed that there is a limit to avoid a kernel stack overflow. He may look into adding arbitrary layers in the future, may be after it is included in the mainline kernel. |
|
I'm new to docker but I believe more layers at the same time is core to the way docker is intended to work. The base image works but using a random image from the registry hits the limit. docker run mbkan/lamp ps aux in syslog. May 28 17:04:17 precise64 kernel: [111971.681350] overlayfs: maximum fs stacking depth exceeded Is it pretty likely overlayfs is going to make it into the mainline kernel? |
abohne
commented
Jun 6, 2013
|
+1 for EL support. I think targeting EL7 may be a more worthwhile effort than trying to move heaven and earth to get it working on EL6. |
nehaldattani
commented
Jun 23, 2013
|
In case someone want to try, I have documented the steps for fedora 18 at http://neh.al/?p=1 |
NuxRo
commented
Jul 5, 2013
|
+1 for EL6 support. Also if you need alternative kernels you can contact elrepo.org who specialises in alt kernels and extra drivers for RHEL and clones. +1 also for using a properly supported filesystem such as btrfs. |
beders
commented
Jul 9, 2013
|
+1 for Redhat/CentOS support |
sontek
commented
Jul 13, 2013
|
+1 For Redhat support |
Mistobaan
commented
Jul 18, 2013
|
+1 for RH / Centos 6.4 support |
jdoss
commented
Jul 18, 2013
|
+1 for RH / Centos 6 support |
mariocar
commented
Jul 19, 2013
|
+1 for RHEL/CentOS 6+ support |
rickysaltzer
commented
Jul 22, 2013
|
+1 For RHEL/Centos 6.x support |
|
++ for Centos6/RHEL6 support |
gbraad
commented
Jul 26, 2013
|
+1 |
failshell
commented
Jul 26, 2013
|
+1 |
perdurabo93
commented
Jul 29, 2013
|
Absolutely +1. Docker isn't all that useful when the majority of commercial development happens on RHEL/Centos/Fedora/Oracle Linux based distros. |
devinus
commented
Jul 30, 2013
|
|
|
+1 otherwise our operations guys will block us from working with Docker. Need a clear upgrade path to RHEL7 and preferably on RHEL6. |
velniukas
commented
Jul 31, 2013
|
+1 this is probably my highest item on wishlist.. several very large On Wed, Jul 31, 2013 at 9:53 PM, Thijs Terlouw notifications@github.comwrote:
|
hura
commented
Jul 31, 2013
|
+1 |
cyberoblivion
commented
Aug 2, 2013
|
+1 For Redhat support |
|
+1 |
hanleybrand
commented
Aug 2, 2013
|
+1 for RHEL 6 - that's my employer's approved linux server distro - ubuntu & others are non starters in our data center. |
blalor
commented
Aug 3, 2013
|
+1 for RHEL/CentOS. |
voltagex
commented
Aug 5, 2013
|
Can people please let me know if using a newer kernel from elRepo is not an option for them? |
velniukas
commented
Aug 5, 2013
|
All the banks i consult at on rhel 5.6-8 just starting to move over to 6.x Sent from my iPhone On 5 Aug, 2013, at 9:34, Adam Baxter notifications@github.com wrote:
|
mhitza
commented
Aug 5, 2013
|
+1 Fedora |
gregretkowski
commented
Aug 5, 2013
|
I just did a write-up on how to install docker on CentOS / RHEL 6. The kernel Jérôme posted works great, you just need to remove the kernel-firmware package first. http://blog.rage.net/2013/08/04/installing-docker-on-centos-6/ Jérôme, any chance of publishing a newer kernel RPM? All the docker docs say we should be using 3.8. I'll update my blog post with the newer kernel if one comes available. |
voltagex
commented
Aug 5, 2013
|
I think the issue is big organisations won't want to shift kernel versions. |
gbraad
commented
Aug 5, 2013
|
Ideally no alternative kernels should be used: people use Red Hat and buy a support contract for a reason. Installing an alternative, non-supported kernel, undermines the idea of running such a distribution. And please, do not compare Fedora and RHEL (CentOS, Scientific Linux, etc) to be the same. For Fedora the situation is different (more experimental in nature and not often deployed in a production environment) |
velniukas
commented
Aug 5, 2013
|
Agreed. Will need to work with whatever we can in this case. Sent from my iPhone On 5 Aug, 2013, at 12:09, Gerard Braad — 吉拉德 notifications@github.com wrote:
|
NuxRo
commented
Aug 6, 2013
|
What's with the grsec dep? |
|
I build our kernels with GRSEC. |
sciurus
commented
Aug 8, 2013
|
Has anyone evaluated using LVM's thinly-provisioned snapshot volumes for docker? |
solomonstre
commented
Aug 8, 2013
|
Yes :) I think LVM is just as viable as BTRFS as a storage backend. I would like to support them both with the plugin API. On Thu, Aug 8, 2013 at 12:55 PM, Brian Pitts notifications@github.com
|
sciurus
commented
Aug 13, 2013
|
I have a spec file for linux 3.10 with aufs at sciurus/docker-rhel-rpm. I haven't had much time to play with it yet, but this kernel and the lxc userspace (spec file also included) seem to get docker working on CentOS 6. I thought I'd also need aufs-utils, but I didn't package them yet and docker didn't complain that they were missing. Please take this for a spin and let me know if you see any problems. Even better, fix them and send me a pull request! If it would be useful, I can try to build and distribute binary RPMs via the OpenSUSE Build Service. |
pdf
commented
Aug 13, 2013
LVM snapshots degrade pretty horribly with depth, because unlike COW-based snapshots, references to changed data must be updated all the way up the snapshot stack on each change. |
sciurus
commented
Aug 13, 2013
|
@pdf I think that's true for traditional LVM snapshots, but not for thinly-provisioned ones. |
pdf
commented
Aug 13, 2013
|
@sciurus looks like you're right (from your link):
Sorry for the noise. I don't know how well thin LVM volumes are tested for ENOSPC though. |
solomonstre
commented
Aug 13, 2013
|
In any case performance concerns over CoW systems (which all pay some sort of performance penalty, however small, including aufs) are usually toned down by judicious use of bind-mounted volumes for performance-sensitive parts of the filesystem. On Mon, Aug 12, 2013 at 10:54 PM, Peter Fern notifications@github.com
|
mahmoudimus
commented
Aug 15, 2013
|
+1 |
sciurus
commented
Aug 16, 2013
|
I've added a package of docker itself at sciurus/docker-rhel-rpm. It's extremely simple as I use the precompiled docker binary and reuse the upstart job from Ubuntu. I've started playing with OBS here but am running into problems. |
|
As I'm hamstrung by EL5, I'm currently messing around with chroot and unionfs as a way to get docker-like functionality when you 100% trust the code you're trying to run and this is the only way of doing it. UnionFS is working perfectly for me. So why hasn't it been mentioned in the conversation so far? |
|
@boffbowsh , excellent news. Question (since I'm absolutely not familiar with UnionFS): does it support multiple layers? Or just one? |
|
Yep, multiple layers. Example from my ghetto heroku clone: unionfs -ocow $OVERLAY_DIR=RW:$SLUG_DIR=RO:$BASE_DIR=RO $MOUNT_DIR |
sciurus
commented
Aug 19, 2013
|
@boffbowsh does the stock EL5 kernel include the UnionFS patches? I can't find the unionfs module or userspace tools on a sample RHEL5 system. |
|
It's based on FUSE, right? On Mon, Aug 19, 2013 at 3:15 PM, Brian Pitts notifications@github.comwrote:
|
sciurus
commented
Aug 20, 2013
|
@jpetazzo No, it doesn't use FUSE. Aufs actually began as a fork of it; see https://lwn.net/Articles/327738/ |
|
The instructions I followed to get it up and running use FUSE. I figure UnionFS is "safe" as it seems like the majority of LiveCDs use it. |
nareshv
commented
Aug 21, 2013
|
I have created a centos-6.x repository for docker.io with aufs. Thanks to @sciurus for the kernel spec file. |
NuxRo
commented
Aug 21, 2013
|
Nareshv, that's great, thanks for your effort! |
failshell
commented
Aug 22, 2013
|
Thanks for your kernel RPM. Awesome work. I've been testing it on RHEL 6.4 and it works like a charm. Also, for those interested, I published a Chef cookbook that takes care of installing and configuring Docker. |
Silberling
commented
Aug 22, 2013
|
+1 |
chenryn
commented
Aug 24, 2013
|
@nehaldattani I try to do the same thing on Fedora19 but when I compile kernel-3.10 with aufs3 patch(3.10 branch too), it failed. Are you try this on fc19? |
nehaldattani
commented
Aug 26, 2013
|
@chenryn : At what point it failed ? Can you load aufs module ? |
chenryn
commented
Aug 26, 2013
|
@nehaldattani |
nehaldattani
commented
Aug 26, 2013
|
@chenryn : Are you using code from kernel.org or the code has been extracted from SRPM ? |
chenryn
commented
Aug 26, 2013
|
@nehaldattani I had both tried the kernel code from git branch 3.10, and from kernel-3.10.7-200.fc19.src.rpm. Got the same result |
nehaldattani
commented
Aug 26, 2013
|
@chenryn : I have created a new F19 vm and applied the aufs patches on kernel 3.10 ( code from kernel.org) and just completed building of kernel rpm with aufs module. In your case, it seems that all aufs header files are not included properly. I also found a typo in my post. ( Though you will get a message "cp:ommiting directory .....) Read "cp include/uapi/linux/ ../linux-2.6/include/uapi/linux/" as "cp include/uapi/linux/aufs_type.h ../linux-2.6/include/uapi/linux/" Please copy header files as mentioned in my post ( exception is above mentioned file) and give it a try. Good Luck !!! |
sciurus
commented
Aug 26, 2013
|
@chenryn have you looked at sciurus/docker-rhel-rpm? If your goal is to build a 3.10 kernel with aufs support, you can use my steps there. I've added some more information about this to the README. |
jlk
commented
Sep 3, 2013
|
+1 - Would love to see solid EL6 support. Curious to see where this goes... |
mztriz
commented
Sep 17, 2013
|
+1 I would like to see some RHEL support as well. |
gamefiend
commented
Sep 17, 2013
|
+1, yes please. |
vannenc
commented
Sep 19, 2013
|
+1, for CentOS/RedHat |
|
That is awesome news. |
jefferai
commented
Sep 25, 2013
|
Is there any information about where this plugin API is being designed? I don't see any relevant branches in the Github repository. I'm very interested in the possibility of a ZoL plugin as an alternative to btrfs and thinly-provisioned device-mapper, but if a btrfs plugin is already in development, it would be nice to be able to see that code to see what it would look like to make a ZoL version. |
doublerebel
commented
Sep 28, 2013
|
|
pungoyal
commented
Oct 10, 2013
|
+1 for Red Hat support |
leifwalsh
commented
Oct 22, 2013
|
+1 centos 6, and 5 if you can get it |
|
A blog post describing the details about RHEL/Fedora support for Docker: http://community.redhat.com/adventures-in-dockerland/ |
sciurus
commented
Nov 26, 2013
|
Now that
I think this issue can be closed. |
|
It's packaged, but not pushed to the repos yet. Please hold on, we'll be there soon! |
blalor
commented
Nov 26, 2013
|
Will Docker 0.7 work on CentOS 6? I think that's still on kernel 2.6, but the Docker docks say 3.8 (?) is required. |
NuxRo
commented
Nov 26, 2013
|
It works nicely: |
blalor
commented
Nov 26, 2013
|
Excellent. yum resolutely refused to find docker-io in epel-testing, but I used “yum localinstall” with the URL to the package IN THE EPEL-TESTING REPO and it works a treat. Yum even found lxc and lxc-libs in epel-testing, just not docker-io. I just started the centos container on CentOS. MY GOD, IT’S FULL OF STARS! |
blalor
commented
Nov 27, 2013
|
Er, not quite. The docker-io service in epel-testing starts with
|
NuxRo
commented
Nov 27, 2013
|
Right, so you either need to create a bridge (docker0) by yourself and specify it in /etc/sysconfig/docker or go for a newer kernel. Kernel-lt from Elrepo works fine at a first glance. |
NuxRo
commented
Nov 27, 2013
|
Until docker-io makes a comeback in epel-testing it can be downloaded from http://koji.fedoraproject.org/koji/packageinfo?packageID=17053 |
blalor
commented
Nov 27, 2013
|
I'm working on a sample Vagrant box and bootstrapping script for CentOS. https://github.com/blalor/vagrant-centos64-docker It's not working yet, but there has been some good discussion on docker-dev today. You don't really need an RPM as you can just grab the docker binary. My Vagrant config still doesn't work (lxc incompatibility?) but I hope to get back to it later tonight to get it working. |
|
Just to follow up with this thread, the bridging and networking on CentOS 6 should be fixed along with the libcgroup/cgconfig fixes in docker-io packages as of last few days. According to Red Hat Bugzilla 1000662 and Fedora build system, the latest docker-io package (docker-io-0.7.0-14) is being pushed to EPEL 6 stable right now. Fedora build system is showing FC19/FC20 being pushed to updates stable repositories shortly as well. Great work everyone! |
blalor
commented
Dec 3, 2013
|
Great news!! Last I looked the |
|
All handled without effort with latest package, at least from what I can tell. :)
|
|
@blalor It's fixed in the latest update. All-- From now on please report all issues regarding I'll update the docs with instruction on how to install Docker on CentOS/RHEL. I guess we can close this issue now. |
goldmann
referenced this issue
Dec 3, 2013
Merged
Instructions on how to install Docker on Red Hat Enterprise Linux / CentOS #3011
|
Let's close it! :) Thanks @goldmann! |
davidreuss commentedMar 26, 2013
This issue is meant for tracking dockers status on RHEL (Red Hat Enterprise Linux)
The one main pain-point i believe is AUFS, which isn't directly available on RHEL.
I haven't myself had as much time to play around with it, but i wanted a central location for storing progress/feedback from people trying to get docker running.