set docker ulimit #2

Open
apachelogger opened this Issue Jul 7, 2015 · 4 comments

Comments

Projects
None yet
2 participants
Contributor

apachelogger commented Jul 7, 2015

random thing I remembered:

you must force the open fd limit to be 1024 in docker. otherwise apt and related helpers (such as apt-extracttemplates) IO is going to slow down installation of new packages by an enormous amount:
https://github.com/blue-systems/pangea-tooling/blob/master/lib/ci/container.rb#L86

in particular apt-extracttemplates, which should be installed after initial image creation, is turning into a slow pile of crap.

without ulimit installing the entire package tree in kubuntu ci takes almost 2 hours:
http://kci.pangea.pub/job/mgmt_daily_promotion_vivid_unstable/531/
with ulimit it takes some 30 minutes:
http://kci.pangea.pub/job/mgmt_daily_promotion_vivid_unstable/584/

in fact you can see it best in the logs when looking for 'Extracting templates from packages'. in the slow build this step took about 45 minutes, in the fast build it took like 2...

from what I can tell it slows down installation all around in all parts of apt, it just happens to be that batch template extraction is pretty much only opening and closing files thus being affected the most by the too high ulimit.

Contributor

aleixpol commented Jul 7, 2015

Are you sure this is necessary?

plasmamobile@b4b4912fca3c:~$ ulimit
unlimited
Contributor

apachelogger commented Jul 7, 2015

root@1b450fec01a5:/# ulimit -n
1048576

Contributor

aleixpol commented Jul 7, 2015

So the problem is that it's too big? return -ENOSENSE;

Contributor

apachelogger commented Jul 7, 2015

yes, I known, right!

I am not sure why you are surprised though, you have seen apt source code have you not? :P

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment