New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docker provider] boot2docker root fs is too small to be targeted by docker provider' rsync #3680

Closed
flevour opened this Issue May 7, 2014 · 16 comments

Comments

Projects
None yet
9 participants
@flevour

flevour commented May 7, 2014

In the transcript below I create a sample project with contains a big file which would get rsynced to boot2docker.
The rsync fails because it hasn't enough space to write in /.

[/tmp/vagrant-docker] cat > Vagrantfile
Vagrant.configure("2") do |config|
  config.vm.provider "docker" do |d|
    d.image = "foo/bar"
  end
end
[/tmp/vagrant-docker] ls -lah
total 1425272
drwxr-xr-x   4 flevour  wheel   136B May  7 17:14 .
drwxrwxrwt  12 root     wheel   408B May  7 17:08 ..
-rw-r--r--   1 flevour  wheel   106B May  7 17:14 Vagrantfile
-rw-r--r--   1 flevour  staff   696M Apr 16 09:24 pg.backup
[/tmp/vagrant-docker] vagrant up --provider=docker
Bringing machine 'default' up with 'docker' provider...
==> default: Docker host is required. One will be created if necessary...
    default: Docker host VM is already ready.
==> default: Syncing folders to the host VM...
    default: Rsyncing folder: /private/tmp/vagrant-docker/ => /mnt/docker_1399475692_6205
There was an error when attempting to rsync a synced folder.
Please inspect the error message below for more info.

Host path: /private/tmp/vagrant-docker/
Guest path: /mnt/docker_1399475692_6205
Command: rsync --verbose --archive --delete -z --no-owner --no-group --rsync-path sudo rsync -e ssh -p 2222 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i '/Users/flevour/.vagrant.d/insecure_private_key' --exclude .vagrant/ /private/tmp/vagrant-docker/ docker@127.0.0.1:/mnt/docker_1399475692_6205
Error: Warning: Permanently added '[127.0.0.1]:2222' (RSA) to the list of known hosts.
rsync: writefd_unbuffered failed to write 16385 bytes [sender]: Broken pipe (32)
rsync: write failed on "/mnt/docker_1399475692_6205/pg.backup": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(298) [receiver=3.0.0]
rsync: connection unexpectedly closed (58 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at /SourceCache/rsync/rsync-42/rsync/io.c(452) [sender=2.6.9]

[/tmp/vagrant-docker] ssh -p 2222 docker@127.0.0.1
docker@127.0.0.1's password: [tcuser]
boot2docker: 0.8.0
docker@boot2docker:~$ df -h
Filesystem                Size      Used Available Use% Mounted on
rootfs                  445.7M    393.7M     52.0M  88% /
tmpfs                   247.6M         0    247.6M   0% /dev/shm
/dev/sda                 38.3G     64.6M     36.3G   0% /mnt/sda
cgroup                  247.6M         0    247.6M   0% /sys/fs/cgroup
/dev/sda                 38.3G     64.6M     36.3G   0% /mnt/sda/var/lib/docker
/dev/loop0               28.0K     28.0K         0 100% /mnt/sda/tmp/tcloop/popt
/dev/loop1              180.0K    180.0K         0 100% /mnt/sda/tmp/tcloop/rsync
docker@boot2docker:~$ mount | grep docker
/dev/sda on /mnt/sda/var/lib/docker type ext4 (rw,relatime,data=ordered)
docker@boot2docker:~$ ls /mnt
docker_1399467553_93618/ docker_1399475538_28375/ docker_1399475692_6205/  sda/                     sr0/
docker@boot2docker:~$
@adamjt

This comment has been minimized.

adamjt commented May 7, 2014

+1 Just ran into this problem, there's a much larger partition at /mnt/sda/var/lib/docker

@adamjt

This comment has been minimized.

adamjt commented May 7, 2014

@flevour I found a nifty easter egg at https://github.com/mitchellh/vagrant/blob/f475df0987d4195b0433273b61109161185c4190/plugins/providers/docker/action/host_machine_sync_folders.rb#L112 that allows you to override the intermediate path. Here's my new config that uses the larger partition.

config.vm.define "app" do |app|
  app.vm.synced_folder ".", "/vagrant", disabled: true
  app.vm.synced_folder ".", "/var/lib/docker/where_i_really_want_my_stuff", docker__exact: true
  app.vm.provider "docker" do |d|
    d.image = "ubuntu:precise"
    d.cmd = ["ls", "/var/lib/docker/"]
  end
end

Since this is undocumented I assume it could disappear in some future release.

mitchellh added a commit that referenced this issue May 7, 2014

@mitchellh

This comment has been minimized.

Member

mitchellh commented May 7, 2014

Fixed, we just put data in /var/lib/docker now.

@mitchellh mitchellh closed this May 7, 2014

@pauljz

This comment has been minimized.

pauljz commented May 11, 2014

Looks like this might still be an issue in action/host_machine_build_dir.rb

https://github.com/mitchellh/vagrant/blob/master/plugins/providers/docker/action/host_machine_build_dir.rb#L31

@sheerun

This comment has been minimized.

sheerun commented May 15, 2014

I have this issues too. Can this be reopened? The configuration above doesn't work:

VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.define "app" do |app|
    app.vm.provider "docker" do |d|
      d.build_dir = "."
      d.has_ssh = true
    end
  end
end
vagrant up --provider=docker
Bringing machine 'app' up with 'docker' provider...
==> app: Docker host is required. One will be created if necessary...
    app: Docker host VM is already ready.
==> app: Syncing folders to the host VM...
    app: Rsyncing folder: /Users/sheerun/Source/wordnet/ => /var/lib/docker/wordnet
    app: Rsyncing folder: /Users/sheerun/Source/wordnet/ => /mnt/docker_build_f5888f1e122bd369618c9d6cae483b8a
There was an error when attempting to rsync a synced folder.
Please inspect the error message below for more info.

Host path: /Users/sheerun/Source/wordnet/
Guest path: /mnt/docker_build_f5888f1e122bd369618c9d6cae483b8a
Command: rsync --verbose --archive --delete -z --copy-links --no-owner --no-group --rsync-path sudo rsync -e ssh -p 2222 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i '/Users/sheerun/.vagrant.d/insecure_private_key' --exclude .vagrant/ /Users/sheerun/Source/wordnet/ docker@127.0.0.1:/mnt/docker_build_f5888f1e122bd369618c9d6cae483b8a
Error: Warning: Permanently added '[127.0.0.1]:2222' (RSA) to the list of known hosts.
rsync: writefd_unbuffered failed to write 16385 bytes [sender]: Broken pipe (32)
rsync: write failed on "/mnt/docker_build_f5888f1e122bd369618c9d6cae483b8a/log/development.log": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(298) [receiver=3.0.0]
rsync: connection unexpectedly closed (1816 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at /SourceCache/rsync/rsync-42/rsync/io.c(452) [sender=2.6.9]
@flevour

This comment has been minimized.

flevour commented May 15, 2014

It should be fixed in Vagrant 1.6.1. What vagrant version are you using?

@sheerun

This comment has been minimized.

sheerun commented May 15, 2014

vagrant -v
Vagrant 1.6.2
@flevour

This comment has been minimized.

flevour commented May 15, 2014

Ah, this must be exactly the issue @pauljz was mentioning above.
@mitchellh I agree the issue should be reopened.

@sheerun

This comment has been minimized.

sheerun commented May 15, 2014

It hangs on my development.log which has 893M.

I can remove it and it works, but still...

@Guria

This comment has been minimized.

Guria commented May 15, 2014

See #3771
"I've just replaced "/mnt" to "/var/lib/docker/" in host_machine_build_dir.rb:31 and it resolved an issue with not enough space to build. But we still have two syncs instead of one."

@sheerun

This comment has been minimized.

sheerun commented May 15, 2014

I confirm, there are always two syncs (OSX).

@potiuk

This comment has been minimized.

potiuk commented Jun 29, 2014

I have the same issue in Vagrant 1.6.3. Can you please reopen the issue?

Changing /mnt/ to /var/lib/docker seems to fix the issue, so maybe you should change it to be default behaviour. Boot2docker is so convenient to use on OSX

@albac

This comment has been minimized.

albac commented Sep 8, 2014

Same problem, also on vagrant 1.6.3, and it was also solved by changing new_build_dir replacing /mnt for /var/lib/docker, maybe there should be a variable for new_build_dir at least.

@schmunk42

This comment has been minimized.

schmunk42 commented Oct 23, 2014

I'd also suggest to re-open this issue, since it persists with 1.6.5 on OS X.

I tried to hack vagrant with:

new_build_dir = "/var/lib/docker/docker_#{Digest::MD5.hexdigest(build_dir)}"

and it starts building from the Dockerfile and triggering apt-get.

I get this error afterwards, but it seems to be another isssue...

Reading package lists...
E: Couldn't make mmap of 25165824 bytes - DynamicMMap (12: Cannot allocate memory)
E: The package lists or status file could not be parsed or opened.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment