[docker provider] boot2docker root fs is too small to be targeted by docker provider' rsync

[flevour]

In the transcript below I create a sample project with contains a big file which would get rsynced to boot2docker.
The rsync fails because it hasn't enough space to write in /.

[/tmp/vagrant-docker] cat > Vagrantfile
Vagrant.configure("2") do |config|
  config.vm.provider "docker" do |d|
    d.image = "foo/bar"
  end
end
[/tmp/vagrant-docker] ls -lah
total 1425272
drwxr-xr-x   4 flevour  wheel   136B May  7 17:14 .
drwxrwxrwt  12 root     wheel   408B May  7 17:08 ..
-rw-r--r--   1 flevour  wheel   106B May  7 17:14 Vagrantfile
-rw-r--r--   1 flevour  staff   696M Apr 16 09:24 pg.backup
[/tmp/vagrant-docker] vagrant up --provider=docker
Bringing machine 'default' up with 'docker' provider...
==> default: Docker host is required. One will be created if necessary...
    default: Docker host VM is already ready.
==> default: Syncing folders to the host VM...
    default: Rsyncing folder: /private/tmp/vagrant-docker/ => /mnt/docker_1399475692_6205
There was an error when attempting to rsync a synced folder.
Please inspect the error message below for more info.

Host path: /private/tmp/vagrant-docker/
Guest path: /mnt/docker_1399475692_6205
Command: rsync --verbose --archive --delete -z --no-owner --no-group --rsync-path sudo rsync -e ssh -p 2222 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i '/Users/flevour/.vagrant.d/insecure_private_key' --exclude .vagrant/ /private/tmp/vagrant-docker/ docker@127.0.0.1:/mnt/docker_1399475692_6205
Error: Warning: Permanently added '[127.0.0.1]:2222' (RSA) to the list of known hosts.
rsync: writefd_unbuffered failed to write 16385 bytes [sender]: Broken pipe (32)
rsync: write failed on "/mnt/docker_1399475692_6205/pg.backup": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(298) [receiver=3.0.0]
rsync: connection unexpectedly closed (58 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at /SourceCache/rsync/rsync-42/rsync/io.c(452) [sender=2.6.9]

[/tmp/vagrant-docker] ssh -p 2222 docker@127.0.0.1
docker@127.0.0.1's password: [tcuser]
boot2docker: 0.8.0
docker@boot2docker:~$ df -h
Filesystem                Size      Used Available Use% Mounted on
rootfs                  445.7M    393.7M     52.0M  88% /
tmpfs                   247.6M         0    247.6M   0% /dev/shm
/dev/sda                 38.3G     64.6M     36.3G   0% /mnt/sda
cgroup                  247.6M         0    247.6M   0% /sys/fs/cgroup
/dev/sda                 38.3G     64.6M     36.3G   0% /mnt/sda/var/lib/docker
/dev/loop0               28.0K     28.0K         0 100% /mnt/sda/tmp/tcloop/popt
/dev/loop1              180.0K    180.0K         0 100% /mnt/sda/tmp/tcloop/rsync
docker@boot2docker:~$ mount | grep docker
/dev/sda on /mnt/sda/var/lib/docker type ext4 (rw,relatime,data=ordered)
docker@boot2docker:~$ ls /mnt
docker_1399467553_93618/ docker_1399475538_28375/ docker_1399475692_6205/  sda/                     sr0/
docker@boot2docker:~$

[adamjt]

+1 Just ran into this problem, there's a much larger partition at /mnt/sda/var/lib/docker

[adamjt]

@flevour I found a nifty easter egg at https://github.com/mitchellh/vagrant/blob/f475df0987d4195b0433273b61109161185c4190/plugins/providers/docker/action/host_machine_sync_folders.rb#L112 that allows you to override the intermediate path. Here's my new config that uses the larger partition.

config.vm.define "app" do |app|
  app.vm.synced_folder ".", "/vagrant", disabled: true
  app.vm.synced_folder ".", "/var/lib/docker/where_i_really_want_my_stuff", docker__exact: true
  app.vm.provider "docker" do |d|
    d.image = "ubuntu:precise"
    d.cmd = ["ls", "/var/lib/docker/"]
  end
end

Since this is undocumented I assume it could disappear in some future release.

[mitchellh]

Fixed, we just put data in /var/lib/docker now.

[pauljz]

Looks like this might still be an issue in action/host_machine_build_dir.rb

https://github.com/mitchellh/vagrant/blob/master/plugins/providers/docker/action/host_machine_build_dir.rb#L31

[sheerun]

I have this issues too. Can this be reopened? The configuration above doesn't work:

VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.define "app" do |app|
    app.vm.provider "docker" do |d|
      d.build_dir = "."
      d.has_ssh = true
    end
  end
end

vagrant up --provider=docker
Bringing machine 'app' up with 'docker' provider...
==> app: Docker host is required. One will be created if necessary...
    app: Docker host VM is already ready.
==> app: Syncing folders to the host VM...
    app: Rsyncing folder: /Users/sheerun/Source/wordnet/ => /var/lib/docker/wordnet
    app: Rsyncing folder: /Users/sheerun/Source/wordnet/ => /mnt/docker_build_f5888f1e122bd369618c9d6cae483b8a
There was an error when attempting to rsync a synced folder.
Please inspect the error message below for more info.

Host path: /Users/sheerun/Source/wordnet/
Guest path: /mnt/docker_build_f5888f1e122bd369618c9d6cae483b8a
Command: rsync --verbose --archive --delete -z --copy-links --no-owner --no-group --rsync-path sudo rsync -e ssh -p 2222 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i '/Users/sheerun/.vagrant.d/insecure_private_key' --exclude .vagrant/ /Users/sheerun/Source/wordnet/ docker@127.0.0.1:/mnt/docker_build_f5888f1e122bd369618c9d6cae483b8a
Error: Warning: Permanently added '[127.0.0.1]:2222' (RSA) to the list of known hosts.
rsync: writefd_unbuffered failed to write 16385 bytes [sender]: Broken pipe (32)
rsync: write failed on "/mnt/docker_build_f5888f1e122bd369618c9d6cae483b8a/log/development.log": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(298) [receiver=3.0.0]
rsync: connection unexpectedly closed (1816 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at /SourceCache/rsync/rsync-42/rsync/io.c(452) [sender=2.6.9]

[flevour]

It should be fixed in Vagrant 1.6.1. What vagrant version are you using?

[sheerun]

vagrant -v
Vagrant 1.6.2

[flevour]

Ah, this must be exactly the issue @pauljz was mentioning above.
@mitchellh I agree the issue should be reopened.

[sheerun]

It hangs on my development.log which has 893M.

I can remove it and it works, but still...

[Guria]

See #3771
"I've just replaced "/mnt" to "/var/lib/docker/" in host_machine_build_dir.rb:31 and it resolved an issue with not enough space to build. But we still have two syncs instead of one."

[sheerun]

I confirm, there are always two syncs (OSX).

[potiuk]

I have the same issue in Vagrant 1.6.3. Can you please reopen the issue?

Changing /mnt/ to /var/lib/docker seems to fix the issue, so maybe you should change it to be default behaviour. Boot2docker is so convenient to use on OSX

[albac]

Same problem, also on vagrant 1.6.3, and it was also solved by changing new_build_dir replacing /mnt for /var/lib/docker, maybe there should be a variable for new_build_dir at least.

[schmunk42]

I'd also suggest to re-open this issue, since it persists with 1.6.5 on OS X.

I tried to hack vagrant with:

new_build_dir = "/var/lib/docker/docker_#{Digest::MD5.hexdigest(build_dir)}"

and it starts building from the Dockerfile and triggering apt-get.

I get this error afterwards, but it seems to be another isssue...

Reading package lists...
E: Couldn't make mmap of 25165824 bytes - DynamicMMap (12: Cannot allocate memory)
E: The package lists or status file could not be parsed or opened.