Permission denied

[purpleidea]

Occasionally when running provision (usually an initial provision has already happened)

[machine1] Rsyncing folder: /home/user/vagrant/foo/ => /vagrant
[machine1] Configuring cache buckets...
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

mkdir -p /tmp/vagrant-cache/yum

Stdout from the command:

Stderr from the command:

mkdir: cannot create directory `/tmp/vagrant-cache': Permission denied

This is using vagrant on F19, with vagrant-libvirt.
Setup described here:
https://ttboj.wordpress.com/2013/12/09/vagrant-on-fedora-with-libvirt/
and here:
https://ttboj.wordpress.com/2013/12/21/vagrant-vsftp-and-other-tricks/

Cheers

[purpleidea]

More info:

Restart nfs-server, provision still fails.
SSH in:

[root@machine1 tmp]# ls -lAh
ls: cannot access vagrant-cache: Permission denied
total 32K
-rw-r--r-- 1 root root 22K Dec 18 00:48 builder.log
drwxrwxrwt 2 root root 4.0K Dec 18 02:57 .ICE-unix
d????????? ? ? ? ? ? vagrant-cache
-rwx--x--x 1 vagrant vagrant 91 Dec 18 02:57 vagrant-shell
[root@annex1 tmp]#

Looks like NFS is borked...

[purpleidea]

[root@machine1 tmp]# mount | grep nfs
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
192.168.142.1:/home/user/vagrant/foo/.vagrant/machines/machine1/cache on /tmp/vagrant-cache type nfs (rw,vers=3,tcp,nolock,addr=192.168.142.1)

[purpleidea]

Workaround and more info (BUT ANNOYING!)

[root@machine1 tmp]# mount | grep nfs
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
192.168.142.1:/home/user/vagrant/foo/.vagrant/machines/machine1/cache on /tmp/vagrant-cache type nfs (rw,vers=3,tcp,nolock,addr=192.168.142.1)
[root@machine1 tmp]# umount /tmp/vagrant-cache 
^C
[root@machine1 tmp]# umount -l /tmp/vagrant-cache
umount.nfs: /tmp/vagrant-cache: not mounted
umount.nfs: /tmp/vagrant-cache: not mounted
[root@machine1 tmp]# mount | grep nfs
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
[root@machine1 tmp]# exit
[vagrant@machine1 ~]$ logout
Connection to 192.168.142.130 closed.
user@host:~/vagrant/foo$ vp machine1
[machine1] Rsyncing folder: /home/user/vagrant/foo/ => /vagrant
[machine1] Configuring cache buckets...
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

mkdir -p /tmp/vagrant-cache/yum

Stdout from the command:



Stderr from the command:

mkdir: cannot create directory `/tmp/vagrant-cache/yum': Permission denied

user@host:~/vagrant/foo$ vssh machine1
This is Vagrant+Puppet-foo! (https://ttboj.wordpress.com/)
[vagrant@machine1 ~]$ sudo -s
[root@machine1 vagrant]# cd /tmp/
[root@machine1 tmp]# ls
builder.log  vagrant-cache  vagrant-shell
[root@machine1 tmp]# ls -lAh
total 36K
-rw-r--r-- 1 root    root     22K Dec 18 00:48 builder.log
drwxrwxrwt 2 root    root    4.0K Dec 18 02:57 .ICE-unix
drwxr-xr-x 2 root    root    4.0K Dec 18 02:57 vagrant-cache
-rwx--x--x 1 vagrant vagrant   91 Dec 18 02:57 vagrant-shell
[root@machine1 tmp]# cd vagrant-cache/
[root@machine1 vagrant-cache]# ls
[root@machine1 vagrant-cache]# ls -lAh
total 0
[root@machine1 vagrant-cache]# cd ..
[root@machine1 tmp]# rmdir vagrant-cache/
[root@machine1 tmp]# exit
[vagrant@machine1 ~]$ logout
Connection to 192.168.142.130 closed.
user@host:~/vagrant/foo$ vp machine1

# WORKS

[tmatilai]

@purpleidea sorry, I don't think I'm gonna find time soon to set up environment to repro this. But a link to full --debug log would probably be useful.

My gut feeling is that this is libvirt/provider issue. The NFS mount is broken before vagrant-cachier even kicks in. I've seen that happening when the directory is deleted on the host, but that shouldn't happen here.

[purpleidea]

Here is a workaround 'fix' script that people can use in the meantime... It's obviously a hack. A real fix would be awesome...

https://gist.github.com/purpleidea/8211614

Cheers

[scashin133]

@tmatilai I just got an error similar to this one although for a different path when using the virtual box provider. I'm running Vagrant 1.4.1 and vagrant-cachier version 0.5.1.

Log: https://gist.github.com/scashin133/80fa63a274c9139da893

I'll try to get a debug log

[scashin133]

Here is the debug output: https://gist.github.com/scashin133/22a1affd97a88ea06bae

[fgrehm]

@scashin133 @purpleidea Folks, is this still an issue for you guys? Could one of you please gist the exact same Vagrantfile you guys are using over there? Thanks

[fgrehm]

Well, I think I just got bitten by this with the vbox provider and I worked around it by changing the cache scope to :machine. I still have no idea what exactly is going on but it seems to be related to folder encryption in my case...

[gionn]

I've the same problem using LXC driver:

mkdir -p /tmp/vagrant-cache/apt/partial
Stdout from the command:
Stderr from the command:
mkdir: cannot create directory `/tmp/vagrant-cache/apt': Permission denied

vagrant@php-postgres:/tmp$ ls -la /tmp/vagrant-cache/ 
total 8
drwxr-xr-x 2  104  108 4096 Feb  6 13:01 .
drwxrwxrwt 4 root root 4096 Feb  6 14:50 ..

Owner and group are the same of the user running vagrant on the host machine (is a know limitation with LXC, we cannot change uid:gid as Virtualbox do on shared folders).
I suppose that the folder creation is made via the vagrant user inside the box, can't the folder be chmod 777 or simply use sudo to create that folder?

[oker1]

Same here with lxc, on the host the cache dir has to be 777 or chowned to 1000:1000.

[fgrehm]

Ok, folks, I'll try to get the chmod in place for the next release.

[fgrehm]

Guys, there's a potential fix for it on 6282b09, if someone wants to give it a go before the new release please build the plugin from sources and let us know how it goes :-)

[inancgumus]

@fgrehm I have kind of the same problem I guess. I have version vagrant-cachier 1.2.0

Standard output
Debug output

When I run: vagrant provision --provision-with test I got these errors. Provision script is in bash and empty just for testing. Before these error I had installed some npm packages via another bash provision script.

When I ssh'ed into the machine and typed sudo chown -Rf vagrant:vagrant ~/.npm the error disappears.

[fgrehm]

Sorry but I won't be able to help you ATM =/

Do you mind opening up another issue so that other people can chime in? I'd reopen this one but it is way too big already