1.3.0 - Only root can write to OSX volumes / Can't change permissions within #581

Open
lynxnathan opened this Issue Oct 20, 2014 · 226 comments

Projects

None yet
@lynxnathan

Currently only root inside the container is able to modify mounted volumes. Also permissions changed from within have no effect. I'm unsure as to whether cannot be managed dynamically on the blog post refers to any of this.

I would suggest updating the readme to let users know about the existence of native support for volumes and also its limitations (especially if this is one of them).

@SvenDowideit SvenDowideit added this to the 1.3.1 milestone Oct 20, 2014
@mnapoli
mnapoli commented Oct 20, 2014

I think I'm having the same issue (latest versions of everything, installed today).

In the container, in a directory shared with the host (i.e. mounted through the virtualbox VM):

  • root user can write files
  • www-data user can't, even in a directory with 777 permissions

Is that normal or not? (i.e. I did something wrong)

@lukaswelte

So means if I create a cache folder for example that writes new files and folders to disk during execution, I cannot do this right now?

Cause I experience the same problem as @mnapoli does..

@bradgessler

I wrote up a simple test case. Let me know if there's anything else I can do to make this bug easier to fix.

First, I create a Dockerfile:

FROM ubuntu:14.04

# Adding user
RUN adduser --system --ingroup staff mugzy

# Setup repos dir.
RUN mkdir /data
RUN chown mugzy:staff -R /data
VOLUME /data

CMD touch /data/file && ls -al /data

Then create a host volume that I'll mount into the Docker instance at runtime:

app[master] → mkdir ./data

When I run the container, I see root:root ownership on the docker /data volume. I'd expect for this to be what I set from the Dockerfile above, mugzy:staff:

app[master] → docker run -i -v $PWD/data:/data c638bc43bfe7
total 4
drwxr-xr-x  1 1000 staff  102 Oct 21  2014 .
drwxr-xr-x 62 root root  4096 Oct 21 01:54 ..
-rw-r--r--  1 1000 staff    0 Oct 21 01:49 file

Obviously when I then run the docker instance as mugzy, the volume has incorrect permissions and I can't write to disk:

app[master] → docker run -i -v $PWD/data:/data -u mugzy c638bc43bfe7
total 4
drwxr-xr-x  1 1000 staff  102 Oct 21  2014 .
drwxr-xr-x 62 root root  4096 Oct 21 01:54 ..
-rw-r--r--  1 1000 staff    0 Oct 21 01:49 file
touch: cannot touch '/data/file': Permission denied

I'd imagine if people are trying to boot services in Docker on Mac, they'd want to be able to setup a volume that a service user can write to for persistence.

@vmaatta
vmaatta commented Oct 22, 2014

Mounting a VBox share as UID 999 will allow the created container user to write as it's usually 999. This is just a fragile hack on the hack but it does allow writing…

sudo mount -t vboxsf -o uid=999,gid=50 your-other-share-name /some/mount/location
@lukaswelte

So can I use this command to set the /Users share to be writeable?
Or do I have to create a special share for every application that needs to be able to write in a folder?

@vmaatta
vmaatta commented Oct 22, 2014

/Users is a bit special as it's mounted by the boot2docker script (any share matching the names listed in release notes), changing it requires customising the code. I just created a custom share anyway as I don't want to share the whole /Users structure with containers.

  1. Overriding the /Users default share on boot2docker start:
boot2docker --vbox-share=$(pwd)/share-location=share-name up
  1. boot2docker ssh in and mount the custom share:
sudo mount -t vboxsf -o uid=999,gid=50 share-name [SHARE-FULL-PATH]/share-location
@lukaswelte

Thanks for the workaround.
Although it would be great to have that behavior built into the boot2docker script (don’t want coworkers to do much extra action).

@SvenDowideit
Member
@tianon
Contributor
tianon commented Oct 23, 2014

This is bizarre - do we need to tweak something about the way we mount the share?

There's nothing special about what we do: https://github.com/boot2docker/boot2docker/blob/master/rootfs/rootfs/etc/rc.d/automount-shares (just uid=...,gid=... as mentioned here)

@tianon tianon modified the milestone: 1.3.1 Oct 23, 2014
@tianon
Contributor
tianon commented Oct 23, 2014

Indeed, I've just tested here:

docker@boot2docker:~$ ls -lAF /
...
drwxr-xr-x    1 docker   staff        12288 Oct 23 16:37 Users/
...
docker@boot2docker:~$ echo test > /Users/test
docker@boot2docker:~$ cat /Users/test
test
docker@boot2docker:~$ rm -v /Users/test
rm: remove '/Users/test'? y
docker@boot2docker:~$

So I definitely need more information about what's going wrong if we're going to figure out how to fix it. 😄

@mnapoli
mnapoli commented Oct 23, 2014

@tianon Now try with a different user and it will not work.

The big problem is that usually, Nginx or MySQL or whatever will run as a different user (www-data, mysql…) so it's impossible to use Docker at all (at least in those situations).

@tianon
Contributor
tianon commented Oct 23, 2014

Right, but how can we fix it?

Permissions with volume mounts are one of those "gotchas" that's always a sticky point, so I don't see a good fix for the general case (besides Docker handling the volume sharing more directly, and thus smoothing permissions issues somehow), especially since once you've got it working for "just this one container", you're going to want to spin up another, and it will probably have different UIDs altogether (think mysql + postgres + wordpress + stuff specific to your own development, etc).

@vmaatta
vmaatta commented Oct 23, 2014

@tianon above in your example you're running as the docker user in the VM. It'll work just fine there but the container and added users are a different story.

  • Outside the container, in the VM, the vboxsf is mounted as UID=1000, GID=50, i.e. docker:staff
  • Inside a running container that UID is not in use by default, but if you'll just useradd new user, 1000 will be the first ID. Ok so far.
  • If you add both a new group and a new user to the freshly created group, inside the container, you'll likely end up with UID / GID 999:999… -> Quite a few containers in the Hub do just that. They add a group and a user for the process to run as. Here's an example from the postgres container* RUN groupadd -r postgres && useradd -r -g postgres postgres. That'll end up with 999:999

Here's some testing I did with my custom uid=999,gid=50 vboxsf:

Test "postgres" container

root@9aa1b6f15b1c:/# groupadd -r postgres && useradd -r -g postgres postgres
root@9aa1b6f15b1c:/# cat /etc/group | grep post
postgres:x:999:

root@9aa1b6f15b1c:/tmp/test# mkdir root-dir
root@9aa1b6f15b1c:/tmp/test# su -c 'mkdir user-dir' postgres
root@9aa1b6f15b1c:/tmp/test# su -c 'touch user-file' postgres
root@9aa1b6f15b1c:/tmp/test# su -c 'echo "test" > user-file' postgres
root@9aa1b6f15b1c:/tmp/test# su -c 'cat user-file' postgres
test
root@9aa1b6f15b1c:/tmp/test# su -c 'ln -s user-file user-link' postgres
root@9aa1b6f15b1c:/tmp/test# su -c 'ln user-file user-hard-link' postgres
ln: failed to create hard link `user-hard-link' => `user-file': Operation not permitted

New container

root@7a795aa575df:/# useradd test-user
root@7a795aa575df:/# cat /etc/passwd
…
test-user:x:1000:1000::/home/test-user:/bin/sh
root@7a795aa575df:/tmp/test# su - test-user
No directory, logging in with HOME=/
$ cd /tmp/test
$ touch test-user-test-file
touch: cannot touch `test-user-test-file': Permission denied

VM

docker@boot2docker:~$ ls -lAF /Users/vmaatta/projects/data/writetest/
total 8
drwxr-xr-x    1 999      staff           68 Oct 23 20:44 root-dir/
drwxr-xr-x    1 999      staff           68 Oct 23 20:45 user-dir/
-rw-r--r--    1 999      staff            5 Oct 23 20:48 user-file
lrwxr-xr-x    1 999      staff            9 Oct 23 20:49 user-link -> user-file

Now, as you said, it's very difficult to come up with a general fix. And this is actually not that different from the situation of running docker on a Linux host without boot2docker or any other virtualisation layer. Issues with volume folder rights are a challenge there as well.

Currently the vboxsf mount is UID / GID 1000:50. That's docker:staff in the VM and, no-one:staff or first user to default group in a container. I changed this to 999:50 which matches the new group and user scenario by UID. GID is still 50 and this allows the VM docker user access and also the container root user is fine. The web server I mount a volume for uses the new group and user scenario as well so it works for me.

I don't know… maybe there's a better / more general UID / GID combination but I've seen 999:999 mentioned already a couple times in the Hub for containers' documentation. No surprise as they add both group and a user. But YMMV and that's why I've just done this in bootlocal.sh instead of a pull request.

And maybe we need something completely different [from vboxsf] to solve this.

  • Postgres being a bit bad example now as initdb dies on hard links now but oh well… there's nothing we can do for that here unfortunately.
@tianon
Contributor
tianon commented Oct 23, 2014

Yeah, in the general Linux case this is easier because the permissions
actually can be munged, generally speaking. With "vboxsf", we have to
choose one mapping, and no matter what we pick we're going to alienate a
non-trivial number of people, so we defaulted to "docker:staff" to at least
make the reasons for the default choice clear and obvious.

Maybe making the exact uid/gid configurable via the persistent storage
"profile" file is the way to go, but that's really just pushing the already
bad situation down on our users (however, with the benefit that they can
actually get themselves to the workaround without a huge amount of effort,
compared to where we're at now with hacks in bootlocal, etc).

@SvenDowideit
Member

tbh, the right place to do all this is in docker client - which would be the one to create a new share every time, and then the vm mount would be specific to the container - aaaand, for that we need help writing code :)

@lukaswelte

Are there any news on this?
I have no really good idea how to solve it, but this issue holds me back from switching to boot2docker, cause I just cannot run my apache2 docker container..

@vmaatta
vmaatta commented Oct 30, 2014

I might be wrong but I don't think there is really much that can be done on the boot2docker side very soon. Every container's needs are going to be different and getting the automatic vbox share working with them all is quite difficult.

If the apache2 container does not need to make hard links to the bind mounted volume you should be able to use a custom share with access rights suitable for the apache user. Above you can find my override of the default share but you could also add another one for use with apache

  1. Add a share location to the VM: $VBoxManage sharedfolder add "boot2docker-vm" --name "apacheshare" --hostpath "/Users/username/shares/data/apache" You could just as well do that via the VirtualBox UI.
  2. Mount it in the VM after bootup: mount -t vboxsf -o uid=999,gid=50 apacheshare /Users/username/shares/data/apache You'll need to find the suitable UID that works for the apache2 user, if it works.

For multiple bind mounts, and different containers, there might need to be multiple different shares depending on the needed UIDs.

I've added my version of number 2 to the bootlocal.sh in the VM so it's done automatically on boot. I don't have the script handy now but I can add it here later.

@SvenDowideit
Member

yup, @vmaatta has it baout right - you could do this by hand, but you probably should consider that you might be better off working out a different way to acheive it - like using volume containers.

@lukaswelte

@vmaatta Would be awesome if you could share your script.
I am not that familiar with virtual box.

@SvenDowideit Volume containers is no real option as we use fig for the development process and only some people use macs. So it would make the process more painful for the non-mac users.

@vmaatta
vmaatta commented Oct 30, 2014

@lukaswelte Here's my /mnt/sda1/var/lib/boot2docker/bootlocal.sh. You'll need to adapt it to your needs.

#!/bin/sh

if [ -e /Users/username ]; then
    umount /Users
fi

mkdir -p /Users/username/projects
mount -t vboxsf -o uid=999,gid=50 projects /Users/username/projects

@SvenDowideit I'm probably missing the point but… Volume containers are great and should always be used where they make sense. But with regards to bind mounting something from the host they don't change anything. They suffer from the same problems any other container does.

@SvenDowideit
Member

@vmaatta yeah - you get around that atm by creating your data container like:

docker run --name data -v /data busybox chmod 777 /data

and then you need to copy that data to your local machine using another container.

I use samba containers for a reason :)

@vmaatta
vmaatta commented Oct 31, 2014

@SvenDowideit yeah, no I don't… need to copy that data to your local machine kinda being the kicker :). I've used the Samba container too but we can copy data into and out of containers manually if we want. And that's really what samba is, just a target location(s) to manually copy data in / out. The real problem is -v host:container. This is the old different workflows discussion that's been had a couple times. Copying data into a location manually for production, say release workflow, might be workable with Samba or something like that. But local change-refresh-change etc, edit in IDE / see change reflected by the running container, jump into a different project with different kinds of files in different locations…

I've yet to see any hint of how is a Samba (or NFS or…) volume container to replace -v host:container. That's one single data location used in two places, host computer and container. Not two copies of data, one on the host and other in some Samba share, and worse, maybe even a third copy where it was copied from a data share folder to an application folder. Manually copying data around in different containers… What is the workflow, 2 seconds to make a change and 2 minutes to monkey around with rsync. I just can't compute this workflow :).

And the point I was making above: Volume container is not really relevant here. It can't bind mount a location from the host any different to any other container. It doens't change the problem at hand in any way, just moves it.

@karolsojko

I'm experiencing the same issue. Apache on my container can't write to neither cache or log folders located in the application that is bind mounted.

I'm using Docker 1.3.1 - is this in any nearest future plans to be fixed ?

@vmaatta vmaatta referenced this issue in docker/docker Nov 10, 2014
Closed

docker volumes w remote daemon #4023

@bradgessler

I've approached this problem with a slightly different work-around since I'm actually creating new docker images. Basically, you'll want to change the uid on the user you want to have access the host-mounted volume to 1000.

That means you'll want to add something like

# Sample docker file, mysql is the example login.
RUN usermod -u 1000 mysql

Then everything seems to work OK.

@txchen
txchen commented Nov 14, 2014

Any updates on this?
With this bug, postgresql cannot start if the data folder is mounted as volume. Because postgresql requires data folder to have 700 permission, and if the folder is mounted as volume (boot2docker, mac), chown/chmod cannot work at all, thus the service failed to start.
IMHO, this is a blocking bug.

@vmaatta
vmaatta commented Nov 14, 2014

@txchen Fixing this issue may not help you anyway. There is more information regarding postgres specifically in the above comments. It might be best to just run a separate data volume container, use the volume from there for postgres and handle any mounting and transfer needs between the data volume container and host.

@txchen
txchen commented Nov 14, 2014

@vmaatta thanks for your input! I will try to switch to data volume container solution, although I still think old style (volume to local folder) is much more convenient.

@splio-kjoyeux

Hello.

Any news on this ? It's really inconvenient.

Thanks.

@theitsmith

I'm experiencing the same issue with Apache 2. In my case, though, I'm not using boot2docker--I'm running Docker inside a CentOS 7 VirtualBox VM.

With this, the default UID for VirtualBox's Shared Folders (within the VM) is root with a GID of vboxsf. In order for the VM to write to the Shared Folder, the non-root user performing the writing must be a member of the vboxsf group.

The way I worked around this problem of not being able to write to Shared Folders from within the Docker container was to change the GID of the apache user (defaults to 48) within the Docker container to match that of the vboxsf group (996, in my case) in the VM.

This allows working around the problem without making any changes to the host or the VM and only requires one small change within the container, something that should be fairly easy to script.

I know this discussion is related to boot2docker specifically, but there are some situations in which boot2docker isn't the best choice--in my case, there are some kernel features missing that were affecting my containers.

As such, I thought it might be important to also consider the affects of this issue outside the context of boot2docker so that the final solution applies there as well. :)

@crucialfelix

I tried vagrant with docker provider and was able to share my data folder and start PostgreSQL. I am guessing that their folder sharing is more robust. This is always an issue with VMs. And b2d is a vm with a container inside. Two levels of folder sharing.

Boot2dpcker still seems more appropriate and I can ssh in and play more. But give vagrant a try too. In the end I'm sure b2d will solve this issue.

@tianon
Contributor
tianon commented Nov 19, 2014

If anyone has a solid solution that will actually work long-term without
extra developer friction (including but not limited to fragile
reconfiguring of external services that are frankly outside our reasonable
realm of control), we're all ears.

Support for "vboxsf" was added as a first-pass solution to the basic
problem of accessing your data from inside the VM, with full knowledge that
it has limitations that make it not necessarily well-suited for a number of
use-cases, this one included (which was why adding it in the first place
was so controversial). It is important to keep in mind that it is strictly
better than what we used to have (which was absolutely nothing).

As has been stated previously, the real solution for this ought to come in
Docker itself, since this is a common problem to anyone whose client and
daemon don't live and play in the same filesystem.

@SvenDowideit
Member

I'm hoping for docker/docker#9250 :)

@yuriteixeira

The @bradgessler is being good enough to me. Thanks for this tip, man 😄

@joeellis

Been hitting this bug hard in 1.3.2 and not sure what else to do.

For context, I'm trying to mount a data volume container in an apache container under /var/www and no amount of chown -R www-data:www-data /var/www/myvolume will change these permissions over.

This bug is particularly blocking because the site I'm trying to install there, prestashop, refuses to complete it's install process until all permissions in it's directory are set to www-data:www-data, which I can't do.

Anyone have advice on how to fix this? Or maybe I'm just doing it all wrong?

@theitsmith

See my comment above for a workaround: http://github.com/boot2docker/boot2docker/issues/581#issuecomment-63724242

This is easily scriptable during container creation (or applied afterwards). I use the following in my start.sh:

groupmod --gidstat --format=%g /srv/volumeapache

@bradgessler

You can also assign uid 1000 to the user www in my workaround above.

Maybe the owner of this thread could close it with a reference to the two workarounds above and point to an issue in Docker that will permanently fix this problem?

On Nov 28, 2014, at 10:07 AM, theitsmith notifications@github.com wrote:

See my comment above for a workaround: http://github.com/boot2docker/boot2docker/issues/581#issuecomment-63724242

This is easily scriptable during container creation (or applied afterwards). I use the following in my start.sh:

groupmod --gid stat --format=%g /srv/volume apache


Reply to this email directly or view it on GitHub.

@theitsmith

Agreed--UID 1000 will work for b2d. However, this may not work in other environments--in CentOS, for example, the mounted volume's UID is 0.

VirtualBox's convention is to always set the Shared Folders group to "vboxfs" in the VM. To allow a VM user to write to the Shared Folder, they advise adding said user to the vboxfs group. Carrying this same convention into the container, I think, adds additional portability beyond just b2d--I can't use b2d, for example, due to kernel compatibility issues with the services I need to containerize.

Alternatively, I've had success creating a raw disk image in Shared Folders and then mounting that disk image directly into the container (though I'm still experimenting some with this). This allows the container to do pretty much whatever it needs to with the mounted disk image--which includes choosing it's own file system format, setting user and group permissions, setting SELinux contexts, and so on.

Using a raw disk image like this is easy to do and allows additional portability between different hypervisors since neither Docker nor the containers have to care much about how the hypervisor shares host folders, which host file system is in use, etc, etc.

But going back to the post of @joeellis, prestashop needs write access to its data directory (per their documentation, prestashop doesn't care about the UID or GID--it just needs to be able to write to its data directory)--since he's using b2d, setting either the UID or the GID will work.

@crucialfelix

I've decided against using docker for postgresql. the data cannot be inside the image, and until I can easily mount a volume without fear of data loss or user permissions issues, I see no benefit, only problems. I wasted many days just trying to get postgres to work with an external volume.

@tkoeppen

We also tried running postgresql (9.3) with boot2docker (1.3.0) and failed
docker run --rm -i -t -v /Users/<user>/dev/data:/var/lib/postgresql

This small script shows that chown does not work.

echo "DATADIR=$DATADIR"
echo "Initializing Postgres Database at $DATADIR"
id
su -c "chown -R postgres:postgres $DATADIR"
ls -la $DATADIR
su -c "chown -R 102:106 $DATADIR"
ls -la $DATADIR
su postgres sh -c "mkdir -p $DATADIR/test"
su postgres sh -c "id"

The result:

DATADIR=/var/lib/postgresql/9.3/main
Initializing Postgres Database at /var/lib/postgresql/9.3/main
uid=0(root) gid=0(root) groups=0(root)
total 0
drwxr-xr-x 1 1000 staff 68 Nov 29 10:43 .
drwxr-xr-x 1 1000 staff 102 Nov 29 10:43 ..
total 0
drwxr-xr-x 1 1000 staff 68 Nov 29 10:43 .
drwxr-xr-x 1 1000 staff 102 Nov 29 10:43 ..
mkdir: cannot create directory '/var/lib/postgresql/9.3/main/test': Permission denied
uid=102(postgres) gid=106(postgres) groups=106(postgres),105(ssl-cert)

The only workaround I found is to use /tmp as DATADIR.
docker run --rm -i -t -v /tmp:/var/lib/postgresql
which makes of course no sense as I can't persist my postgres data on the host system.

@anentropic anentropic referenced this issue in kartoza/docker-postgis Dec 4, 2014
Closed

Struggling to use data volume #6

@paolomainardi
Contributor

+1 same issue here...

@kai-zhong

+1 same issue here.................. ^ ^

@kai-zhong

@paolomainardi Thanks. remount is a good solution.

@paolomainardi
Contributor

@cnaicc remount is not a solution, containers should have access to multiple uid/gid and they need to be isolated by the host environment.

@benjy
benjy commented Dec 29, 2014

This worked for me, good for your local dev environment. I'm using Boot2Docker.

RUN usermod -u 1000 www-data
@tobyhede tobyhede referenced this issue in whitesmith/fig-tree Jan 2, 2015
Open

permissions issue on db volume #3

@brunoric
brunoric commented Jan 6, 2015

Same problem here. =(

@aladagemre

I have the same problem with boot2docker: Docker v1.4.1, fig 1.0.1
It's fine on Linux: Docker v1.4.0, fig 1.0.1

I can't mount rabbitmq or mongodb just for persistence.
Any news on this?

@fcvarela
fcvarela commented Jan 6, 2015

Not sure how helpful this is but i always create a user w/ id 1000 in the
container and run the processes from that. Works perfectly fine. I also ADD
the data needed and them add a VOLUME to the same location. You can choose
to -v vol:vol in production and not to in dev, thereby using the same image
for both envs.

Hope this helps

On Tue, Jan 6, 2015 at 12:00 PM, Ahmet Emre Aladağ <notifications@github.com

wrote:

I have the same problem with boot2docker: Docker v1.4.1, fig 1.0.1
It's fine on Linux: Docker v1.4.0, fig 1.0.1

I can't mount rabbitmq or mongodb just for persistence.
Any news on this?


Reply to this email directly or view it on GitHub
#581 (comment)
.

@fcvarela
fcvarela commented Jan 6, 2015

Actually the other way around, you can docker run -v stuff:stuff in dev and
not to in prod :) Sorry about that.

On Tue, Jan 6, 2015 at 12:28 PM, Filipe Varela fcvarela@gmail.com wrote:

Not sure how helpful this is but i always create a user w/ id 1000 in the
container and run the processes from that. Works perfectly fine. I also ADD
the data needed and them add a VOLUME to the same location. You can choose
to -v vol:vol in production and not to in dev, thereby using the same image
for both envs.

Hope this helps

On Tue, Jan 6, 2015 at 12:00 PM, Ahmet Emre Aladağ <
notifications@github.com> wrote:

I have the same problem with boot2docker: Docker v1.4.1, fig 1.0.1
It's fine on Linux: Docker v1.4.0, fig 1.0.1

I can't mount rabbitmq or mongodb just for persistence.
Any news on this?


Reply to this email directly or view it on GitHub
#581 (comment)
.

@volkanunsal volkanunsal referenced this issue in kartoza/docker-postgis Jan 9, 2015
Closed

Mounting volumes #17

@hgl
hgl commented Jan 16, 2015

I think this should be fixed from the docker end. Related issue: docker/docker#7198

Every time a host directory is mounted, you should be able to specify uid/gid of the mounted directory. That way, boot2docker can keep using whatever uid & gid for vboxsf, and still allow different users in different containers to write to it.

A lot of people affects by this, not sure why that issue is still not fixed, probably not very easy to implement?

@sjtuadamyang

@vmaatta , so it's not possible to set up boot2docker + postgres with /var/lib/postgresql/data mounted from host machine since postgres needs to create hardlink in /var/lib/postgresql/data which is not supported by vboxsf??

@vmaatta
vmaatta commented Jan 20, 2015

@sjtuadamyang Pretty much, yes. You should probably go with a data volume container for Postgres. In the data volume container make /var/lib/postgresql/data just a normal volume. Additionally add some other path such as /mnt/hostdata as a host volume to some location on your host.

Now, as mentioned you still have to get data into and out of the database but it's not as bad as it sounds in Postgres' case. Just like you can open psql in a container, you can pg_dump data in a temporary container into/out the extra host mounted folder:

docker run -ti --volumes-from postgres-data --link postgres:postgres --rm postgres sh -c 'exec pg_dump -h "$POSTGRES_PORT_5432_TCP_ADDR" -p "$POSTGRES_PORT_5432_TCP_PORT" -U postgres -Fd -f /mnt/hostdata/backup/some_database_local_20150120_bak <some_database>'

That assumes there's a data volume container postgres-data and a running Postgres database in container postgres. All of these containers are postgres:latest.

@sjtuadamyang

@vmaatta , thanks for you explanation. Correct me if I understand it wrong, the reason I need a data-only container here is that I can keep the data-only container alive, and just run the postgres container one-off to dump the data into host directory. Theoretically, I can do the same thing by just get into the postgres container when it's serving and dump the database by issue the pg_dump command inside the postgres container, right?

@vmaatta
vmaatta commented Jan 25, 2015

@sjtuadamyang Yes, you can make it work without a data volume container; you can just change the --volumes-from to your running postgres and otherwise modify as needed. Generally I would not "get into", i.e. SSH (or nsenter etc.) into a container.

You can read up about data volume containers and why maybe use them in the documentation and only you can really know whether to use them or not.

@neciu
neciu commented Feb 10, 2015

Same here lads.

@pkerpedjiev

Pardon my ignorance, by why doesn't the mounted folder within the container have group write permissions? Other users could then be added to the 'staff' group and everything would be much easier.

@Vrakfall

Thanks @bradgessler for that workaround, I really like it. I'll use a groupadd instead as it seems to be a less violent solution to me.

The --vbox-share=[] option of boot2docker seems to be another good solution to me if you know the uid in advance but I still prefer a groupadd to this.

I see no other solution than the workaround because, for me, it all comes from Virtualbox, as @paolomainardi said. It would be better Virtualbox supports POSIX permissions "on the go". Sad that remounting doesn't actually work. :'(

I checked a bit, but not really much, and I didn't find this "usermod/groupadd" workaround in the boot2docker documentation. Shouldn't it be there for less users to lose time while trying to figure out why mysql (e.g.) can't write on its on database ?

@paolomainardi
Contributor

@Vrakfall switch to NFS to fix those issues.

@Vrakfall

Mmmh, what about performances ? Isn't vboxfs better for hard hdd usages ?

@paolomainardi
Contributor
@Vrakfall

Wow, I wasn't expecting such a difference !
Thanks a lot ! You made me change my mind for the NFS solution here ! =D (Would be hard for Windows users btw).

@Vrakfall

Arh ! Any clue or howto about mounting nfs from host to a container ? I find myself stuck because my container can't ping my server.

Thanks in advance !

@darwingr

Use Sven's 'Folder Sharing' instructions in the readme but instead of cifs put:
nfs://192.168.59.103/data
Then just copy or clone your files into that network share I guess.

@Vrakfall

Hosting folders with samba to read them with nfs... Meh, that sounds really weird to me. Ok, with cifs that would work but... I'd really much prefer using nsf (hoster on the host machine if possible) in this case. Any clue ? @paolomainardi ?

@paolomainardi
Contributor

@Vrakfall if you are using osx:

  1. Edit /etc/exports as follows:
/Users -mapall=[youruser]:staff [boot2docker-vm-ip]

Where [youruser] is your localuser as reported by whoami and boot2docker-vm is the ip as reported by boot2docker ip

  1. Restart nfsd services:
sudo nfsd stop && sudo nfsd start
  1. Create a script on boot2docker vm on /var/lib/boot2docker/bootlocal.sh:
docker@boot2docker:~$ cat /var/lib/boot2docker/bootlocal.sh
#|/bin/bash
sudo umount /Users
sudo /usr/local/etc/init.d/nfs-client start
sudo mount 192.168.59.3:/Users /Users -o rw,async,noatime,rsize=32768,wsize=32768,proto=tcp

You're done.

@paolomainardi
Contributor

And in the meanwhile i'm working on improving NFS/CIFS (samba) shares using FSCACHE kernel implementation, for boot2docker.
If you are curious/interested follow this thread: #699

@Vrakfall

Thanks for the help. Yes, I'm working on osx at work.
It feels I should copy it as root to /var/lib/boot2docker/bootlocal.sh but I can't find the root password anywhere. The docker user can't write to that directory or am I doing something wrong ?

I tried this:

$boot2docker ssh
$cp /Users/osxUser/path/bootlocal.sh /var/lib/boot2docker/bootlocal.sh
cp: overwrite '/var/lib/boot2docker/bootlocal.sh'? y
cp: can't create '/var/lib/boot2docker/bootlocal.sh': Permission denied

I'm forced to using command lines because my goal is to create an automated script.

@paolomainardi
Contributor

Just try "sudo su"

Sorry for typos, sent by mobile.
Il 17/feb/2015 10:11 "Jérémy Lecocq" notifications@github.com ha scritto:

Thanks for the help. Yes, I'm working on osx at work.
It feels I should copy it as root to /var/lib/boot2docker/bootlocal.sh
but I can't find the root password anywhere. The docker user can't write to
that directory or am I doing something wrong ?

I tried this:

$boot2docker ssh
$cp /Users/osxUser/path/bootlocal.sh /var/lib/boot2docker/bootlocal.sh
cp: overwrite '/var/lib/boot2docker/bootlocal.sh'? y
cp: can't create '/var/lib/boot2docker/bootlocal.sh': Permission denied

I'm forced to using command line because my goal is to create an automated
script.


Reply to this email directly or view it on GitHub
#581 (comment)
.

@Vrakfall

What was I thinking. xD

Oh and thank you for #699. I'm indeed curious AND interested in any better way of sharing host's files to containers.

I'm still stuck trying to mount the nfs share:

docker@boot2docker:~$ sudo sh /var/lib/boot2docker/bootlocal.sh
mount: RPC: Remote system error - Connection refused
mount: mounting 192.168.59.3:/Users on /Users failed: Bad file descriptor

Weird because here is my osx /etc/exports file:

/Users -mapall=stagiaire:staff 192.168.59.103(rw,non_root_squash,no_subtree_check)

I'm used to Linux for a long time but I'm new to osx so I might be missing some obvious points.

@paolomainardi
Contributor

Try to start nfs-rpc services:
/usr/local/etc/init.d/nfs-client start

Sorry for typos, sent by mobile.
Il 17/feb/2015 10:45 "Jérémy Lecocq" notifications@github.com ha scritto:

What was I thinking. xD

Oh and thank you for #699
#699. I'm indeed curious
AND interested in any better way of sharing host's files to containers.

I'm still stuck trying to mount the nfs share:

docker@boot2docker:~$ sudo sh /var/lib/boot2docker/bootlocal.sh
mount: RPC: Remote system error - Connection refused
mount: mounting 192.168.59.3:/Users on /Users failed: Bad file descriptor

Weird because here is my /etc/exports file:

/Users -mapall=stagiaire:staff 192.168.59.103(rw,non_root_squash,no_subtree_check)

I'm used to Linux for a long time but I'm new to osx so I might be missing
some obvious points.


Reply to this email directly or view it on GitHub
#581 (comment)
.

@Vrakfall

It was done by the .sh script. Sorry, I've removed the obvious lines from the output juste to make it clearer.
Here the original one:

docker@boot2docker:~$ sudo sh /var/lib/boot2docker/bootlocal.sh
umount: can't umount /Users: Invalid argument #Because I tried it multiple times
/usr/local/sbin/rpcbind is already running
808
/usr/local/sbin/rpc.statd is already running
810
Starting nfs client utilities.
mount: 192.168.59.3:/Users failed, reason given by server: Permission denied
mount: mounting 192.168.59.3:/Users on /Users failed: Bad file descriptor
@paolomainardi
Contributor

I see a "permission denied" on osx side.
Are you sure that boot2docker IP on export is correct ? Have you tried to
restart nfsd on osx ?

Sorry for typos, sent by mobile.
Il 17/feb/2015 10:53 "Jérémy Lecocq" notifications@github.com ha scritto:

It was done by the .sh script. Sorry, I've removed the obvious lines from
the output juste to make it clearer.
Here the original one:

docker@boot2docker:~$ sudo sh /var/lib/boot2docker/bootlocal.sh
umount: can't umount /Users: Invalid argument #Because I tried it multiple times
/usr/local/sbin/rpcbind is already running
808
/usr/local/sbin/rpc.statd is already running
810
Starting nfs client utilities.
mount: 192.168.59.3:/Users failed, reason given by server: Permission denied
mount: mounting 192.168.59.3:/Users on /Users failed: Bad file descriptor


Reply to this email directly or view it on GitHub
#581 (comment)
.

@Vrakfall

Yup for both statements.

$ boot2docker ip
192.168.59.103
@paolomainardi
Contributor

Try to remove the options you specified () on /etc/exports and restart the
server.

Sorry for typos, sent by mobile.
Il 17/feb/2015 11:03 "Jérémy Lecocq" notifications@github.com ha scritto:

Yup for both statements.

$ boot2docker ip
192.168.59.103


Reply to this email directly or view it on GitHub
#581 (comment)
.

@Vrakfall

That did it. :3 It was kinda my fault. I've removed the async option from /var/lib/boot2docker/bootlocal.sh and it worked as soon as I put it back (and after removing options from /etc/exports).
I'm not used to nfs servers. :s
I was thinking about moving the discussion to the emails or by skype or any IM way because we kinda were spamming everyone. =/ Sorry for that. But now it's working so it's too late.

Thank you a lot again !

@ludofleury

@mnapoli for your use case, I ended with a solution mixing volume container & custom path for the Symfony/php app cache/logs.

I've got a volume container for /var/cache/my-app & another one for /var/log/my-app.
Pretty neat.

@mikehaertl

I checked a bit, but not really much, and I didn't find this "usermod/groupadd" workaround in the boot2docker documentation.

+1

The usermod -u 1000 www-data fixes it for us. This should really be added as a simple workaround to the documentation.

@cordoval
Contributor

just in case i think @ludofleury meant this http://www.craftitonline.com/2015/02/docker-symfony-development-box-with-data-containers/ perhaps it can be useful to some that ran into this.

@mnapoli
mnapoli commented Feb 24, 2015

@ludofleury thank you! I need to give another try at docker… That whole boot2docker with virtualbox + problems with write permissions was really a cold shower, lost so much time on this and then gave up… I need to find another week end to spare :p

@Vrakfall

@mnapoli If you find yourself stuck with permissions, try mounting your volums with nfs on the boot2docker vm rather than vbox shares. ;D It's pretty neat and supports uid and gid permissions.

@pdonorio

For application containers (PHP and Python) I use nfs approach, so I clone project to my mac, map that folder to container, edit code directly with IDE and look for results inside container.

@krasilich yes, been there. I am working with python and a dev user inside the container.

But mapping that folders as read-only inside the container is just fine, they need to be edited from my editor on osx side, and then run inside the container. So there should be no need for NFS even if you use vbox mount. Am i missing something?

By the way Virtualbox shared folders have been hunting me since first ubuntu/windows fights inside my pc...

@krasilich

@pdonorio

But mapping that folders as read-only inside the container is just fine, they need to be edited from my editor on osx side, and then run inside the container. So there should be no need for NFS even if you use vbox mount. Am i missing something?

Yes, it works until your application has no need to write something to it's own folder, but usually (maybe it's me unlucky to work only with such kind of applications) they want to create folder for cache and/or session files, any kinds of imports/exports and so on.

@koryonik

Thanks @krasilich, nice nfs workaround !

@sebwebdev

I have an php-fpm box running on OSX with shared volumes (/var/www). As mentioned by @saada this fixed the issue for me:

RUN usermod -u 1000 www-data
RUN usermod -G staff www-data

Thanks!

@rjain15
rjain15 commented Dec 7, 2015

@pdonorio I have followed your steps to create docker volumes cli approach

$docker volume create --name data
$docker volume ls
$docker run --name mongo -v data:/data/db -d mongo
$docker exec $(docker ps -l -q) chown -R mongodb:mongodb /data/db
$docker exec $(docker ps -l -q) ls -l /data/db
$docker volume inspect data
$docker-machine ssh default sudo ls /mnt/sda1/var/lib/docker/volumes/data/_data

As @krasilich mentioned, this is creating the data volume inside the boot2docker vm, I want to create them on my OSX host.

Has anyone figured a way out?

@pdonorio
pdonorio commented Dec 9, 2015

@krasilich @rjain15 about sessions / cache you can still create other volumes or fit all inside the same volume. The important thing is to have persistence, doesn't matter where as long as you have control. And you can write stuff in this volumes if you apply the user id as i said in my previous posts.

Of course I note that if you loose your Vbox your volumes get lost too, so we may have import/export/backup like suggested for data containers:

# Export 'myvolume' content
$ docker run -v $(pwd):/backup -v myvolume:/dbdata ubuntu tar cvf /backup/backup.tar /dbdata

# Import from another Vbox
$ docker run -v $(pwd):/backup -v myvolume:/dbdata ubuntu bash -c "cd /dbdata && tar xvf /backup/backup.tar"

The point is that any data volume is not content you might want to edit, is just stuff you want to store somewhere. I think all possible actions can be transformed into pure docker volumes, avoiding to mount from the OSX host. I also think that volumes will give more options such as backup natively in future docker releases, as it happened for all the other stuff so far.

Finally: you might also write a script in cron to mount interested volumes and back them up.

I know it's still more work than before, but also cleaner than having sparse directories all around your file system.

@smarques

@krasilich solution worked wonderful for me, but today out of the blue I get:
conflicting exports for /Users, 192.168.99.100
exports:4: export option conflict for /Users

anyone had this problem? TIA

@yosifkit yosifkit referenced this issue in docker-library/elasticsearch Dec 23, 2015
Closed

Unable to access 'path.data' #74

@ariault-blanchard

What does it work with a docker-compose file ?

Actually i'm building a new docker image with Centos and the apache user inside is not allow to write...

Can somebody meet the problem ?

@nhooey
nhooey commented Dec 29, 2015

I get the same issue as @smarques did when he used @krasilich's solution.

I get this error:

$ ruby <(curl -L -s https://git.io/vvvco) default
Get vboxnet ip addres ... 192.168.99.1
Update /etc/exports ...
conflicting exports for /Users, 192.168.99.100
exports:3: export option conflict for /Users
@nhooey
nhooey commented Dec 29, 2015

I figured it out, I had an extra line in /etc/exports that conflicted. Just delete all lines in that file that have your Docker Machine IP address in them, so they don't conflict with the lines added by @krasilich's script.

@smarques

@odrzutowiec

Hello I am using php:7-apache docker container. I had problem with php writing files to mounted directory. Solution by @sebwebdev works perfectly for me.

RUN usermod -u 1000 www-data
RUN usermod -G staff www-data
@Bregor Bregor added a commit to Bregor/mysql that referenced this issue Jan 4, 2016
@Bregor Bregor Fixes #99
`docker-machine` and `boot2docker` in OS X with VirtualBox as backend
mounts vboxsf as UID=1000, GID=50, so user with UID=1000 will have permissions
to write to the data volume mounted as `-v /Users/somebody/mysqldata:/var/lib/mysql`
for example.

For details see:
- boot2docker/boot2docker#581
- boot2docker/boot2docker#587 (comment)
- docker/machine#2660 (comment)
6468d07
@motin
motin commented Jan 7, 2016

A related workaround, showing how image maintainers can make non-root users write to locally mounted folders on osx: docker-library/mysql#99 (comment)

@wpp
wpp commented Jan 11, 2016

Big thank you to @krasilich! Also it seems like adlogix/docker-machine-nfs#17 has been fixed too.

docker-machine-nfs test --shared-folder=/var/www --nfs-config="-alldirs -maproot=0"

@smarques
smarques commented Feb 3, 2016

@nhooey thank you will try that 😄

@pdonorio
pdonorio commented Feb 5, 2016

For Mac users a recent alternative is to use xhyve lightweight OS X virtualization solution instead of boot2docker/virtualbox.

It is very simple to install and use it with Docker with the help of the binary dlite. I read and followed the instructions on this post and I was quickly up and running.

Quoting:
"DLite also mounts your entire /Users directory into the virtual machine via NFS", and (i tested it successfully) you can change permissions just fine.

@orangejenny orangejenny referenced this issue in dimagi/commcare-hq Feb 17, 2016
Merged

Docker for mac #10332

@koushikckm

Is this issue still open for Mac OS X->Virtualbox-> mysql?
I m really struggling on this

@koushikckm

@nachimehta Were u able to overcome this issue on OS X?

@haobird
haobird commented Feb 22, 2016

Here is a solution。
1.运行命令:docker-machine ls 查看当前你使用的虚拟机
2.根据名字进入虚拟机:docker-machine ssh default
3.在虚拟机中编辑这个文件:sudo vi /mnt/sda1/var/lib/boot2docker/bootlocal.sh
4.在文件中写入如下命令:

#!/bin/sh

if [ -e /Users/jie ]; then
        sudo umount /Users
fi

sudo mount -t vboxsf -o uid=1001,gid=50 Users /Users

5.保存后,重启虚拟机:sudo reboot
6.docker run 新的container即可。如:docker run -it -P --name web -v /Users/jie/DockerWork:/www ubuntu14.04:latest /bin/bash

@KIVagant

@haobird, да, всё стало сразу понятнее, продолжай в том же духе! :)

@haobird
haobird commented Feb 22, 2016

@KIVagant I hope it will help someone. ^_^

@emaiax
emaiax commented Feb 23, 2016

@mnapoli I'm still experiencing the same issue with dinghy. codekitchen/dinghy#31

Can you tell me how have you accomplished this? :/

@mnapoli
mnapoli commented Feb 23, 2016

@emaiax I have no issues at all with dinghy :/ I just installed it following the readme and did nothing specific.

@istinspring

holy crap i can't even imagine how such a basic things are still broken.

So it's basically impossible to use docker on OSX for development? (if mongo image can't use your local databases files)

@Moghedrin
Contributor

@istinspring If it's basic, make a PR ;D Open Source ftw! \o/

@JulienD
JulienD commented Mar 9, 2016

@mnapoli, Issuing the same problem than @emaiax. Trying to install Mysql to a shared directory with Dinghy make Mysql crash due to a permission problem. I would be interested how you get rid of the problem. Looking at dinghy there is still an issue about this point codekitchen/dinghy#31

@KIVagant

@JulienD, use LXC / LXD and/or Vagrant in OSX.

@pierrerigal

@JulienD : I had the same problem, I switched to dlite https://github.com/nlf/dlite, it just works.

@paolomainardi
Contributor

Guys to solve all the permissions problems you just have to switch to NFS
instead of vboxfs, it''s not really a matter of dlite or whatever.

Just Google: docker-machine-nfs
Il 10/mar/2016 15:04, "Pierre" notifications@github.com ha scritto:

@JulienD https://github.com/JulienD : I had the same problem, I
switched to dlite https://github.com/nlf/dlite, it just works.


Reply to this email directly or view it on GitHub
#581 (comment)
.

@denji
denji commented Mar 10, 2016

@paolomainardi Another 3rd module hack boot2docker, it doesn't fix the underlying architectural problem.

@paolomainardi
Contributor

The architecture doesn't have any problem guys, the problem here is vboxfs,
just change it to NFS.
Il 10/mar/2016 16:29, "Denis Denisov" notifications@github.com ha scritto:

@paolomainardi https://github.com/paolomainardi Another 3rd module
hack, it doesn't fix the underlying architectural problem.


Reply to this email directly or view it on GitHub
#581 (comment)
.

@JulienD
JulienD commented Mar 13, 2016

@paolomainardi, any hint on how to do that ?

@paolomainardi
Contributor

Yes, you can automate using this script:
https://github.com/adlogix/docker-machine-nfs

It's a a matter of just a few commands.
Il 13/mar/2016 13:39, "Julien Dubreuil" notifications@github.com ha
scritto:

@paolomainardi https://github.com/paolomainardi, any hint on how to do
that ?


Reply to this email directly or view it on GitHub
#581 (comment)
.

@binabot
binabot commented Mar 15, 2016

RUN usermod -u 1000 www-data Worked great !

@agborkowski

@binabot is this command should run inside container ? Im using kitematic and there is not to much configuration to run command after start the container..

@binabot
binabot commented Mar 17, 2016

@agborkowski u can run usermod -u 1000 www-data inside your container OR add RUN usermod -u 1000 www-data into your Dockerfile. Never used kitematic so dont know what to do with your situation ;) gl

@tanhaa
tanhaa commented Mar 17, 2016

I would really suggest mounting your shared folder as NFS as it's a better solution than individually changing your uid/gid for the containers that need access. There are several comments in this thread that point to how to do that.

@ivicaned
ivicaned commented Apr 8, 2016

I used RUN usermod -u 1000 www-data into Docker file and it worked great.
I am waiting for Docker for OSX which is currently in beta, which should fix this issue with permissions: https://blog.docker.com/2016/03/docker-for-mac-windows-beta/

@SydOps
SydOps commented Apr 25, 2016

Thanks, @ivicaned . Your fix is perfect.

You save me the whole day that I stuck on this issue.

@SydOps
SydOps commented Apr 25, 2016

and @ivicaned

Regarding native docker for OSX , I already got token and installed. Yes, there is no this permission issue with same images.

@raptor235

Hey guys could use some help here.. got everything working properly.. I'm also exposing mariadb to a mounted volume so data is persistent.

The problem is I can set everything up... run docker-compose build, then docker-compose up, I see the db folder gets built by mariadb (those files are restricted to osx root only user) and things are running fine.

However when I run another docker-compose build command I run into issues from my php container complaining about access to the db folder which is setup from mariadb container... I'm not sure if this has something to do with /etc/exports and permissions of that folder but I can't build anything else unless I delete the db folder in the root...

I'm not sure if any of the options for docker-machine-nfs will help with this issue...

`#docker-machine ip default
web:
    build: .
    container_name: web
    ports:
        - "9090:80"
    volumes:
        - /Users/bartdabek/Sites/hgv/hgv_data/sites/:/code
        - ./site.conf:/etc/nginx/conf.d/site.conf
        - ./nginx.conf:/etc/nginx/nginx.conf
        - ./sites-enabled:/etc/nginx/sites-enabled
    links:
        - php
        #- mariadb
        - memcached
php:
    container_name: php
    build: .
    dockerfile: Dockerfile-php
    volumes:
        - /Users/bartdabek/Sites/hgv/hgv_data/sites/:/code
        - ./Tideways.php:/usr/local/lib/php/extensions/no-debug-non-zts-20151012/Tideways.php
    links:
        #- mariadb
        - memcached
memcached:
  container_name: memcached
  image: sameersbn/memcached:latest
  ports:
    - "11211:11211"
  restart: always
mariadb:
  container_name: mariadb
  image: mariadb
  volumes:
    - /Users/bartdabek/Sites/docker/db/:/var/lib/mysql
  environment:
    DB_ADMIN_PASS: pass
    MYSQL_ROOT_PASSWORD: root
  ports:
    - "3306:3306"
`

image

image

@raptor235
raptor235 commented May 19, 2016 edited

Basically I need to change permissions back on the db folder to get things to build again

drwxr-xr-x  11 bartdabek  staff    374 19 May 00:02 .
drwxr-xr-x  13 bartdabek  staff    442 16 May 15:36 ..
-rw-r--r--   1 bartdabek  staff    365 18 May 22:58 Dockerfile
-rw-r--r--   1 bartdabek  staff    267 18 May 22:26 Dockerfile-data
-rw-r--r--   1 bartdabek  staff   1642 18 May 23:06 Dockerfile-php
-rw-r--r--@  1 bartdabek  staff  43748 17 May 21:44 Tideways.php
drwxr-xr-x  11 999        999      374 19 May 00:02 db
-rw-r--r--   1 bartdabek  staff   1006 19 May 00:02 docker-compose.yml
-rw-r--r--   1 bartdabek  staff    743 18 May 23:09 nginx.conf
-rw-r--r--   1 bartdabek  staff    508 16 May 15:38 site.conf
drwxr-xr-x   3 bartdabek  staff    102 18 May 23:13 sites-enabled
@motin
motin commented May 19, 2016 edited

A related workaround, showing how image maintainers can make non-root users write to locally mounted folders on osx: docker-library/mysql#99 (comment)

Check this out. Many seem to have this problem with mysql/mariadb and a workaround has been around for quite some time - using usermod to change the user/group ids to match the ownership of the host volume folder:

Step 1: Add the script run.sh somewhere in your project:

#!/bin/bash
set -e

# Script to workaround docker-machine/boot2docker OSX host volume issues: https://github.com/docker-library/mysql/issues/99

echo '* Working around permission errors locally by making sure that "mysql" uses the same uid and gid as the host volume'
TARGET_UID=$(stat -c "%u" /var/lib/mysql)
echo '-- Setting mysql user to use uid '$TARGET_UID
usermod -o -u $TARGET_UID mysql || true
TARGET_GID=$(stat -c "%g" /var/lib/mysql)
echo '-- Setting mysql group to use gid '$TARGET_GID
groupmod -o -g $TARGET_GID mysql || true
echo
echo '* Starting MySQL'
chown -R mysql:root /var/run/mysqld/
/entrypoint.sh mysqld --user=mysql --console

Step 2: Change the command in docker-compose.yml to run this script instead of the ordinary command.

Example:

# Local database server to mimic a cloud database
localdb:
  image: mysql:5.6.27
  volumes:
    - ./stack/localdb/.db/mysql:/var/lib/mysql:rw
    - ./stack/localdb/:/stack/localdb:rw
  ports:
    - "3306"
  environment:
    MYSQL_ROOT_PASSWORD: "local-mysql-pass"
  command: "/stack/localdb/run.sh"
@raptor235

Thanks @motin that worked!

However I'm now stuck as the maridb hostname is not longer linked to any of the other containers... any idea why that default functionality is no longer working?

@raptor235

Nevermind last comment... some sort of hickup working again! Thanks

@millerdev millerdev added a commit to dimagi/docker-riak-cs that referenced this issue Jun 8, 2016
@millerdev millerdev Fix permissions in set-admin-keys.sh
Some files in /var/lib/riak were ending up owned by root, possibly because riak-admin was run as root.

Update: while this still seems to work, the change was probably not necessary. The real problem seems to be docker os Mac OS X has problems with permissions. See boot2docker/boot2docker#581 (comment)

One thing done here that is pretty important is to only set user-exists if user creation succeeds (I'm assuming the erl command exits with an error code when user creation fails).
00e1b49
@cpapidas

I have a container with apache, php and mysql. The solution for me was

usermod -u 1000 www-data

@carlosmunoz carlosmunoz referenced this issue in zanata/zanata-server Jun 17, 2016
Merged

feat: add docker development images #1211

@jolic
jolic commented Jun 19, 2016 edited

@cpapidas

Could you please send a small info or a step by step howto how you did this?

@cpapidas
cpapidas commented Jun 19, 2016 edited

Of course i can @jolic

  1. Run your docker image. In my case the command is
    $ docker run -it -p 80:80 -v .../Project/PHP:/root/project myImageName
  2. Run $ usermod -u 1000 www-data
  3. Then if you to see the files' permissions, will be similar to the image bellow

screen shot 2016-06-20 at 12 16 42 am

@ababushkin

Has anyone worked out a simple - StackOverflow style - solution to this problem yet?

I've tried all the conventional solutions, such as:

  1. Creating and configuring the permissions script (as per @motin's) comment
  2. Manually running usermod -u 1000 mysql in a custom Dockerfile (that inherits from this one)

The only 'feasible' solution I see is a custom project created by @dgraziotin, which deviates from the main MySQL / MariaDB docker images (https://github.com/dgraziotin/osx-docker-mysql). This hardly seems like an optimal solution, especially if Docker is to get even more rapid adoption throughout the community.

@ernestom
ernestom commented Sep 2, 2016

@ababushkin using docker-machine-nfs worked for me.

@yosifkit
Contributor
yosifkit commented Sep 2, 2016 edited

With docker-library/mysql#161 you should be able to run mysql as the owner of the directory in question:

docker run -d -e MYSQL_ROOT_PASSWORD=foobar1234 --user 1000:50 -v /Users/my-user/mysql-data/:/var/lib/mysql/ mysql:5.7

This will fix the permissions problem, but I cannot guarantee that mysql will always run on a VirtualBox Shared Folder. MongoDB for example cannot run on the shared folder since the file system does not support everything it needs.

@motin
motin commented Sep 5, 2016

@ababushkin I have read reports from users of native Docker for Mac that they no longer need any workaround. I created the permissions script to make it work on Docker Toolbox.

@dgraziotin

@ababushkin I confirm that with Docker v1.12.0 for Mac, there is no need to use my ugly workaround anymore :-) @motin

@ababushkin

@dgraziotin @motin Ouch, looks like I'm still running Docker v1.11.0.

I'll upgrade and give it a whirl.

@yosifkit thanks for the tip!

@ababushkin
ababushkin commented Sep 5, 2016 edited

@ernestom did you notice any performance improvement for disk sync when using that solution? Symfony runs really poorly for me at the moment. I have the same issue when using Vagrant and VirtualBox shared volumes and worked around it by using an NFS mount.

Update: I just noticed that the new version of docker has its own VM and a new OSX dedicated file system layer. I'll try this out to see if there are still performance issues :)

@ernestom
ernestom commented Sep 5, 2016

@ababushkin I didn't notice any significant impact in performance with NFS for Docker, and I've been using it for years on my vagrant/vbox projects without issues.

@krasilich

@ababushkin I have been using Docker for Mac for a couple of months now, and unfortunately performance issues still there for me. My case is - project on local, mounted inside container, running inside container. For example some project on Symfony, I run some heavy with high I/O app/console command and waiting tens of minutes or even hours to complete instead of up to 10 minutes on pure local.

@ababushkin

@ernestom @krasilich Thats a bummer.

@ernestom do you have a boilerplate docker-compose file that's using the NFS solution by any chance?

@dend
dend commented Oct 24, 2016 edited

So just to follow-up here, the issue seems to be gone on the Mac if you install the native Docker beta (use beta channel here). That obviously doesn't solve the problem much for automated scenarios, but works well for local dev.

@ababushkin

@dend Yup thats correct, permission issues are fixed with Docker for Mac. You don't need to install the beta version, the stable release fixes it as well.

Unfortunately performance issues have not been fixed. At this stage I've been getting around performance issues by doing smarter builds of my images (e.g. mounting volumes that don't need lots of read/write operations by the app)

@blueimp blueimp added a commit to blueimp/docker-node that referenced this issue Nov 7, 2016
@blueimp blueimp Add node group/user with gid/uid 1000 and a home directory.
This is a workaround for boot2docker issue 581, see
boot2docker/boot2docker#581
c1607e1
@blueimp blueimp added a commit to blueimp/docker-node that referenced this issue Nov 14, 2016
@blueimp blueimp Add node group/user with gid/uid 1000 and a home directory.
This is a workaround for boot2docker issue 581, see
boot2docker/boot2docker#581
fbe7593
@blueimp blueimp added a commit to blueimp/docker-node that referenced this issue Nov 15, 2016
@blueimp blueimp Add node group/user with gid/uid 1000 and a home directory.
This is a workaround for boot2docker issue 581, see
boot2docker/boot2docker#581
55c72ae
@blueimp blueimp added a commit to blueimp/docker-node that referenced this issue Nov 15, 2016
@blueimp blueimp Add node group/user with gid/uid 1000 and a home directory.
This is a workaround for boot2docker issue 581, see
boot2docker/boot2docker#581
3d8dc76
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment