File mount does not update with changes from host #15793

Closed
greg0ire opened this Issue Aug 24, 2015 · 50 comments

Comments

Projects
None yet
@greg0ire

I have this nginx container, on which I'm mounting a vhost configuration file (indirectly, in fact, I'm using a data volume container). I would like to reload the configuration without stopping the container, with docker-compose kill -s HUP webserver, but when I change the configuration file on the host, it does not change on the guest, and vice versa. When I restart the container however, the guest takes into account the host changes. Here is an excerpt of my docker-compose.yml :

data:
    build: ./docker
    volumes:
        - .:/srv
        - ../symfony-1.4.20:/usr/lib/symfony-1.4
        - ./docker/conf/nginx_vhost.conf:/etc/nginx/sites-enabled/app.conf
        - .home-developer:/home/developer
        - $SSH_AUTH_SOCK:/tmp/agent.sock
webserver:
    image: greg0ire/nginx
    volumes_from:
        - data
    env_file:
        - ./docker-compose.env
    environment:
        - DNSDOCK_IMAGE=web
    links:
        - appserver
@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Aug 24, 2015

Contributor

Can you provide output of docker info and docker version?
Have you inspected the data in the container after updating to verify?

Contributor

cpuguy83 commented Aug 24, 2015

Can you provide output of docker info and docker version?
Have you inspected the data in the container after updating to verify?

@greg0ire

This comment has been minimized.

Show comment
Hide comment
@greg0ire

greg0ire Aug 24, 2015

Containers: 33
Images: 207
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 273
Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.19.0-26-generic
Operating System: Ubuntu 15.04
CPUs: 4
Total Memory: 3.85 GiB
Name: grosbill
ID: UFPX:PUSX:ZCTO:KZYC:VJD5:5R4A:32U7:H4QL:MHLZ:TBF3:J5GS:T3JB
WARNING: No swap limit support

uname -a
Linux grosbill 3.19.0-26-generic #28-Ubuntu SMP Tue Aug 11 14:16:32 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

docker version
Client version: 1.7.1
Client API version: 1.19
Go version (client): go1.4.2
Git commit (client): 786b29d
OS/Arch (client): linux/amd64
Server version: 1.7.1
Server API version: 1.19
Go version (server): go1.4.2
Git commit (server): 786b29d
OS/Arch (server): linux/amd64

Have you inspected the data in the container after updating to verify?

To check if the data was up-to date, I did this : docker exec -it carel_webserver_1 /bin/bash

I'm running all these containers in on a linux host.

Here is the Dockerfile for the nginx container. It

FROM ubuntu:trusty ENV DEBIAN_FRONTEND noninteractive RUN apt-get update && \
    apt-get install --yes nginx && \
    rm --recursive --force /var/lib/apt/lists/*
RUN ln --symbolic --force /dev/stdout /var/log/nginx/access.log
RUN ln --symbolic --force /dev/stderr /var/log/nginx/error.log
EXPOSE 80 443 ADD monitoring.conf /etc/nginx/conf.d/
ADD monitoring.html /usr/share/nginx/html/monitoring.html
ADD coverage.conf /etc/nginx/conf.d/
ADD dev.conf /etc/nginx/conf.d/
ENTRYPOINT ["nginx"]
CMD ["-g", "daemon off;"]

Here is the dockerfile for my data container :

FROM ubuntu:trusty

VOLUME /srv
VOLUME /etc/nginx/sites-enabled/app.conf
VOLUME /var/lib/mysql
VOLUME /home/developer/.composer

Not sure whether the VOLUME declaration in the data container is necessary BTW…

Containers: 33
Images: 207
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 273
Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.19.0-26-generic
Operating System: Ubuntu 15.04
CPUs: 4
Total Memory: 3.85 GiB
Name: grosbill
ID: UFPX:PUSX:ZCTO:KZYC:VJD5:5R4A:32U7:H4QL:MHLZ:TBF3:J5GS:T3JB
WARNING: No swap limit support

uname -a
Linux grosbill 3.19.0-26-generic #28-Ubuntu SMP Tue Aug 11 14:16:32 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

docker version
Client version: 1.7.1
Client API version: 1.19
Go version (client): go1.4.2
Git commit (client): 786b29d
OS/Arch (client): linux/amd64
Server version: 1.7.1
Server API version: 1.19
Go version (server): go1.4.2
Git commit (server): 786b29d
OS/Arch (server): linux/amd64

Have you inspected the data in the container after updating to verify?

To check if the data was up-to date, I did this : docker exec -it carel_webserver_1 /bin/bash

I'm running all these containers in on a linux host.

Here is the Dockerfile for the nginx container. It

FROM ubuntu:trusty ENV DEBIAN_FRONTEND noninteractive RUN apt-get update && \
    apt-get install --yes nginx && \
    rm --recursive --force /var/lib/apt/lists/*
RUN ln --symbolic --force /dev/stdout /var/log/nginx/access.log
RUN ln --symbolic --force /dev/stderr /var/log/nginx/error.log
EXPOSE 80 443 ADD monitoring.conf /etc/nginx/conf.d/
ADD monitoring.html /usr/share/nginx/html/monitoring.html
ADD coverage.conf /etc/nginx/conf.d/
ADD dev.conf /etc/nginx/conf.d/
ENTRYPOINT ["nginx"]
CMD ["-g", "daemon off;"]

Here is the dockerfile for my data container :

FROM ubuntu:trusty

VOLUME /srv
VOLUME /etc/nginx/sites-enabled/app.conf
VOLUME /var/lib/mysql
VOLUME /home/developer/.composer

Not sure whether the VOLUME declaration in the data container is necessary BTW…

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Aug 27, 2015

Contributor

I bet I know what's happening here...

If you are using some editor like vim, when you save the file it does not save the file directly, rather it creates a new file and copies it into place.
This breaks the bind-mount, which is based on inode. Since saving the file effectively changes the inode, changes will not propagate into the container.
When the container is restarted the new inode.
If you edit the file in place you should see changes propagate.

This is a known limitation of file-mounts and is not fixable.

Does this accurately describe the issue?

Contributor

cpuguy83 commented Aug 27, 2015

I bet I know what's happening here...

If you are using some editor like vim, when you save the file it does not save the file directly, rather it creates a new file and copies it into place.
This breaks the bind-mount, which is based on inode. Since saving the file effectively changes the inode, changes will not propagate into the container.
When the container is restarted the new inode.
If you edit the file in place you should see changes propagate.

This is a known limitation of file-mounts and is not fixable.

Does this accurately describe the issue?

@greg0ire

This comment has been minimized.

Show comment
Hide comment
@greg0ire

greg0ire Aug 27, 2015

If you edit the file in place you should see changes propagate.

I just restarted my container, used nano to do the editing and it works! Thanks a lot. Feel free to close this, I am not sure if it should be considered as a bug or not.

If you edit the file in place you should see changes propagate.

I just restarted my container, used nano to do the editing and it works! Thanks a lot. Feel free to close this, I am not sure if it should be considered as a bug or not.

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Aug 27, 2015

Contributor

Closing, it's not really a bug, just how linux works unfortunately.

Contributor

cpuguy83 commented Aug 27, 2015

Closing, it's not really a bug, just how linux works unfortunately.

@cpuguy83 cpuguy83 closed this Aug 27, 2015

@tcmal

This comment has been minimized.

Show comment
Hide comment
@tcmal

tcmal Nov 29, 2015

Is there any workaround for this? I don't think using nano is really viable in a project larger than 1 file. I'm using Sublime Text and I'd like to keep it that way.

tcmal commented Nov 29, 2015

Is there any workaround for this? I don't think using nano is really viable in a project larger than 1 file. I'm using Sublime Text and I'd like to keep it that way.

@greg0ire

This comment has been minimized.

Show comment
Hide comment
@greg0ire

greg0ire Nov 29, 2015

After reading this, I guess if I type set noswapfile, I might be able to continue using vim. Maybe Sublime Text has a similar option ?

After reading this, I guess if I type set noswapfile, I might be able to continue using vim. Maybe Sublime Text has a similar option ?

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Nov 29, 2015

Contributor

@oscarorsomething Mount a dir instead of a file.

Contributor

cpuguy83 commented Nov 29, 2015

@oscarorsomething Mount a dir instead of a file.

@tcmal

This comment has been minimized.

Show comment
Hide comment
@tcmal

tcmal Nov 29, 2015

@cpuguy83 I am mounting a dir.

After some googling, Sublime text has atomic_save instead so Adding "atomic_save": false to user preferences worked (After a restart). Thanks @greg0ire

tcmal commented Nov 29, 2015

@cpuguy83 I am mounting a dir.

After some googling, Sublime text has atomic_save instead so Adding "atomic_save": false to user preferences worked (After a restart). Thanks @greg0ire

@greg0ire

This comment has been minimized.

Show comment
Hide comment
@greg0ire

greg0ire Nov 29, 2015

I am mounting a dir.

Are you sure ? That's really strange… @cpuguy83 : maybe he is mounting a file and a dir (dunno if this is possible).

I am mounting a dir.

Are you sure ? That's really strange… @cpuguy83 : maybe he is mounting a file and a dir (dunno if this is possible).

@tiaguinho

This comment has been minimized.

Show comment
Hide comment
@tiaguinho

tiaguinho Dec 1, 2015

@greg0ire I try use set noswapfile, but nothing change..
Isn't possible use vim with volumes?

@greg0ire I try use set noswapfile, but nothing change..
Isn't possible use vim with volumes?

@greg0ire

This comment has been minimized.

Show comment
Hide comment
@greg0ire

greg0ire Dec 1, 2015

Isn't possible use vim with volumes?

I don't know, but I'm interested in your results.

This breaks the bind-mount, which is based on inode. Since saving the file effectively changes the inode, changes will not propagate into the container.
When the container is restarted the new inode.

  1. Try finding out the inode of your file before, change your file with vim, and check that the inode changes. This should validate @cpuguy83 's theory
  2. If it turns out to be valid, then do the same, but set noswapfile before changing the file.
  3. Please report back with your findings.

greg0ire commented Dec 1, 2015

Isn't possible use vim with volumes?

I don't know, but I'm interested in your results.

This breaks the bind-mount, which is based on inode. Since saving the file effectively changes the inode, changes will not propagate into the container.
When the container is restarted the new inode.

  1. Try finding out the inode of your file before, change your file with vim, and check that the inode changes. This should validate @cpuguy83 's theory
  2. If it turns out to be valid, then do the same, but set noswapfile before changing the file.
  3. Please report back with your findings.
@tiaguinho

This comment has been minimized.

Show comment
Hide comment
@tiaguinho

tiaguinho Dec 2, 2015

I made some tests @greg0ire, and this is the result.

With or without set noswapfile the inode changes, everytime the file is saved, but I made a little research how to make vim don't change the inode of a file.
The result was to use set backupcopy=yes, I try and the inode don't change when the file is modified.

Thank you guys.

I made some tests @greg0ire, and this is the result.

With or without set noswapfile the inode changes, everytime the file is saved, but I made a little research how to make vim don't change the inode of a file.
The result was to use set backupcopy=yes, I try and the inode don't change when the file is modified.

Thank you guys.

@greg0ire

This comment has been minimized.

Show comment
Hide comment
@greg0ire

greg0ire Dec 2, 2015

And thank you for your feedback, it is interesting.

greg0ire commented Dec 2, 2015

And thank you for your feedback, it is interesting.

@tcmal

This comment has been minimized.

Show comment
Hide comment
@tcmal

tcmal Dec 5, 2015

This issue has come back for me a few days later. In both Sublime Text (3, with atomic save false) and Atom, The inode doesn't change at all. I assume this is an issue with the editors?

tcmal commented Dec 5, 2015

This issue has come back for me a few days later. In both Sublime Text (3, with atomic save false) and Atom, The inode doesn't change at all. I assume this is an issue with the editors?

@tcmal

This comment has been minimized.

Show comment
Hide comment
@tcmal

tcmal Dec 5, 2015

It doesn't seem to be an editor issue (at least on atom)

The files do update but only after around a minute which is better but still incredibly annoying.

tcmal commented Dec 5, 2015

It doesn't seem to be an editor issue (at least on atom)

The files do update but only after around a minute which is better but still incredibly annoying.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Dec 5, 2015

Member

@oscarorsomething I suspect that's because VirtualBox shared folders don't support inotify; https://www.virtualbox.org/ticket/10660

Member

thaJeztah commented Dec 5, 2015

@oscarorsomething I suspect that's because VirtualBox shared folders don't support inotify; https://www.virtualbox.org/ticket/10660

@tcmal

This comment has been minimized.

Show comment
Hide comment
@tcmal

tcmal Dec 5, 2015

@thaJeztah I'm not using Virtual box. Arch Linux Docker version 1.9.1, build a34a1d5-dirty

tcmal commented Dec 5, 2015

@thaJeztah I'm not using Virtual box. Arch Linux Docker version 1.9.1, build a34a1d5-dirty

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Dec 5, 2015

Member

@oscarorsomething oh, ignore my comment then 👍 (I know many people ran into that when using boot2docker/VirtualBox)

Member

thaJeztah commented Dec 5, 2015

@oscarorsomething oh, ignore my comment then 👍 (I know many people ran into that when using boot2docker/VirtualBox)

@sebwebdev

This comment has been minimized.

Show comment
Hide comment
@sebwebdev

sebwebdev Dec 14, 2015

Turning off sendfile off for Nginx solved this issue for me. Just put this in your vhost conf:

sendfile off;

The equivalent for Apache it is "EnableSendfile off".
See https://abitwiser.wordpress.com/2011/02/24/virtualbox-hates-sendfile/ for reference.

Turning off sendfile off for Nginx solved this issue for me. Just put this in your vhost conf:

sendfile off;

The equivalent for Apache it is "EnableSendfile off".
See https://abitwiser.wordpress.com/2011/02/24/virtualbox-hates-sendfile/ for reference.

@greg0ire

This comment has been minimized.

Show comment
Hide comment
@greg0ire

greg0ire Dec 15, 2015

@sebwebdev : thanks, but I think you are a bit off topic : obviously, you are not mounting the file you are providing with your vhost one by one. The issue at hand is about file mounts.

@sebwebdev : thanks, but I think you are a bit off topic : obviously, you are not mounting the file you are providing with your vhost one by one. The issue at hand is about file mounts.

@sebwebdev

This comment has been minimized.

Show comment
Hide comment
@sebwebdev

sebwebdev Dec 15, 2015

You are right, but @oscarorsomething 's problem sounds exactly like my issue. Maybe it points someone in the right direction.

You are right, but @oscarorsomething 's problem sounds exactly like my issue. Maybe it points someone in the right direction.

@greg0ire

This comment has been minimized.

Show comment
Hide comment
@greg0ire

greg0ire Dec 15, 2015

I reread @OscarOrSomething and, after all he does not state if he was having the issue with the nginx configuration or with assets served by his webserver (but since he is having the issue with several files, the latter is more likely). So you might have the same problem indeed.

@OscarOrSomething : if you are having problems with assets, you should probably check the contents of the file from inside docker directly to know if the problems lies with docker or your webserver. The problem could be an HTTP caching problem if the webserver is at fault.

I reread @OscarOrSomething and, after all he does not state if he was having the issue with the nginx configuration or with assets served by his webserver (but since he is having the issue with several files, the latter is more likely). So you might have the same problem indeed.

@OscarOrSomething : if you are having problems with assets, you should probably check the contents of the file from inside docker directly to know if the problems lies with docker or your webserver. The problem could be an HTTP caching problem if the webserver is at fault.

@tcmal

This comment has been minimized.

Show comment
Hide comment
@tcmal

tcmal Dec 15, 2015

Hi,

I checked and they are updating on the docker so it must be Apache.

Is this right?

/etc/apache2/sites-available/000-default.conf 

<VirtualHost *:80>
        # ...

        EnableSendfile Off
        EnableMMAP Off

        CustomLog ${APACHE_LOG_DIR}/access.log combined

        # ...
</VirtualHost>

tcmal commented Dec 15, 2015

Hi,

I checked and they are updating on the docker so it must be Apache.

Is this right?

/etc/apache2/sites-available/000-default.conf 

<VirtualHost *:80>
        # ...

        EnableSendfile Off
        EnableMMAP Off

        CustomLog ${APACHE_LOG_DIR}/access.log combined

        # ...
</VirtualHost>
@betoharres

This comment has been minimized.

Show comment
Hide comment
@betoharres

betoharres Mar 16, 2016

I'm facing this problem too, none of the suggestions worked for me. Any help? This kind of problem keeps me from developing in docker :( which I really want to

Im running docker machine in my OSX 10.11.4
My docker info:

Containers: 258
Images: 160
Server Version: 1.9.1
Storage Driver: aufs
 Root Dir: /mnt/sda1/var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 682
 Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.1.13-boot2docker
Operating System: Boot2Docker 1.9.1 (TCL 6.4.1); master : cef800b - Fri Nov 20 19:33:59 UTC 2015
CPUs: 1
Total Memory: 1.956 GiB
Name: default
ID: 624N:T5WM:NX7T:4RRP:GLAV:3TSL:OQXM:EEGM:WSUE:CSUL:AQQG:6UU6
Debug mode (server): true
 File Descriptors: 28
 Goroutines: 46
 System Time: 2016-03-16T18:43:53.655167222Z
 EventsListeners: 0
 Init SHA1:
 Init Path: /usr/local/bin/docker
 Docker Root Dir: /mnt/sda1/var/lib/docker
Username: betoharres
Registry: https://index.docker.io/v1/
Labels:
 provider=virtualbox

docker version:

Client:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.5.1
 Git commit:   a34a1d5
 Built:        Sat Nov 21 00:49:19 UTC 2015
 OS/Arch:      darwin/amd64

Server:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.3
 Git commit:   a34a1d5
 Built:        Fri Nov 20 17:56:04 UTC 2015
 OS/Arch:      linux/amd64

I'm facing this problem too, none of the suggestions worked for me. Any help? This kind of problem keeps me from developing in docker :( which I really want to

Im running docker machine in my OSX 10.11.4
My docker info:

Containers: 258
Images: 160
Server Version: 1.9.1
Storage Driver: aufs
 Root Dir: /mnt/sda1/var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 682
 Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.1.13-boot2docker
Operating System: Boot2Docker 1.9.1 (TCL 6.4.1); master : cef800b - Fri Nov 20 19:33:59 UTC 2015
CPUs: 1
Total Memory: 1.956 GiB
Name: default
ID: 624N:T5WM:NX7T:4RRP:GLAV:3TSL:OQXM:EEGM:WSUE:CSUL:AQQG:6UU6
Debug mode (server): true
 File Descriptors: 28
 Goroutines: 46
 System Time: 2016-03-16T18:43:53.655167222Z
 EventsListeners: 0
 Init SHA1:
 Init Path: /usr/local/bin/docker
 Docker Root Dir: /mnt/sda1/var/lib/docker
Username: betoharres
Registry: https://index.docker.io/v1/
Labels:
 provider=virtualbox

docker version:

Client:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.5.1
 Git commit:   a34a1d5
 Built:        Sat Nov 21 00:49:19 UTC 2015
 OS/Arch:      darwin/amd64

Server:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.3
 Git commit:   a34a1d5
 Built:        Fri Nov 20 17:56:04 UTC 2015
 OS/Arch:      linux/amd64
@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Mar 16, 2016

Contributor

@betoharres Either don't use virtualbox, or disable sendfile in apache/nginx.

Contributor

cpuguy83 commented Mar 16, 2016

@betoharres Either don't use virtualbox, or disable sendfile in apache/nginx.

@betoharres

This comment has been minimized.

Show comment
Hide comment
@betoharres

betoharres Mar 16, 2016

@cpuguy83 thanks, disabling sendfile didn't worked at all, now the files sometimes changes and sometimes changes back to previous state when reloading the page without updating to the newest changes, urgh

@cpuguy83 thanks, disabling sendfile didn't worked at all, now the files sometimes changes and sometimes changes back to previous state when reloading the page without updating to the newest changes, urgh

@stcalica

This comment has been minimized.

Show comment
Hide comment
@stcalica

stcalica Apr 14, 2016

I just have a node express app running in docker using docker-compose volume to map my html files to the container. Getting the same behavior. It is not updating in the docker container. i did the same with vim and add set nobackup and other settings. Does anyone have any updates on this?

I just have a node express app running in docker using docker-compose volume to map my html files to the container. Getting the same behavior. It is not updating in the docker container. i did the same with vim and add set nobackup and other settings. Does anyone have any updates on this?

@moneal

This comment has been minimized.

Show comment
Hide comment
@moneal

moneal Apr 14, 2016

I'm having the same issue with phpstorm and docker via virtualbox. The apache sendfile change didn't seem to help. I've also tried adjusting the save settings in phpstprm and no change.

moneal commented Apr 14, 2016

I'm having the same issue with phpstorm and docker via virtualbox. The apache sendfile change didn't seem to help. I've also tried adjusting the save settings in phpstprm and no change.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Apr 14, 2016

Member

@stcalica @moneal you may be interested in https://blog.docker.com/2016/03/docker-for-mac-windows-beta/, which gets rid of VirtualBox and resolves these kind of issues

Member

thaJeztah commented Apr 14, 2016

@stcalica @moneal you may be interested in https://blog.docker.com/2016/03/docker-for-mac-windows-beta/, which gets rid of VirtualBox and resolves these kind of issues

@daithi-coombes

This comment has been minimized.

Show comment
Hide comment
@daithi-coombes

daithi-coombes Apr 14, 2016

I'm having the same issue.
host: Debian 8 (jessie)
docker: Docker version 1.10.3, build 20f81dd

docker-compose.yml:

web:
  build: .
  links:
    - mysql
  environment:
    - WORDPRESS_DB_PASSWORD=****
    - WORDPRESS_DB_USER=root
    - WORDPRESS_DB_NAME=asdf
  ports:
    - 0.0.0.0:8080:80
  working_dir: /var/www/html
  volumes:
    - ./plugins:/var/www/html/wp-content/plugins
    - ./themes:/var/www/html/wp-content/themes
mysql:
  image: mysql
  environment:
    - MYSQL_ROOT_PASSWORD=****
    - MYSQL_DATABASE=asdf

Dockerfile:

FROM wordpress

RUN echo 'EnableSendfile Off' >> /etc/apache2/apache2.conf

This issue persists on both Atom and SublimeText3 and is killing me. I've added the EnableSendfile Off to apache conf, which has reduced the waiting time from a few mins to about 1 minute.

If, however, I grep the edited file from within the docker container then the changes will take effect instantly. So my workflow now is:

  1. make changes on host
  2. in container run cat path_to_file | grep 'foo'

From here I'm lost as to a solution (barely understand inode and filesystems etc)

I'm having the same issue.
host: Debian 8 (jessie)
docker: Docker version 1.10.3, build 20f81dd

docker-compose.yml:

web:
  build: .
  links:
    - mysql
  environment:
    - WORDPRESS_DB_PASSWORD=****
    - WORDPRESS_DB_USER=root
    - WORDPRESS_DB_NAME=asdf
  ports:
    - 0.0.0.0:8080:80
  working_dir: /var/www/html
  volumes:
    - ./plugins:/var/www/html/wp-content/plugins
    - ./themes:/var/www/html/wp-content/themes
mysql:
  image: mysql
  environment:
    - MYSQL_ROOT_PASSWORD=****
    - MYSQL_DATABASE=asdf

Dockerfile:

FROM wordpress

RUN echo 'EnableSendfile Off' >> /etc/apache2/apache2.conf

This issue persists on both Atom and SublimeText3 and is killing me. I've added the EnableSendfile Off to apache conf, which has reduced the waiting time from a few mins to about 1 minute.

If, however, I grep the edited file from within the docker container then the changes will take effect instantly. So my workflow now is:

  1. make changes on host
  2. in container run cat path_to_file | grep 'foo'

From here I'm lost as to a solution (barely understand inode and filesystems etc)

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Apr 14, 2016

Contributor

These are not issues with Docker.
Mounting individual files is tricky.
A file path and a file inode are different things.

A file path is more akin to a git tag where as an inode is like a commit hash.
Just like a git tag can be changed to point to a different hash, so can a file path point to a new inode.
A commit hash is definite, it doesn't change, likewise for an inode.
So, file paths are just references to an inode. These references can and do change depending on the actions you perform (e.g. mv /foo /bar defeferences /foo from the underlying inode and creates a new reference /bar to that same inode)

When you bind mount a path (file or dir), you are mounting an inode to a new location (kind of like creating a new git tag pointing to the same commit hash).

If you do something like move a bind-mounted dir to a new location, and then replace it with another dir, the bind-mount will follow the original dir, same for files.
With many editors, the editor will write to a temp file, then replace the real file with the temp file on save, in effect dereferencing the inode (like pointing a git tag to a different commit) from the given path and a new inode taking the reference.
Since mounts follow the inode, not the path reference, on the host side you are reading/writing to a new inode, on the container side you are reading/writing from the original inode.

Contributor

cpuguy83 commented Apr 14, 2016

These are not issues with Docker.
Mounting individual files is tricky.
A file path and a file inode are different things.

A file path is more akin to a git tag where as an inode is like a commit hash.
Just like a git tag can be changed to point to a different hash, so can a file path point to a new inode.
A commit hash is definite, it doesn't change, likewise for an inode.
So, file paths are just references to an inode. These references can and do change depending on the actions you perform (e.g. mv /foo /bar defeferences /foo from the underlying inode and creates a new reference /bar to that same inode)

When you bind mount a path (file or dir), you are mounting an inode to a new location (kind of like creating a new git tag pointing to the same commit hash).

If you do something like move a bind-mounted dir to a new location, and then replace it with another dir, the bind-mount will follow the original dir, same for files.
With many editors, the editor will write to a temp file, then replace the real file with the temp file on save, in effect dereferencing the inode (like pointing a git tag to a different commit) from the given path and a new inode taking the reference.
Since mounts follow the inode, not the path reference, on the host side you are reading/writing to a new inode, on the container side you are reading/writing from the original inode.

@daithi-coombes

This comment has been minimized.

Show comment
Hide comment
@daithi-coombes

daithi-coombes Apr 14, 2016

@cpuguy83 that explains inode a lot better than anything I've come across past few days - thanks ;)

But I'm still lost as to how greping the changed file triggers the changes. Also note that if I grep a string that doesn't exist in the file then the problem persists - this is very weird behavior for me. Also if I cat the file, without grep, I can see the changes but the issue persists. I've also tried tailing the file with no joy.

So by piping cat into grep, and only when grep finds a result, something is triggered that lets apache know that the file has updated. Maybe a flag or something?

@cpuguy83 that explains inode a lot better than anything I've come across past few days - thanks ;)

But I'm still lost as to how greping the changed file triggers the changes. Also note that if I grep a string that doesn't exist in the file then the problem persists - this is very weird behavior for me. Also if I cat the file, without grep, I can see the changes but the issue persists. I've also tried tailing the file with no joy.

So by piping cat into grep, and only when grep finds a result, something is triggered that lets apache know that the file has updated. Maybe a flag or something?

@daithi-coombes

This comment has been minimized.

Show comment
Hide comment
@daithi-coombes

daithi-coombes Apr 15, 2016

I have to report that my previous hack of running cat path_to_file | grep 'foobar' was a false positive. I got 2 hours today of docker working and then its back to make a change, reload, reload, reload, reload...

I don't see how this issue isn't considered a serious blocker if it means that nobody can ever develop on Docker.

I have to report that my previous hack of running cat path_to_file | grep 'foobar' was a false positive. I got 2 hours today of docker working and then its back to make a change, reload, reload, reload, reload...

I don't see how this issue isn't considered a serious blocker if it means that nobody can ever develop on Docker.

@moneal

This comment has been minimized.

Show comment
Hide comment
@moneal

moneal Apr 15, 2016

I updated to the docker beta on mac and so far running grep has always shown the correct update but apache is still not seeing the change.

moneal commented Apr 15, 2016

I updated to the docker beta on mac and so far running grep has always shown the correct update but apache is still not seeing the change.

@moneal

This comment has been minimized.

Show comment
Hide comment
@moneal

moneal Apr 15, 2016

Turns out it was the opcache settings in the wordpress image I was using. Adding the below lines to my Dockerfile fixed the problem for me (no problems in an hour). Now the files are showing in the container and when previewing with apache.

FROM wordpress:latest
RUN rm -rf /usr/local/etc/php/conf.d/opcache-recommended.ini

moneal commented Apr 15, 2016

Turns out it was the opcache settings in the wordpress image I was using. Adding the below lines to my Dockerfile fixed the problem for me (no problems in an hour). Now the files are showing in the container and when previewing with apache.

FROM wordpress:latest
RUN rm -rf /usr/local/etc/php/conf.d/opcache-recommended.ini
@daithi-coombes

This comment has been minimized.

Show comment
Hide comment
@daithi-coombes

daithi-coombes Apr 19, 2016

@moneal +1 - that seems to have done the trick

@moneal +1 - that seems to have done the trick

@seeekr

This comment has been minimized.

Show comment
Hide comment
@seeekr

seeekr May 25, 2016

Came here because I had a similar issue, stating my solution for others searching on the web:

If nginx is not sending updated static files, make sure to add

sendfile off;

to your nginx config. Does the trick for me. (It's probably not something you want to leave that way for heavy-traffic production sites with lots of static content, but only for development using mounted volumes.)

seeekr commented May 25, 2016

Came here because I had a similar issue, stating my solution for others searching on the web:

If nginx is not sending updated static files, make sure to add

sendfile off;

to your nginx config. Does the trick for me. (It's probably not something you want to leave that way for heavy-traffic production sites with lots of static content, but only for development using mounted volumes.)

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 May 25, 2016

Contributor

@seeekr or use Docker4Mac and it also just works.

Contributor

cpuguy83 commented May 25, 2016

@seeekr or use Docker4Mac and it also just works.

@seeekr

This comment has been minimized.

Show comment
Hide comment
@seeekr

seeekr May 25, 2016

@cpuguy83 True, that's something that works really well in Docker for Mac. But other things don't (yet), so for some things and for some folks Docker Toolbox / Docker Machine-based Docker on Mac is still necessary, for now. (I've hit roadblocks with Docker Compose (known DNS resolver issue) as well as with rancher/k8s (though that one may have been simply due to my inexperience with those technologies and less about D4M).)

seeekr commented May 25, 2016

@cpuguy83 True, that's something that works really well in Docker for Mac. But other things don't (yet), so for some things and for some folks Docker Toolbox / Docker Machine-based Docker on Mac is still necessary, for now. (I've hit roadblocks with Docker Compose (known DNS resolver issue) as well as with rancher/k8s (though that one may have been simply due to my inexperience with those technologies and less about D4M).)

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah May 25, 2016

Member

k8s isn't fully compatible with Docker 1.11 yet, also k8s does some things outside of the "api" for Docker (direct access to files in /var/lib/docker, working around standard networking in docker), which is, well, not always the most reliable approach.

Member

thaJeztah commented May 25, 2016

k8s isn't fully compatible with Docker 1.11 yet, also k8s does some things outside of the "api" for Docker (direct access to files in /var/lib/docker, working around standard networking in docker), which is, well, not always the most reliable approach.

@buildog

This comment has been minimized.

Show comment
Hide comment
@buildog

buildog May 27, 2016

I'm experiencing a similar issue on Mac 10.11.4, everything work fine except the volume update, while the same config is working fine on ubuntu.

Docker version 1.10.0, build 590d5108

Dockerfile:

FROM node:5-slim

# create a layer for dependencies so they're cached
RUN mkdir -p /reactjs
WORKDIR /reactjs
COPY package.json package.json
RUN npm install
RUN npm install -g concurrently

# copy the source and build
COPY . /reactjs

# add user and transfer ownership
RUN useradd -d /reactjs adrien \
    && chown -R adrien:adrien /reactjs \
    && chmod -R g+rw /reactjs

Docker-compose:

web:
    image: 1a35b1ab7c06
    command: npm run dev:watch
    volumes:
        - ./src:/reactjs/src
    ports:
        - "8083:8083"
    environment:
        NODE_ENV: development
        PORT: "8083"

buildog commented May 27, 2016

I'm experiencing a similar issue on Mac 10.11.4, everything work fine except the volume update, while the same config is working fine on ubuntu.

Docker version 1.10.0, build 590d5108

Dockerfile:

FROM node:5-slim

# create a layer for dependencies so they're cached
RUN mkdir -p /reactjs
WORKDIR /reactjs
COPY package.json package.json
RUN npm install
RUN npm install -g concurrently

# copy the source and build
COPY . /reactjs

# add user and transfer ownership
RUN useradd -d /reactjs adrien \
    && chown -R adrien:adrien /reactjs \
    && chmod -R g+rw /reactjs

Docker-compose:

web:
    image: 1a35b1ab7c06
    command: npm run dev:watch
    volumes:
        - ./src:/reactjs/src
    ports:
        - "8083:8083"
    environment:
        NODE_ENV: development
        PORT: "8083"

@magbicaleman

This comment has been minimized.

Show comment
Hide comment
@magbicaleman

magbicaleman Aug 17, 2016

I have Docker for Mac. Adding a new file with code works instantly. Yay, right? When I modify the file with Atom text editor, the changes take around a minute.

The image I'm using is the official Wordpress image, even if Docker for Mac fixes the inode stuff, it wouldn't seem like it. Why? Because there seems to be a caching of files as @moneal pointed out earlier.

For example make a new file:

// my-plugins/stopper.php
<?php
  die('edit me 1');
// This will happen instantly when you reload
// my-plugins/stopper.php
<?php
  die('edit me 2');
// This will not instantly change, we changed it to number 2
// my-plugins/stopper.php
<?php
  opcache_reset();
  die('edit me 3');
// This will work instantly, we changed it to number 3

I'd hate to run the following, removing some intended functionality for production servers, is there a work around for local development, a hook or something that only runs locally.

RUN rm -rf /usr/local/etc/php/conf.d/opcache-recommended.ini


I ended making a file in mu-plugins/_.php with the following code

<?php
  opcache_reset();

File modifications seem to be appearing instantly from what I can tell. This is probably not efficient or desired, but I'll just end up adding it to the git-ignore. Hope this helps.

magbicaleman commented Aug 17, 2016

I have Docker for Mac. Adding a new file with code works instantly. Yay, right? When I modify the file with Atom text editor, the changes take around a minute.

The image I'm using is the official Wordpress image, even if Docker for Mac fixes the inode stuff, it wouldn't seem like it. Why? Because there seems to be a caching of files as @moneal pointed out earlier.

For example make a new file:

// my-plugins/stopper.php
<?php
  die('edit me 1');
// This will happen instantly when you reload
// my-plugins/stopper.php
<?php
  die('edit me 2');
// This will not instantly change, we changed it to number 2
// my-plugins/stopper.php
<?php
  opcache_reset();
  die('edit me 3');
// This will work instantly, we changed it to number 3

I'd hate to run the following, removing some intended functionality for production servers, is there a work around for local development, a hook or something that only runs locally.

RUN rm -rf /usr/local/etc/php/conf.d/opcache-recommended.ini


I ended making a file in mu-plugins/_.php with the following code

<?php
  opcache_reset();

File modifications seem to be appearing instantly from what I can tell. This is probably not efficient or desired, but I'll just end up adding it to the git-ignore. Hope this helps.

@unmultimedio

This comment has been minimized.

Show comment
Hide comment
@unmultimedio

unmultimedio Nov 15, 2016

In case anyone here searching for the problem, and having a Dockerized Rails5 app, this might be helpful: http://stackoverflow.com/questions/37144122/dockerized-rails-5-rc1-application-not-picking-up-updates-to-controllers-and-mod

UPDATE
Not really a Docker expert, but I've found that when using docker-compose and faced with the issue, what I used to do was docker-compose restart [service_name], which wouldn't solve the problem. When I faced the problem started doing docker-compose stop [service_name_or_blank] and then again docker-compose up. Haven't face the issue again since.

unmultimedio commented Nov 15, 2016

In case anyone here searching for the problem, and having a Dockerized Rails5 app, this might be helpful: http://stackoverflow.com/questions/37144122/dockerized-rails-5-rc1-application-not-picking-up-updates-to-controllers-and-mod

UPDATE
Not really a Docker expert, but I've found that when using docker-compose and faced with the issue, what I used to do was docker-compose restart [service_name], which wouldn't solve the problem. When I faced the problem started doing docker-compose stop [service_name_or_blank] and then again docker-compose up. Haven't face the issue again since.

@JeNeSuisPasDave

This comment has been minimized.

Show comment
Hide comment
@JeNeSuisPasDave

JeNeSuisPasDave Nov 26, 2016

I've worked up a solution that uses rsync to push local file and directory changes to the data volume (either classic data volumes or data volume containers). This works well for Docker Toolbox scenarios on a Mac and should also work for Windows folks via Bash for Windows. See the examples at https://github.com/JeNeSuisPasDave/asd-sync-src-to-container and a discussion of the issue at https://www.develves.net/blogs/asd/2016-11-26-still-using-docker-toolbox/. I've been using this mechanism on a variety of projects to great success.

JeNeSuisPasDave commented Nov 26, 2016

I've worked up a solution that uses rsync to push local file and directory changes to the data volume (either classic data volumes or data volume containers). This works well for Docker Toolbox scenarios on a Mac and should also work for Windows folks via Bash for Windows. See the examples at https://github.com/JeNeSuisPasDave/asd-sync-src-to-container and a discussion of the issue at https://www.develves.net/blogs/asd/2016-11-26-still-using-docker-toolbox/. I've been using this mechanism on a variety of projects to great success.

@dacz

This comment has been minimized.

Show comment
Hide comment
@dacz

dacz Dec 16, 2016

Maybe little bit later but I discovered, that changes on php (wordpress development) in docker done in Atom editor do not propagate to container (changes not visible until container restart), using sublime text editor is ok and working.

Probably different way Atom editor and Sublime work with files as mentioned by @cpuguy83 .

dacz commented Dec 16, 2016

Maybe little bit later but I discovered, that changes on php (wordpress development) in docker done in Atom editor do not propagate to container (changes not visible until container restart), using sublime text editor is ok and working.

Probably different way Atom editor and Sublime work with files as mentioned by @cpuguy83 .

@klaszlo

This comment has been minimized.

Show comment
Hide comment
@klaszlo

klaszlo Dec 25, 2016

@cpuguy83: It could be easily workarounded inside docker.

Instead of watching the file's inode, it should be watching the parent's directory inode, then resolve the file change from there.

Or at the VERY LEAST a simple warning should be displayed, when a docker run is issued.

klaszlo commented Dec 25, 2016

@cpuguy83: It could be easily workarounded inside docker.

Instead of watching the file's inode, it should be watching the parent's directory inode, then resolve the file change from there.

Or at the VERY LEAST a simple warning should be displayed, when a docker run is issued.

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Dec 25, 2016

Contributor

@klaslo docker does not monitor anything.in this regard. It relies on the kernel, and the kernel does not support this.
The right place to "fix" this is the kernel.
We could introduce hacks, but I'm not sure it's worth it, at least not right now.

Contributor

cpuguy83 commented Dec 25, 2016

@klaslo docker does not monitor anything.in this regard. It relies on the kernel, and the kernel does not support this.
The right place to "fix" this is the kernel.
We could introduce hacks, but I'm not sure it's worth it, at least not right now.

@klaszlo

This comment has been minimized.

Show comment
Hide comment
@klaszlo

klaszlo Dec 25, 2016

@cpuguy83:

  1. displaying a warning does not require kernel support
  2. monitoring how many path points to an inode is already supported.
    So overwriting a file with a new one is detectable.

docker run ... -d /home/user/data/something.txt:/data/something.txt
echo "new content" > /home/user/data/something2.txt
mv something2.txt something.txt

Then the inode table have one row less, so it is detectable. Docker already knows the original path (/home/user/data/something.txt), also knows the inside path (/var/lib/docker/volume/.../something.txt),
so he can check and remount the new path.

Or file path in volumes should be disabled. It is more trouble than good.

klaszlo commented Dec 25, 2016

@cpuguy83:

  1. displaying a warning does not require kernel support
  2. monitoring how many path points to an inode is already supported.
    So overwriting a file with a new one is detectable.

docker run ... -d /home/user/data/something.txt:/data/something.txt
echo "new content" > /home/user/data/something2.txt
mv something2.txt something.txt

Then the inode table have one row less, so it is detectable. Docker already knows the original path (/home/user/data/something.txt), also knows the inside path (/var/lib/docker/volume/.../something.txt),
so he can check and remount the new path.

Or file path in volumes should be disabled. It is more trouble than good.

@aiphee

This comment has been minimized.

Show comment
Hide comment
@aiphee

aiphee Aug 4, 2017

@cpuguy83 s answer helped me. Now i do this:

nano copy_authorized_container_keys # edit copy of the file
chattr -i authorized_container_keys # remove RO status
cat copy_authorized_container_keys > authorized_container_keys # rewrite file with copy inplace
chattr +i authorized_container_keys # return RO status (so the inode wont break)

aiphee commented Aug 4, 2017

@cpuguy83 s answer helped me. Now i do this:

nano copy_authorized_container_keys # edit copy of the file
chattr -i authorized_container_keys # remove RO status
cat copy_authorized_container_keys > authorized_container_keys # rewrite file with copy inplace
chattr +i authorized_container_keys # return RO status (so the inode wont break)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment