Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File not changing inside docker container after updating from 2.1.0.5 to 2.2.0.0 #5530

Open
thesupad opened this issue Jan 22, 2020 · 216 comments
Open

Comments

@thesupad
Copy link

@thesupad thesupad commented Jan 22, 2020

  • I have tried with the latest version of my channel (Stable or Edge)
  • I have uploaded Diagnostics
  • Diagnostics ID: 0B5E8EE6-4822-4D32-BE64-EFD8FDFBF13F/20200122095329

Expected behavior

When a file is changed in the windows filesystem, it should be changed inside the container.

Actual behavior

When a file is changed in the windows filesystem, it's not changed, it's the same as when the container started. The file's modification time is not changing as well.

Information

  • Windows Version: 10
  • Docker Desktop Version: 2.2.0.0
  • Are you running inside a virtualized Windows e.g. on a cloud server or on a mac VM: no

Steps to reproduce the behavior

  1. start a container with shared folder
  2. edit in windows a file in the shared folder
  3. the file is not changed in the container
@kohlerdominik

This comment has been minimized.

Copy link

@kohlerdominik kohlerdominik commented Jan 22, 2020

I can confirm this. Same Setup, running as "Linux Containers" if this is somewhat helpfull.

Does anyone already have a solution or workaround?

@Brainshaker95

This comment has been minimized.

Copy link

@Brainshaker95 Brainshaker95 commented Jan 22, 2020

My workaround was downgrading to 2.1.0.5

@SamuelReinfelder

This comment has been minimized.

Copy link

@SamuelReinfelder SamuelReinfelder commented Jan 22, 2020

How do you downgrade?

@argentinaluiz

This comment has been minimized.

Copy link

@argentinaluiz argentinaluiz commented Jan 22, 2020

Me too, I am downgrading to 2.1.0.5

@Xammie

This comment has been minimized.

Copy link

@Xammie Xammie commented Jan 22, 2020

I'm also downgrading.

@SamuelReinfelder This is the download link for 2.1.0.5
https://download.docker.com/win/stable/40693/Docker%20Desktop%20Installer.exe

@mightyfineyall

This comment has been minimized.

Copy link

@mightyfineyall mightyfineyall commented Jan 22, 2020

Same issue here - running a few gulp tasks using nginx / urbantrout's Craft CMS image / mysql, it will compile once and then never change the file. Obviously can't develop when your files don't update.

@paslandau

This comment has been minimized.

Copy link

@paslandau paslandau commented Jan 22, 2020

Same issue. In addition it also seemed to interfere with files loaded in my local IDE (PHPStorm):

  • files were unable to be loaded by the IDE
  • in one case, the contents of a file I was modifying was completely deleted
@andrei-dascalu

This comment has been minimized.

Copy link

@andrei-dascalu andrei-dascalu commented Jan 22, 2020

Same issue, the only thing that helped was downgrading. Any previous issue is ok

@mat007

This comment has been minimized.

Copy link

@mat007 mat007 commented Jan 22, 2020

Sorry for the inconvenience, uploading more diagnostics and/or specific reproduction steps would help a lot.
Thanks!

@argentinaluiz

This comment has been minimized.

Copy link

@argentinaluiz argentinaluiz commented Jan 22, 2020

Hi @mat007 , for me, I only set in docker-compose.yaml:

volumes:
    - .:/var/www

All changes in any files doesnt persist in container.
Thanks!

@frankmohaupt

This comment has been minimized.

Copy link

@frankmohaupt frankmohaupt commented Jan 22, 2020

Steps to reproduce:
Change a shared file on your host.
Exec in the docker container
check the file in the container that you have changed in your host before.

RESULT: The changes you have made to the file in the host are not "transferred" to the container.

If you restart the docker Environment the changes are there.

@BenjaminKalle

This comment has been minimized.

Copy link

@BenjaminKalle BenjaminKalle commented Jan 22, 2020

Same Issue;
Few differences / Inclusions though;

  • The issue was removed for the better part of an Hour after updating windows to it's latest version.
  • The issue was, for a short time after reappearance, fixable when forcing your cache to clear.
  • If you wait for the better part of 5 minutes the changes to the files will be passed through to the container.
  • There where differences between different types of files in the extend and/or timing of the changes.
    (E.G.: changes to an HTML file 'seemed' to be quicker in changing the result instantly while any changes to PHP files took longer, but eventually they take the same amount of time before the changes are visible from within the container.)
@rfay

This comment has been minimized.

Copy link

@rfay rfay commented Jan 22, 2020

I also find the converse: Files changed in the container do not show up on the host. The file mount consistency of 2.2.0.0 is not working out.

@kasp3r

This comment has been minimized.

Copy link

@kasp3r kasp3r commented Jan 23, 2020

Same issue, but I noticed, that file in container starts being updated after I nano file.html inside container.

@blackangelxl

This comment has been minimized.

Copy link

@blackangelxl blackangelxl commented Jan 23, 2020

My Solution:

Set ":consistent" command behind the volume path

Example:
"./www/:/usr/share/nginx/html/www:consistent"

Volume ":cached" is now by default...

@KevinVonJocoon

This comment has been minimized.

Copy link

@KevinVonJocoon KevinVonJocoon commented Jan 23, 2020

We're just experiencing similar problems.
My colleagues are at version 2.2+ and I just stayed at 2.1.0.5.

My observation:

We're having a php composer project where we're symlinking local git repositories into the vendor dir of composer, e.g.:
src/vendor/codename/abc -> src/modules/codename/abc

(changed for human readability and to get the concept of mixing static composer deps with local dev projects using autoloading)

The directories are volume-bound with the whole project dir, so the "symlink" is a local NTFS junction automatically created by composer. We tested volume bindings with no option (default is :cached for the major project files), :cached, :delegated and :consistent - while we're still using :delegated and for local dev DBs, profiler data and logs.

While this still works on my Docker CE 2.1.0.5 on Windows, we just found out manually opening the same PHP file via nano in the container renders different results on later versions of Docker on Windows - though, in theory, it is the same file:

  • src/modules/codename/abc/backend/class/app.php
  • src/vendor/codename/abc/backend/class/app.php

The latter one does not reflect changes performed on the host, while the first one does.
This just came to my mind: the new inotify-capability might not reflect NTFS junctions, local symlinks or something inbetween - which might be even a Windows issue?

@ventayol

This comment has been minimized.

Copy link

@ventayol ventayol commented Jan 23, 2020

I'm having the same issue, I do change in the host but the changes are not reflected in the docker container.

Information
Windows Version: 10
Docker Desktop Version: 2.2.0.0
WSL Version: 2.0

Restarting the docker containers actually updates the content of the files.

@ventayol

This comment has been minimized.

Copy link

@ventayol ventayol commented Jan 23, 2020

I tried @blackangelxl solution of using :consistent but it doesn't work, changes are not reflected in the docker container.

@stephen-turner stephen-turner mentioned this issue Jan 23, 2020
2 of 2 tasks complete
@Aex

This comment has been minimized.

Copy link

@Aex Aex commented Jan 23, 2020

Chiming in on this issue, I've noticed my files are not always reflecting changes in the volume as well. I've exec'd into the container and noticed that the date on the file changes to Jan 1, 1970 when the file refuses to update.

Inside the container, if I touch the file, it will then update based on the changes in my volume and this may persist for a short time (shows the correct date at this time) before it stops updating again and the date changes to Jan 1, 1970.

total 0
drwxrwxrwx 1 root root 0 Jan 23 19:22 ./
drwxrwxrwx 1 root root 4096 Jan 23 01:01 ../
-rwxr-xr-x 1 root root 343 Jan 1 1970 data.cfc*
-rwxr-xr-x 1 root root 45 Jan 23 19:38 index.cfm*

@stephen-turner

This comment has been minimized.

Copy link

@stephen-turner stephen-turner commented Jan 24, 2020

Thanks, @Aex, that's a good observation. It may indicate that this is the same as #5543.

I see that this issue has lots of "me too"s, but I haven't been to reproduce it. Does anyone have a simple, reliable repro with step by step instructions?

@stephen-turner

This comment has been minimized.

Copy link

@stephen-turner stephen-turner commented Jan 24, 2020

@KevinVonJocoon I think your issue with junctions is different. I've created a new issue for it, #5582.

@kohlerdominik

This comment has been minimized.

Copy link

@kohlerdominik kohlerdominik commented Jan 24, 2020

Hi @stephen-turner

It might be an issue, that a lot of people downgraded because it's critical work environment, as for me.

However, in our team we have different setups, while one works and the other doesn't. Therefore i try to describe the difference and maybe you can guess where the issue comes from:

Working Setup:
using mingw for bash executing, calling docker-compose on windows (version 1.24.1 because of #7169)

Not Working Setup
Using WSL (Debian), calling docker-compose binary in linux (version 1.23.1, because forgot to update this), and using the exposed deamon (localhost:2375) on windows

everything else should be the same: using a compose-file with multiple containers. I noticed the bug in our php-container (php:7.3.4-fpm-alpine3.9), didn't test other containers. volumes are mounted as - ${APP_CODE_PATH}:/var/www:cached

@johnrom

This comment has been minimized.

Copy link

@johnrom johnrom commented Mar 27, 2020

Can confirm, I also use a data container and have this issue.

@djs55

This comment has been minimized.

Copy link

@djs55 djs55 commented Mar 27, 2020

@kamermans thanks a lot for the compose file, code and instructions -- very helpful! There does indeed seem to be a problem with the use of volumes_from. Looking at the logs Desktop thinks the tester container has no volumes at all, so it only watches the directory while the data container is running. When data exits, the watch is removed. This is very exciting! I'll dig into the code this weekend because we have an edge release due out in a few days and I'd love to fix this.

@kamermans

This comment has been minimized.

Copy link

@kamermans kamermans commented Mar 27, 2020

Sure, no problem, happy to help! Let me know if there's anything else I can do to assist.

@thesupad

This comment has been minimized.

Copy link
Author

@thesupad thesupad commented Mar 28, 2020

You are beyond my comprehension of docker but i think i am in the same spot :
image
As you can see the application container is exited and that's normal from my point of view.

More recent version of laradock do not have this behavior, which would explain why they are not so many of us having the problem.

@kamermans

This comment has been minimized.

Copy link

@kamermans kamermans commented Mar 28, 2020

Yes @thesupad, that's the same scenario that I've identified. For now, you can work around this bug by starting another container that maps your local application directory like this:

docker run --rm -d -v your-local-dir:/app alpine tail -f .dockerenv

(replace your-local-dir with your dir - for Laravel I imagine it would be your project root)

That container will stay open doing nothing but keeping your volume mount alive :)

@thesupad

This comment has been minimized.

Copy link
Author

@thesupad thesupad commented Mar 28, 2020

Thx for the workaround, but for now i'll stay on previous version :)

@djs55

This comment has been minimized.

Copy link

@djs55 djs55 commented Mar 28, 2020

@kamermans @thesupad @johnrom @ouwejan I've got a prototype fix for this particular issue. I've made a test build based on stable 2.2.0.5: https://download-stage.docker.com/win/stable/43834/Docker%20Desktop%20Installer.exe . When I docker-compose up @kamermans 's example and wait for the first container to exit, I now see the path is still being watched (previously the watch would be removed and the cache would become stale):

PS > docker run -it -v /run:/run djs55/grpcfuse-control:2.2.0.4 debug
{"InjectFilesystemEvents":true,"CacheInvalidation":true,"Watches":[{"Path":"C:\\Users\\djs\\bugs\\5530\\docker-win-fsbug\\test\\","NumInvalidateAlls":1,"HeartbeatOK":true}],"NumInjected":18,"EventServerPingOK":true}

If you have time to try it let me know if it works (or not). If it doesn't please upload a diagnostics report and I'll take a look at it. Thanks again for your patience with this one!

@ouwejan

This comment has been minimized.

Copy link

@ouwejan ouwejan commented Mar 28, 2020

@djs55 I'm sorry, no change for me. To be sure I've installed it correctly: Version 2.2.0.5(43834).
Hopefully a fix for the others.

@kamermans

This comment has been minimized.

Copy link

@kamermans kamermans commented Mar 29, 2020

Thanks for the update @djs55 - it solves the data container problem for me!

@djs55

This comment has been minimized.

Copy link

@djs55 djs55 commented Mar 30, 2020

@ouwejan I had a fresh look at the logs you provided. Initially I see containers starting and stopping and the expected calls to register both external ports and shared volumes:

vpnkitExposer.Add(b112b2347bf495edaf85270d7b15a3c552989acca2db71acc72b9c6f9c3cf25f, [TCP 0.0.0.0:5432 -> 127.0.0.1:5432])
...
grpcfuseSharer.Add(a1cb3534367a850198d84f4c5805eb3acc53fadc9f7bcd4fb444260b6bd5d4f1, [])

but later I see a container restart which coincided with a lot of events being transmitted to the VM which caused the cache to be invalidated (this is not a surprise by itself):

FS_Q_OVERFLOW detected
invalidating all caches
grpcfuseSharer.Add(fc123eaf86b630f48db8e2a39084062d6ed4acc5e9ec8486df1f5a4af06c6590,
invalidating all caches

and then after that I see ports being exposed but nothing more about volume sharing. I didn't notice this originally because there are no explicit errors in the main log, but the subsequent lack of volume sharing logs is definitely a problem. I suspected the thread handling the filesystem watch registrations and unregistrations has deadlocked and confirmed this by finding the error in the diagnostics logs where the registration list timed-out (because the mutex was held). I suspect the cause may be due to invalidating caches which were already in the process of being invalidated ... but I need to investigate more. Anyway, I just wanted to say that this is a good lead so thanks again for the logs!

@ouwejan

This comment has been minimized.

Copy link

@ouwejan ouwejan commented Mar 30, 2020

@djs55 Thanks David, glad to be of help. As stated: I'm fairly new to Docker and the logs I've provided are generated using a project I'm working on at work. It's pretty complicated as far as I can tell. It's my only project at the moment so I'm afraid the only one I can use at this moment.
Thanks again for looking into this. I love being able to use Docker for Windows, thanks all for your efforts.

@nickwilde1990

This comment has been minimized.

Copy link

@nickwilde1990 nickwilde1990 commented Mar 31, 2020

@djs55 your test build of 2.2.0.5 fixed it for me (at least no loss of link after more than 24 hours).

@stephen-turner

This comment has been minimized.

Copy link

@stephen-turner stephen-turner commented Apr 2, 2020

@kamermans @thesupad @johnrom @ouwejan The volumes_from bug is fixed in 2.2.0.5.

@rabauss

This comment has been minimized.

Copy link

@rabauss rabauss commented Apr 3, 2020

Unfortunately for me the initial problem still exists in Docker 2.2.0.5 - downgrading to 2.1.0.5 again!
It is really annoying when file changes do not arrive in the container :-(

@mat007

This comment has been minimized.

Copy link

@mat007 mat007 commented Apr 3, 2020

@rabauss did you by chance capture a diagnostics?

@rabauss

This comment has been minimized.

Copy link

@rabauss rabauss commented Apr 3, 2020

@mat007 can you please give me a short instruction how to do that? this issue is quite huge...

@mat007

This comment has been minimized.

Copy link

@mat007 mat007 commented Apr 3, 2020

@rabauss from the Troubleshoot window which you can open from the systray menu, you click on «Run Diagnostics», this will run for a little while then you can upload it and copy its ID to paste it here.
Thanks a lot!

@stephen-turner

This comment has been minimized.

Copy link

@stephen-turner stephen-turner commented Apr 3, 2020

@rabauss Also please describe your exact reproduction steps. This bug is now fixed for most people. There may be a corner case we've missed (presumably there is), but we are unlikely to find it without a detailed repro. Thanks.

@idavidson

This comment has been minimized.

Copy link

@idavidson idavidson commented Apr 3, 2020

I am also seeing the same issue as @rabauss . It appears to work intermittently. If it doesn't work and you then go onto the container and touch the file it then starts working. The length of time it then works for has no consistency, could be one save and then same issue or alternatively multiple saves before stops again. Nothing reported in the logs at the time of issues either.

@djs55

This comment has been minimized.

Copy link

@djs55 djs55 commented Apr 3, 2020

@ouwejan I think I've found the cause of the deadlock triggered by a container restart which I could see evidence for in your logs. I have a candidate fix and, In case you have time to try it, I've made a prototype build here: https://download-stage.docker.com/win/edge/43996/Docker%20Desktop%20Installer.exe . If you do get a chance to try it, let me know if it's any better. If there are still problems, please upload another diagnostic report and I'll take another look. In any case, thanks again for all your help so far!

@rabauss

This comment has been minimized.

Copy link

@rabauss rabauss commented Apr 3, 2020

@djs55 looks good for me now - I will continue testing the prototype over the next few days. Quick test was successful without the initial problem!

@ouwejan

This comment has been minimized.

Copy link

@ouwejan ouwejan commented Apr 3, 2020

@djs55 David, It looks like your 43996-version did the trick! I'll be testing some more but so farr I haven't been able to reproduce any of the earlier errors. This version is working for me. Thank you very much!

@divad942 divad942 mentioned this issue Apr 4, 2020
@TheDutchCoder

This comment has been minimized.

Copy link

@TheDutchCoder TheDutchCoder commented Apr 4, 2020

This also happens on Mac on the latest release, it's incredibly frustrating. I used to use the edge release because it seemed to function better, but the latest release broke nginx.

@creocoder

This comment has been minimized.

Copy link

@creocoder creocoder commented Apr 4, 2020

Still not fixed by 2.2.0.5

@djs55

This comment has been minimized.

Copy link

@djs55 djs55 commented Apr 4, 2020

@TheDutchCoder the nginx/IPv6 issue is a different one -- if you'd like to try an early build with a candidate fix, see this comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.