Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow fine grained excludes configuration for Mutagen volume mounts #4621

Closed
majames opened this issue May 28, 2020 · 15 comments
Closed

Allow fine grained excludes configuration for Mutagen volume mounts #4621

majames opened this issue May 28, 2020 · 15 comments

Comments

@majames
Copy link

@majames majames commented May 28, 2020

Hi! I was really excited to see the announcement at DockerCon that Docker for MAC will now support Mutagen for more performant host volume mounting! We actually built a tool at the organisation I work out that does something very similar (using Watchman and docker cp/rm under the hood).

One killer feature that is currently missing is the ability to specify fine-grained excludes for the file sync list. I notice that there is limited support for excludes by creating a docker volume, however our mono-repo is quite large and contains many generated nested directories and, therefore, creating an individual volume for each of them is infeasible.

Are there plans to implement functionality that allow users to specify fine-grained includes/excludes? Something similar to a dockerignore or Watchman expressions would be great!

@stephen-turner
Copy link
Contributor

@stephen-turner stephen-turner commented Jun 1, 2020

@djs55 Is this option available in the Mutagen config file?

@djs55
Copy link
Contributor

@djs55 djs55 commented Jun 1, 2020

In more recent development builds e.g.https://desktop-stage.docker.com/mac/edge/45494/Docker.dmg (discussed originally in the context of symlinks #4595 (comment) ) it's possible to add paths to ignore to the global mutagen config file e.g.

$ cat ~/.mutagen.yml
sync:
  defaults:
    ignore:
      paths:
        - somethingtoignore
      vcs: true

Thanks for the suggestions about dockerignore and Watchman expressions -- I'll investigate them!

@majames
Copy link
Author

@majames majames commented Jun 1, 2020

Thanks for the quick response @djs55 !

Had a look at the comment you linked. Does that mean that Docker Desktop will use the ~/.mutagen.yml file when determining which files to sync? If so, then global ignores will probably be enough to unblock us from using this for now (although per directory ignores would be a nice feature!)

@djs55
Copy link
Contributor

@djs55 djs55 commented Jun 1, 2020

@majames yes, in the recent development build I linked Docker Desktop will use the ~/.mutagen.yml file so you can override global defaults. Unfortunately the patch didn't make the current edge release. Let me know if it works for you!

@Wilkolicious
Copy link

@Wilkolicious Wilkolicious commented Jun 30, 2020

Does the latest edge (2.3.2.0) use the ~/.mutagen.yml?

I've tried all sorts of combinations to try to ignore some paths and they are still being sync'd across.

~/.mutagen.yml

sync:
  defaults:
    ignore:
      paths:
        - "/gomo/authoring/app/public/logs"
        - "app/public/logs"
        - "/app/public/logs"
        - "app/public/logs/"
        - "/app/public/logs/"
      vcs: true

And the file share path in docker config sync'd is: /Users/phil.wilkinson/git/gomo-docker/gomo/authoring

I restart docker after changing this file as I don't know when the mutagen config is used by docker. Not sure where I am going wrong?

@Brenne
Copy link

@Brenne Brenne commented Jul 9, 2020

@Wilkolicious I was just also playing around with the ignore paths. I'd recommend to read the mutagen documentation

If you want your path /Users/phil.wilkinson/git/gomo-docker/gomo/authoring to be ignored, I'd expect the following .mutagen.yml to work

sync:
  defaults:
    ignore:
      paths:
        - authoring

Afterwards disable and enable the sync again (at least this helped for me).

@djs55
Copy link
Contributor

@djs55 djs55 commented Jul 14, 2020

There's a new experimental build which adds support for .dockerignore files: https://desktop-stage.docker.com/mac/edge/46638/Docker.dmg . This allows fine-grained per-project control over which directories should not be synchronized. Note that, like the ~/.mutagen.yml it will only take effect after the sync is disabled and re-enabled. There are a few other improvements so I've added a mini-changelog to this public gist: https://gist.github.com/djs55/0e8957e1e36431a43cbfd3caaffa9165 . If you get a chance to try it, let me know what you think!

@stephen-turner
Copy link
Contributor

@stephen-turner stephen-turner commented Jul 21, 2020

We have just released Desktop Edge 2.3.3.2, which adds support for the .dockerignore file as well as the ~/.mutagen.yml. So I think this ticket can be closed now.

@LiorRabin
Copy link

@LiorRabin LiorRabin commented Jul 21, 2020

Hi @stephen-turner @djs55.
What if I want my local environment to use mutagen.yml and building to staging/prod/other envs use the dockerignore file?
Is it possible to define somewhere by which of the files to sync/not to sync?
Thanks

@JussiMoisio
Copy link

@JussiMoisio JussiMoisio commented Aug 11, 2020

Mutagen ignores don't work for me with latest Edge 2.3.4.0 (46980) (tried both ~/.mutagen.yml and .dockerignore). Struggled with this for a while, then installed the 46638 version from @djs55 link above and it works. Regression? Is it supposed to work with the latest Edge?

@thaJeztah
Copy link
Member

@thaJeztah thaJeztah commented Aug 11, 2020

I think the feature was updated in later versions, and now looks at a .dockersyncignore file, so that .dockerignore (which was meant to be used for docker build only can have different exclude rules than those used for syncing

@JussiMoisio
Copy link

@JussiMoisio JussiMoisio commented Aug 11, 2020

Thanks, just read the changelog and noticed the .dockerignore -> .dockersyncignore change for 2.3.4.0. Works with that!

@thaJeztah
Copy link
Member

@thaJeztah thaJeztah commented Aug 11, 2020

@docker-desktop-robot
Copy link
Collaborator

@docker-desktop-robot docker-desktop-robot commented Oct 2, 2020

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked

@docker docker locked and limited conversation to collaborators Oct 2, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

10 participants