Allow fine grained excludes configuration for Mutagen volume mounts

[majames]

Hi! I was really excited to see the announcement at DockerCon that Docker for MAC will now support Mutagen for more performant host volume mounting! We actually built a tool at the organisation I work out that does something very similar (using Watchman and docker cp/rm under the hood).

One killer feature that is currently missing is the ability to specify fine-grained excludes for the file sync list. I notice that there is limited support for excludes by creating a docker volume, however our mono-repo is quite large and contains many generated nested directories and, therefore, creating an individual volume for each of them is infeasible.

Are there plans to implement functionality that allow users to specify fine-grained includes/excludes? Something similar to a dockerignore or Watchman expressions would be great!

[stephen-turner]

@djs55 Is this option available in the Mutagen config file?

[djs55]

In more recent development builds e.g.https://desktop-stage.docker.com/mac/edge/45494/Docker.dmg (discussed originally in the context of symlinks #4595 (comment) ) it's possible to add paths to ignore to the global mutagen config file e.g.

$ cat ~/.mutagen.yml
sync:
  defaults:
    ignore:
      paths:
        - somethingtoignore
      vcs: true

Thanks for the suggestions about dockerignore and Watchman expressions -- I'll investigate them!

[majames]

Thanks for the quick response @djs55 !

Had a look at the comment you linked. Does that mean that Docker Desktop will use the ~/.mutagen.yml file when determining which files to sync? If so, then global ignores will probably be enough to unblock us from using this for now (although per directory ignores would be a nice feature!)

[djs55]

@majames yes, in the recent development build I linked Docker Desktop will use the ~/.mutagen.yml file so you can override global defaults. Unfortunately the patch didn't make the current edge release. Let me know if it works for you!

[Wilkolicious]

Does the latest edge (2.3.2.0) use the ~/.mutagen.yml?

I've tried all sorts of combinations to try to ignore some paths and they are still being sync'd across.

~/.mutagen.yml

sync:
  defaults:
    ignore:
      paths:
        - "/gomo/authoring/app/public/logs"
        - "app/public/logs"
        - "/app/public/logs"
        - "app/public/logs/"
        - "/app/public/logs/"
      vcs: true

And the file share path in docker config sync'd is: /Users/phil.wilkinson/git/gomo-docker/gomo/authoring

I restart docker after changing this file as I don't know when the mutagen config is used by docker. Not sure where I am going wrong?

[Brenne]

@Wilkolicious I was just also playing around with the ignore paths. I'd recommend to read the mutagen documentation

If you want your path /Users/phil.wilkinson/git/gomo-docker/gomo/authoring to be ignored, I'd expect the following .mutagen.yml to work

sync:
  defaults:
    ignore:
      paths:
        - authoring

Afterwards disable and enable the sync again (at least this helped for me).

[djs55]

There's a new experimental build which adds support for .dockerignore files: https://desktop-stage.docker.com/mac/edge/46638/Docker.dmg . This allows fine-grained per-project control over which directories should not be synchronized. Note that, like the ~/.mutagen.yml it will only take effect after the sync is disabled and re-enabled. There are a few other improvements so I've added a mini-changelog to this public gist: https://gist.github.com/djs55/0e8957e1e36431a43cbfd3caaffa9165 . If you get a chance to try it, let me know what you think!

[stephen-turner]

We have just released Desktop Edge 2.3.3.2, which adds support for the .dockerignore file as well as the ~/.mutagen.yml. So I think this ticket can be closed now.

[LiorRabin]

Hi @stephen-turner @djs55.
What if I want my local environment to use mutagen.yml and building to staging/prod/other envs use the dockerignore file?
Is it possible to define somewhere by which of the files to sync/not to sync?
Thanks

[JussiMoisio]

Mutagen ignores don't work for me with latest Edge 2.3.4.0 (46980) (tried both ~/.mutagen.yml and .dockerignore). Struggled with this for a while, then installed the 46638 version from @djs55 link above and it works. Regression? Is it supposed to work with the latest Edge?

[thaJeztah]

I think the feature was updated in later versions, and now looks at a .dockersyncignore file, so that .dockerignore (which was meant to be used for docker build only can have different exclude rules than those used for syncing

[JussiMoisio]

Thanks, just read the changelog and noticed the .dockerignore -> .dockersyncignore change for 2.3.4.0. Works with that!

[thaJeztah]

Ah, yes, it's in the release notes for v2.3.4.0 https://github.com/docker/docker.github.io/blob/9050cd471791a8e7c393ac1986f36c103183abc3/docker-for-mac/edge-release-notes.md#docker-desktop-community-2340

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked