Mutagen stuck when stopping environment

[lstellway]

Environment

• Docker Desktop v2.3.0.5
• Warden v0.9.1
• Mutagen 0.11.7
• magento2 environment type

Description

When executing warden env stop, the command is halted by the mutagen sync call.
I believe the issue may be with Mutagen as calling mutagen sync terminate or mutagen sync pause directly results in the same experience.

Currently, I am forcefully shutting down the docker containers when I am done working on my project.
Here are a few things I tried:

• Reinstalled both warden and mutagen and updated to the latest versions.
• Rebuilt my environment containers (several times)
• Deleted the ~/.mutagen directory

I came across mutagen-io/mutagen/issues/174 while searching for answers. I thought I would post the issue here since Warden relies on the tool for MacOS users - has anyone else experienced this? Any ideas?

Thank you

[Lex50]

has anyone else experienced this?

Yeah, me too.

I started experiencing this after updating to Docker Desktop 2.3.0.5 from 2.3.0.4, on warden 0.9.1.

[mhhansen]

happened same thing to me, you'll have to downgrade Docker Desktop from 2.3.0.5 to 2.3.04 (as @Lex50 said)
and mutagen will work as expected. don't know the specifics as to the why this happens

here's the 2.3.0.4 download page
https://docs.docker.com/docker-for-mac/release-notes/#docker-desktop-community-2304

[lstellway]

Got it - confirming that downgrading to Docker Desktop 2.3.0.4 did the trick for me.
Thank you both!

[davidalger]

Known issues🔗
The clock_gettime64 system call returns EPERM rather than ENOSYS in i386 images. To work around this issue, disable seccomp by using the --privileged flag. See docker/for-win#8326.

I'm wondering if this known-issue, added to known-issues 2 days ago, might be the culprit. If Mutagen were getting incorrect timing data, it would stand to reason it could be endlessly scanning looking for changed files within the container. Seems like it affects Docker Desktop on stable and edge across Win and Mac.

[Vinai]

@loganstellway May I ask some questions...
Why did you choose 2.3.0.4? Did you try to upgrade to the latest version, too?
Did you download the version from https://docs.docker.com/docker-for-mac/edge-release-notes/ ? Problem is I can't see 2.3.0.4 in that list 🤔
EDIT: Found it in the release notes for the stable releases, https://docs.docker.com/docker-for-mac/release-notes/
Thanks!

[xenoscopic]

Sorry that I didn't catch this earlier. This was probably the same underlying issue as in mutagen-io/mutagen#223. I've just back-ported the fix to v0.11.8, so hopefully that fixes the problem, but please let me know if not.

[lstellway]

Hi @Vinai !
We are in the process of upgrading, but some of our custom modules need updating first.
Edit: Just realized this was a question about the Docker version.. haha

@havoc-io
This seems to have done the trick -
Just tested with Docker 2.4.0.0 and Mutagen 0.11.8.

Thank you! 🙏

[davidalger]

@havoc-io Thanks for jumping over here to let us know. This resolved the issue in my testing with Docker Desktop 2.4.0.0 as well, and I'm assuming it does the same for 2.3.0.5 where the issue initially cropped up.

One note here for others coming across this is: In order for Mutagen to start back up following my upgrade from 0.11.7 to 0.11.8, I had to first kill off the mutagen-agent process(es) which were running:

The error encountered post-mutagen-upgrade

$ brew upgrade mutagen
...
$ mutagen daemon stop
$ warden sync start
Attempting to start Mutagen daemon...                                           
Error: unable to connect to daemon: connection timed out (is the daemon running?)
ERROR: Command `mutagen sync terminate --label-selector "warden-sync=${WARDEN_ENV_NAME}"` at /opt/warden/commands/sync.cmd:45 failed with exit code 1
$ mutagen daemon start
$ warden sync start
Attempting to start Mutagen daemon...                                           
Error: unable to connect to daemon: connection timed out (is the daemon running?)
ERROR: Command `mutagen sync terminate --label-selector "warden-sync=${WARDEN_ENV_NAME}"` at /opt/warden/commands/sync.cmd:45 failed with exit code 1
$ mutagen list
Attempting to start Mutagen daemon...                                           ^C

Resolution allowing me to start the daemon and sync sessions:

$ ps uax | egrep [m]utagen-agent
davidalger       66304   0.0  0.0  4453876  14468   ??  S     8:47AM   0:00.09 com.docker.cli exec --interactive --workdir /home/www-data 2f188160da03c3cfbe841bcaa2c4e027f577f3e9da5033d117f1dd22370a0b3f .mutagen/agents/0.11.7/mutagen-agent synchronizer
$ kill 66304
$ warden sync start
Started Mutagen daemon in background (terminate with "mutagen daemon stop")     
Created session sync_q0grUTlCASwNgXT7g3hx27uHriexeYDWJsUXh3mGpjv                
Waiting for initial synchronization to complete
...

As can be seen in commit 44e13f3, the upcoming Warden 0.10.0 release will require Mutagen 0.11.8 or later when starting sync sessions to ensure users upgrade to avoid the failure.