Sync issue on MacOS 13.0 #826
Comments
|
Thank you for providing all the informations. I will not be able to work on that issue,since i have no mac to test with. So you will need to dive yourself or somebody else can help out. On the first sight, i see that your ocalm version is 4.14 while the ocalm version on the docker-image is 4.12 - this might lead to issues (most probably). You might want to contribute a patch to add 4.14 to the build matrix https://github.com/EugenMayer/docker-image-unison/blob/main/.github/workflows/build.yml and see if that works. You can of course just build the image yourself and see if that helps (locally) - see the docs on https://github.com/EugenMayer/docker-image-unison and the github workflow for details how to build. You will then need to select you image as the unison image in docker sync, see https://docker-sync.readthedocs.io/en/latest/getting-started/configuration.html#sync-strategy-image |
|
Unfortunately, building and using the custom image with the versions from If you have any other suggestions for what to try or ideas for where I could begin contributing to address this, I'd gladly hear it. Thanks again. |
|
@kyrrr you should start checking the unison logs on both ends. See the tourble shooting guides at
Try to find out where the actual issue comes from and seriously, ensure that you really picked the right image when you tried your own build. Start the sync and check if the container that has been started by docker-sync is based on your custom image. (othwise you will go hunt for ghosts here, IMHO) |
|
Confirming that building with ocaml 4.14.0 does not solve the issue. The image is definitely the self-built version as inspected from Docker Desktop, and output of docker-sync also states ocaml 4.14.0 now. As to unison logs on het other end: I have no idea where to look. |
|
Let me sum up the current issue and please correct me:
Is that the current state? |
|
That's correct. upon executing As for my confguration: it's about the same as @kyrrr , running on Apple's M1 silicon. |
|
Ok so then what you are looking for is an issue with the filesystem even implementation, not unison itsefl. So that part that tells unison to start a new sync (that runs on MacOS natively). I assume that syncing from container to macos will work (changes on container are tracked) |
|
If you are assuming that synching from the container to the native system works: that seems not to be correct. This is what I tested:
|
|
So both directions are broken. So either unison crashes after the initial sync or somehow is stuck on whatever loop. You might want to ensure unsion is actually working with MacOS 13 in https://github.com/bcpierce00/unison |
|
I've not had the time to dig into getting this working, but have found mutagen.io to be a suitable replacement for my needs. |
|
@kyrrr Thanks for confirming that mutagen doesn't have the same problem. I had a look at it in the past, but found it a bit more overwhelming than docker-sync. It was one of the options to continue my daily development, and I'll consider it if nothing comes up from docker-sync in the following days. For the time being, I reverted to regular (slow) internal docker volumes. But I hate to see a good tool crumble down, so I will spend a little more time on trying to (help to) find what goes wrong here... |
|
for me 'native_osx' strategy is still working, but a lot slower than with the unison_image for arm64 before macos 13.0 - if i can help somehow, let me know ... |
|
I've updated unison to 2.53.0 and ocaml to 4.14.0 and tested it again on my local. It seems that syncing from the container out to the host works fine, however syncing into the container does not. I do not see any errors in the sync logs. I've followed the debugging steps in https://docker-sync.readthedocs.io/en/latest/troubleshooting/native-osx-troubleshooting.html Everything through step 4 works fine. The file diff is detected and then attempted to sync, then the diff goes away. However, the change is not synced into the container. On step 5, there is no /tmp/unison.log file inside the container. Instead the /tmp/supervisord.log file has the following. It seems that something is killing unison inside the docker-sync container. |
|
Found a new observation, if I change a file on the host, the dev container does not reflect the change. However if I go into the dev container and update the same file, the save. That file will now show the latest change from the host after writing to it. file.txt starts with content of "Number 1" So HOST edits file.txt and replaces content with "Number 2"
Dev Container edits file.txt and replaces content with "Number 3"
Dev Container edits file.txt and replaces content with "Number 4"
|
|
FYI: update to MacOS 13.0.1 (22A400) and latest Xcode Command Line tools does not seem to solve the problem... |
|
For anyone struggling: I switched to Mutagen solution few weeks back and I would recommend to try it out, its faster, much more reliable (didnt need to restart my sync containers since i first started them) and switch from Im sorry to promote mutagen here, but honestly docker-sync was such a pain for me these couple of years that i needed to write it here for other people that are struggling aswell. |
No bad feelings. If there is a good solution, just go for it. I rather doubt that mutagen is good in all the cases (similar to the fact that people use very different sync strategies in docker sync and all of them think 'their' is the only viable one). Mutagen follows the same path, it uses a local fs watcher with the very same issues. Means mutagen is rather the 'unison' like and has no option for 'native_osx'. The fs watcher vs native_osx options are vital and really depend on your payload, neither of these is simply superior. That said, i'am very please with people finding better alternatives to docker sync, be it mutagen or anything else. I also assume mutagen is the better 'syncer' to unison - but it is not simply something that solves the 'fs watching isssue' automagically. docker-sync was about offering different sync options for different payloads. This complicates things, which might not have played out well in all the facets for sure. On thing is a fact for sure, mutagen is by far better maintained in general, people seem to be very active over there. That is a big plus, for sure! |
|
@EugenMayer It seems like the issue is inside the unison image, do you have any insight on what could be causing the unsion process to be continuously terminated as per the logs above (Nov 7th)? I'll try to get some time to debug it further in the next few days. |
|
@EugenMayer It seems the old trick works. According to this issue: #346 In short, I make it work again by adding The While this work around does not fix the root cause or explain anything. I hope that it helps others. |
|
This will not fix it for me, are there any other options? |
|
I have the same issue as @kyrrr when running in background. But if i run in foreground(docker-sync start -f), the synchronization work as expected. Changes are synced (Mac Os Ventura 13.0, Intel). |
|
For everyone who needs a super simple, "just-work" solution: https://ddev.com/ It uses mutagen and is working really fine for me now! Sorry for advertising another thing here, but that's just my 2 cents. |
|
I have found the solution to this problem Mac os version: Ventura 13.1 The errors in.docker-sync/deamon.log After some research i found these issues on ruby In note 12 it states why docker-sync did not work anymore when upgrading to mac os ventura To help mitigate these issues these pr's where made These pr's were implemented in ruby version 2.7.7 So when you are using ruby 2.7.3 and are using rvm you can use this command During installation you get some question, after reading answer yes to these questions After this command dock-sync should work again Confirm that you use 2.7.7 with this command |
|
@ggkoning wonderful work! Thank you very much for digging so deep for this one! So all we need to announce is upgrading ruby - does macos ventura come with 2.7.3 as system ruby or how do it happen that people run the 2.7.x branch without upgrading to the latest version? |
|
At the compony i work for they use an internal tool that installs rvm for us with ruby version 2.7.3 so we are going to update that tool to install 2.7.7 The ruby versions that are preinstalled on Mac OS Mac OS Monterey 12.1 comes preinstalled with ruby version 2.6.8 Mac OS Monterey 12.6.1 Updates the preinstalled ruby version to 2.6.10 Mac OS Ventura 13.0 Updates the preinstalled ruby version to 2.6.10 The fix in ruby that i mentioned before is also implemented in the following versions Ruby version 3.0 and up Ruby version 3.1 and up |
|
We ran into the same issue in Ruby 3.0.4 and it seems to be resolved in 3.0.5 |
Error
Changes made on host not being synced to guest/container.
Works using none/native_osx, but is dreadfully slow.
Initial sync works, but subsequent syncing does not take place, so a potential workaround is to restart the sync occasionally, but that seems less than optimal.
app_sync in the sync container contains the correct files, but are not updated from host.
Did a full purge of all containers, networks, volumes, but the issue persists.
Docker Driver
❯ docker --version
Docker version 20.10.20, build 9fdeb9c
❯ docker-sync --version
1.0.4
Docker Desktop 4.13.0 (89412)
Sync strategy
unison
your docker-sync.yml
OS
MacOS 13.0 (M1) (worked on 12.6)
unison version 2.52.1 (ocaml 4.14.0)
eugenmayer/dockersync/unox: stable 0.2.0, HEAD
also attempted using:
autozimu/homebrew-formulas/unison-fsmonitor as unox replacement
Verbose output
Some project specific details removed/changed (directory names and file lists)
+ docker-sync start --config=docker-sync.yml ok Starting unison for sync my-sync ok my-sync container not running ok Starting precopy precopy docker run --rm -v "my-sync:/app_sync" -e APP_VOLUME=/app_sync -e TZ=$(basename $(dirname `readlink /etc/localtime`))/$(basename `readlink /etc/localtime`) -e UNISON_SRC="-socket 5000" -e UNISON_DEST="/app_sync" -e MONIT_ENABLE="false" -e MONIT_INTERVAL="" -e MONIT_HIGH_CPU_CYCLES="" -e UNISON_ARGS="-ignore='Name ../../src/node_modules' -ignore='Name ../../src/vendor' -ignore='Name .docker-sync/daemon.log' -ignore='Name .docker-sync/daemon.pid'" -e UNISON_WATCH_ARGS="" -e HOSTSYNC_ENABLE="0" -e UNISONSOCKET_ENABLE="1" --name my-sync eugenmayer/unison:2.52.1-4.12.0-ARM64 /usr/local/bin/precopy_appsync doing initial sync with unison Unison 2.52.1 (ocaml 4.12.0): Contacting server... Looking for changes Reconciling changes Propagating updates Unison 2.52.1 (ocaml 4.12.0) started propagating changes at 15:44:14.11 on 25 Oct 2022 [BGN] Copying from /app_sync to /host_sync [END] Copying Unison 2.52.1 (ocaml 4.12.0) finished propagating changes at 15:44:14.11 on 25 Oct 2022, 0.000 s Saving synchronizer state Synchronization complete at 15:44:14 (1 item transferred, 0 skipped, 0 failed) chown ing file to uid 0 real 0m 0.04s user 0m 0.02s sys 0m 0.01s initial sync done using unison ok creating my-sync container command docker run -p '127.0.0.1::5000' -v my-sync:/app_sync -e APP_VOLUME=/app_sync -e TZ=$(basename $(dirname `readlink /etc/localtime`))/$(basename `readlink /etc/localtime`) -e UNISON_SRC="-socket 5000" -e UNISON_DEST="/app_sync" -e MONIT_ENABLE="false" -e MONIT_INTERVAL="" -e MONIT_HIGH_CPU_CYCLES="" -e UNISON_ARGS="-ignore='Name ../../src/node_modules' -ignore='Name ../../src/vendor' -ignore='Name .docker-sync/daemon.log' -ignore='Name .docker-sync/daemon.pid'" -e UNISON_WATCH_ARGS="" -e HOSTSYNC_ENABLE="0" -e UNISONSOCKET_ENABLE="1" --name my-sync -d eugenmayer/unison:2.52.1-4.12.0-ARM64 ok starting initial sync of my-sync command docker inspect --format='{{(index (index .NetworkSettings.Ports "5000/tcp") 0).HostPort}}' my-sync command unison -testserver "/Users/username/myproject/src" "socket://127.0.0.1:50632" command docker inspect --format='{{(index (index .NetworkSettings.Ports "5000/tcp") 0).HostPort}}' my-sync command docker inspect --format='{{(index (index .NetworkSettings.Ports "5000/tcp") 0).HostPort}}' my-sync command unison -ignore='Name ../../src/node_modules' -ignore='Name ../../src/vendor' -ignore='Name .docker-sync/daemon.log' -ignore='Name .docker-sync/daemon.pid' '/Users/username/myproject/src' -auto -batch -prefer '/Users/username/myproject/src' -copyonconflict socket://127.0.0.1:50632 ok Synced /Users/username/myproject/src output Warning: No archive files were found for these roots, whose canonical names are: /Users/username/myproject/src //149a1c298105//app_sync This can happen either because this is the first time you have synchronized these roots, or because you have upgraded Unison to a new version with a different archive format. Update detection may take a while on this run if the replicas are large. Unison will assume that the 'last synchronized state' of both replicas was completely empty. This means that any files that are different will be reported as conflicts, and any files that exist only on one replica will be judged as new and propagated to the other replica. If the two replicas are identical, then no changes will be reported. If you see this message repeatedly, it may be because one of your machines is getting its address from DHCP, which is causing its host name to change between synchronizations. See the documentation for the UNISONLOCALHOSTNAME environment variable for advice on how to correct this. Donations to the Unison project are gratefully accepted: http://www.cis.upenn.edu/~bcpierce/unison file ----> .gitignore ... dir ----> web 0% 40:12 ETA success Unison server started command docker inspect --format='{{(index (index .NetworkSettings.Ports "5000/tcp") 0).HostPort}}' my-sync command docker inspect --format='{{(index (index .NetworkSettings.Ports "5000/tcp") 0).HostPort}}' my-sync command unison -ignore='Name ../../src/node_modules' -ignore='Name ../../src/vendor' -ignore='Name .docker-sync/daemon.log' -ignore='Name .docker-sync/daemon.pid' '/Users/username/myproject/src' -auto -batch -prefer '/Users/username/myproject/src' -copyonconflict socket://127.0.0.1:50632 ok Synced /Users/username/myproject/src success Starting Docker-Sync in the backgroundThe text was updated successfully, but these errors were encountered: