MacOS cmount command: directory content appears empty after a while #4591
Comments
What that sequence looks like is that something unmounted the directory externally or possibly killed the rclone processes. Can you check system, kernel, launchd logs to see if there is any evidence of that? |
|
Only mention of rclone in Some diagnostic reports (found via And Not sure if any of this is useful. Let me know if there's more I can dig up. Edit: By the way, should I remove OSXFuse now that rclone is using cmount? It's still installed, and I suspect it needs to remain installed, but I figured I'd ask. |
|
This looks like the OS throttled rclone
And this looks like launchd killed rclone
It would be worthwhile digging into those a bit. I know very little about macOS so I'm not sure exactly where to look next.
Rclone is still using osxfuse so you need it installed. |
|
I'd like to add that a new kind of diagnostic log was produced recently, which corresponds with the issue of the contents of the directory not showing anymore, despite the And this other one: |
|
As far as I can see this is the kernel killing rclone for doing too many wakeups. However rclone only wakes up in response to the kernel, so I don't think I can fix rclone.
If the limit is 150 wakeups per second then you could try adding |
|
I'll give that a shot. Is there a way to determine how many requests are being made without setting this? For example, if I set it to |
|
I added I then found the following in the Obviously neither of these seem to indicate what went wrong. I'm not familiar with Go development, or MacOS development, so this is really the limit of my help here, unfortunately. The only observation I can make is that this is a new issue, and not one I saw until I started using the new (temporary) I should note that I have Plex running a conservative amount of scans on this, but also note that this has been a non-issue when using the regular |
|
Thanks for the update. If you could confirm whether this is just I'm a bit hampered looking at this as I don't have a mac but other rclone devs do. @darthShadow - I think you are a macOS user - have you seen this issue? |
|
I am a macOS user but that is just for general dev work because it's quite a few years old and not capable enough to run any heavy-duty software like Plex. My productivity machine is still a Linux system and that's where I use rclone mounts with Plex etc. That said, I will still try with just a normal rclone mount (without Plex) and see if I can reproduce that. |
|
I've been using rclone 1.53.2-DEV on MacOS 10.15.7 with |
|
Just tried again and it's happened again. I was copying a file into a rclone mount, and I was downloading a file directly into the rclone mount. ( using mount + vfs-cache-mode writes ) Although this time, the directory didn't end up with "device not configured" - I could delete the mounted directory. Diagnostic report:: |
This looks like the fundamental problem - macOS is saying that However I haven't found a way of setting this limit or exempting rclone from it - that would be the most useful thing. Can you? Another thing we could do is slow rclone down to make each wakeup take 1/150 s = 6.6mS. This seems like a bad idea but it would probably fix the problem. |
This was a mount that Plex was using, so probably quite a lot, though it was configured to only perform its operations manually, so it was at least controlled and not running at all hours. |
|
Can you see if rclone is in "App Nap" when sitting idle: https://developer.apple.com/library/archive/documentation/Performance/Conceptual/power_efficiency_guidelines_osx/AppNap.html Can you try also preventing App Nap like this: https://www.doornik.com/appnap.html |
|
Can you try |
|
@ncw About a week ago I upgraded It never once unmounted or provided the errors I originally described. I must have kept the mount active about five days before I had to reboot the computer for unrelated reasons a couple days ago. The fact it stayed mounted for five days is significant, because it never had gone that long since describing the original problem after upgrading to I have not tried your latest option, but I will. I'll try to report back by end of next week. Edit: By the way, I had updated and tested again recently so I could test "App Nap," but I could never actually see it in that state because it never happened. I did not downgrade to the old homebrew version that I used when I opened the ticket, so I couldn't say if it ever got into that state; given that the latest version downloaded directly from the site worked without issues, I didn't put additional time into answering the question. |
The major cjange is that Did you upgrade osxfuse/macfuse? That might be a change? |
I actually upgraded from a 2014 MBP to a 2018 MBP in January, so the latest version of osxfuse almost certainly changed as it was a new install. The version I had successful mounts (noted in my last comment) was |
|
I'm not experiencing the issue I originally described anymore. The mount has been running all week without issue. While it's unsatisfactory, I'd be willing to pin this on a imperfect combination of versions localized to my environment at the time. Since using the latest version of rclone and macfuse, the problem has disappeared. |
|
I was optimistic too soon. The problem returned. The primary difference from the last week to the current failure is how rclone was being called. It may be nothing, or it may be everything. During the last week I had been running the command directly from the terminal: It was stable. After that week-long test I figured it was time to get it mounted automatically and persist through any problems, and in the MacOS world, that's done using the <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>KeepAlive</key>
<true/>
<key>Label</key>
<string>danemacmillan.etc.rclone</string>
<key>ProgramArguments</key>
<array>
<string>/usr/local/bin/rclone</string>
<string>mount</string>
<string>gd:</string>
<string>/Users/danemacmillan/mnt/gd</string>
<string>--allow-other</string>
<string>--dir-cache-time=960h</string>
<string>--poll-interval=15s</string>
<string>--umask=002</string>
<string>--vfs-cache-mode=full</string>
<string>--vfs-read-ahead=1G</string>
<string>--vfs-cache-max-age=480h</string>
<string>--vfs-cache-max-size=140G</string>
<string>--cache-dir=/Users/danemacmillan/.cache/rclone</string>
<string>--bwlimit=4M:25M</string>
<string>--stats=1s</string>
<string>--log-level=INFO</string>
<string>--log-file=/Users/danemacmillan/tmp/rclone-mount-gd.log</string>
</array>
<key>RunAtLoad</key>
<true/>
</dict>
</plist>Running the following command will allow rclone to mount the remote upon boot and whenever there's a problem, like if rclone was killed: I have not rebooted the computer yet, but ever since rclone died from running as a The activity that occurred on the mount was the same before and after configuring it to launch with In reviewing the Based on all this, I'm leaning towards Here is one of the latest dumps from |
What is the problem you are having with rclone?
I started using the latest version of
rclonelast night to test both the new VFS--vfs-cache-mode=fullfeature and the newcmountcommand for MacOS. In testing I noticed that after a bit of usage, the contents of the mounted directory disappears. The content is still safely in the cloud storage service, but there's just nothing to see when listing the directory contents. The mount still appears in the list of mounted locations when themountcommand is run, and the--statsoutput continues to update according to the1sinterval I configured. Moreover, I'm unable toumount [path]the path for the dead mount, but I can re-issue therclone cmountcommand and it works, but the listing of mounts visible when issuing themountcommand shows the location mounted multiple times (https://i.imgur.com/0VzE3O9.png). The mount does work, though, for an indeterminate amount of time until it silently fails again, which is the reason for documenting this issue in GitHub.I've just remounted with
--log-level=DEBUG, so hopefully the next time this fails, actionable log data will be available, and I'll update this ticket.What is your rclone version (output from
rclone version)rclone v1.53.1
Which OS you are using and how many bits (eg Windows 7, 64 bit)
MacOS Catalina 10.15.6 (19G2021). 64bit.
Which cloud storage system are you using? (eg Google Drive)
Google Drive
The command you were trying to run (eg
rclone copy /tmp remote:tmp)rclone cmount gd: /Users/danemacmillan/mnt/gd --allow-other --dir-cache-time=1000h --poll-interval=15s --umask=002 --vfs-cache-mode=full --bwlimit=0 --stats=1s --fast-list --read-only --log-file=/Users/danemacmillan/tmp/rclone-mount-gd.log --log-level=INFOA log from the command with the
-vvflag (eg output fromrclone -vv copy /tmp remote:tmp)I'm remounting now with full verbosity, and after some time the issue will probably occur, as there are no errors anywhere with the above command.
Update 1
The issue just happened after a few minutes, which more or less corresponds with my experience so far. The mount only works for a few minutes and then it stops working. Here's the
DEBUGlevel log output that was generated the moment it happened:Note that the attempts to re-mount it immediately after it dies is the configured MacOS
launchdservice doing its job. For reference, that service file looks like this:Mounting without the service will do the same thing: in other words, there's no difference whether the command is run manually or as a service.
This is what a directory listing looks like after the mount fails:
Here are the logs after mounting it manually after it failed:
The text was updated successfully, but these errors were encountered: