nfsmount — not functional on macOS for writing files to OneDrive via Finder #7503

scuba-tech · 2023-12-12T14:06:02Z

Hello! I'm writing with an active issue that seems to affect rclone nfsmount on macOS.

The associated forum post URL from `https://forum.rclone.org`

https://forum.rclone.org/t/rclone-v1-65-0-release/43100/25?u=scuba-tech

What is the problem you are having with rclone?

ALL attempts to copy files of any size or type, from Finder into OneDrive, via nfsmount fail.

It is not possible to write from Finder or save work. It is possible to read/open files.

Simple writing operations like touch and cat > filename from a shell do write properly into nfsmount.

What is your rclone version (output from `rclone version`)

1.65.0 via brew

Which OS you are using and how many bits (e.g. Windows 7, 64 bit)

macOS Sonoma (14.1.1) // ARM64

Which cloud storage system are you using? (e.g. Google Drive)

Microsoft OneDrive (personal family // 6TB)

The command you were trying to run (e.g. `rclone copy /tmp remote:tmp`)

rclone nfsmount --vfs-cache-mode full --filter-from "/Users/<user>/.config/rclone/rclone-filter-rules.txt" OD: ~/MOUNT-OneDrive --verbose

Notes:

I am using --vfs-cache-mode full to allow writing with OneDrive. Using none gives an (expected) read-only error message.
Here are the contents of my filter rules file:

- **/.DS_Store
- **/.hidden*
- **/._*

A log from the command with the `-vv` flag (e.g. output from `rclone -vv copy /tmp remote:tmp`)

[ERROR] No file for lookup of ._GH010215.mp4
[ERROR] call to 0x106<###>  failed: No such file or directory
INFO  : SdT-Inboxes/GH010215.mp4: vfs cache: removed cache file as file deleted
[ERROR] No file for lookup of ._7F2A7976.JPG
[ERROR] call to 0x106<###>  failed: No such file or directory
[ERROR] err loading attrs for SdT-Inboxes/GH010215.mp4: file does not exist
[ERROR] No file for lookup of ._7F2A7977.JPG
[ERROR] call to 0x106<###>  failed: No such file or directory
[ERROR] err loading attrs for SdT-Inboxes/GH010215.mp4: file does not exist

Confirming that the copy into the mount did fail, checking with Finder, ls -hAls, and even OneDrive webUI... all of these confirm that the file does not actually "already exist" as the error message above claims!

I hope this report helps with getting nfsmount sorted. :)

How to use GitHub

Please use the 👍 reaction to show that you are affected by the same issue.
Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
Subscribe to receive notifications on status change and new comments.

The text was updated successfully, but these errors were encountered:

scuba-tech · 2023-12-12T14:12:26Z

Including a further --verbose excerpt for context as I believe it is related; it appears that the filter rules are not respected.

NOTICE: NFS Server running at 127.0.0.1:55165
[ERROR] No file for lookup of ._.
[ERROR] call to 0x106<###> failed: No such file or directory
[ERROR] No file for lookup of .hidden
[ERROR] call to 0x106<###> failed: No such file or directory
[ERROR] No file for lookup of .DS_Store
[ERROR] call to 0x106<###> failed: No such file or directory
[...]
[ERROR] call to 0x106<###> failed: No such file or directory
[ERROR] No file for lookup of ._7F2A8214.JPG
[ERROR] call to 0x106<###> failed: No such file or directory
[ERROR] No file for lookup of ._7F2A8215.JPG
[ERROR] call to 0x106<###> failed: No such file or directory
[ERROR] No file for lookup of ._7F2A8196.JPG
[ERROR] call to 0x106<###> failed: No such file or directory
[ERROR] No file for lookup of ._7F2A8216.JPG
[ERROR] call to 0x106<###> failed: No such file or directory
[ERROR] No file for lookup of ._7F2A8197.JPG

[repeat approx 7200x times]

swiftbird07 · 2023-12-25T22:01:23Z

+1 I have the exact same issue. @scuba-tech did you figure it out?

scuba-tech · 2023-12-25T23:12:41Z

+1 I have the exact same issue. @scuba-tech did you figure it out?

Negative. Presently, this command is non-functional for me. There is no solution that I have found.

swiftbird07 · 2023-12-26T11:05:13Z

Bummer. I guess I will use Mountain Duck now (sadly not Open Source tho)

scuba-tech · 2023-12-26T14:52:56Z

Bummer. I guess I will use Mountain Duck now (sadly not Open Source tho)

I will look at Mountain Duck, thank you for the suggestion. As much as we may love open source and libre software, sometimes one simply needs things to work!!

EDIT: Mountain Duck mounting works perfectly. I'll move back to rclone as soon as reasonable, of course.

I will also keep an eye on this thread and am happy to help the developers with any further troubleshooting or testing.

I would even be happy to send a Mac Mini (or give them 24/7 VNC+Parsec+SSH access with root) to any active rclone developers who wish to have a Mac as a dev/testing target... that is how much I love this project and want to give back!

kapitainsky · 2024-01-24T21:58:20Z

@ncw - does nfsmount has a maintainer? As it is now this feature is not usable at all. At least on macOS. Its existence in released rclone version creates only confusion.

nielash · 2024-01-25T17:56:04Z

I took a brief look at this. The following changes seem to have fixed this particular issue for me: nielash@0781842 (Sonoma 14.2.1 / Apple M1 Ultra)

However I'm not very familiar with this part of the codebase, and it is possible that I'm not fully understanding all the consequences.

Also while debugging this I noticed some encoding issues that probably should be fixed (not addressed by these changes).

kapitainsky · 2024-01-25T18:06:00Z

Nice that fixing writes is possible.

Another old issue is NFC/NFD drama on macOS. I suspect that nsfmount does not fix it neither.

#7072

kapitainsky · 2024-01-25T18:26:42Z

I have tried nielash@0781842 and indeed now I can write files to nfs mount.

However NFC/NFD problem is still there:

mount was never fully working on macOS. Maybe a lot of people do not even realise because they mostly use nice names without any problematic characters:) But it is massive issue for users in Asia for example.

nielash · 2024-01-25T19:10:05Z

Yes, those are the kinds of encoding issues I was noticing. Great writeup on #7072. Is anyone working on that?

kapitainsky · 2024-01-25T19:15:13Z

Yes, those are the kinds of encoding issues I was noticing. Great writeup on #7072. Is anyone working on that?

I hope @ncw has it on his radar?:) Unfortunately it is beyond my programming skills so I can only test and complain:)

nielash · 2024-02-08T06:22:39Z

Hi @kapitainsky, continuing our discussion from #7072 about nfsmount fixes here. I think I fixed the two issues you mentioned (--volname and hanging after external unmount). If you have a chance, let me know if this fixes it for you? https://github.com/nielash/rclone/tree/nfsmount-fixes

One weird thing about --volname is that Finder shows the correct name in most places:

except the sidebar, which still shows "localhost". It seems like that is just a limitation of mount_nfs https://apple.stackexchange.com/a/188068 But let me know if you know any cool tricks for that.

kapitainsky · 2024-02-08T06:31:41Z

Will test today.

except the sidebar, which still shows "localhost". It seems like that is just a limitation of mount_nfs https://apple.stackexchange.com/a/188068 But let me know if you know any cool tricks for that.

Then it is similar to FUSE-T (which also uses nfs server):

I think sidebar shows server name which is different to share name. Maybe there is a way to give it is own name? E.g. nfsmount? this is however purely cosmetic thing I think should not stop or delay this PR from going live:)

kapitainsky · 2024-02-08T06:39:16Z

If you have a chance, let me know if this fixes it for you? https://github.com/nielash/rclone/tree/nfsmount-fixes

I have compiled it already and all looks good. volname works and when unmounted rclone nfsmount terminates. I will keep using it now to see if anything else is missing.

nielash · 2024-02-08T06:44:04Z

I think sidebar shows server name which is different to share name. Maybe there is a way to give it is own name? E.g. nfsshare?

Under the hood, nfsmount is just running the standard mount command, like:

mount -o port=xxxxx -o mountport=xxxxx localhost:volname /Users/username/path/to/mount

kapitainsky · 2024-02-08T06:47:18Z

Under the hood, nfsmount is just running the standard mount command

As fuse-t does..

Here rclone nfsmount

So what fuse-t does is renaming localhost to fuse-t. Shame that it is not open source so we can't have a look how it is done

nielash · 2024-02-08T06:49:40Z

Shame that it is not open source so we can't have a look how it is done

Agreed.

kapitainsky · 2024-02-08T06:59:21Z

fixed rclone nfsmount is not using NFD normalisation yet? correct?

nielash · 2024-02-08T07:02:42Z

fixed rclone nfsmount is not using NFD normalisation yet? correct?

Correct. But you could just clone and cherry-pick 90dbe8b and it should work...

I opened the PR for that, btw #7620

kapitainsky · 2024-02-08T07:17:11Z

I have noticed something weird. When used with polling remote like onedrive.

When I delete 2 files from the remote directly - only one file is removed from nsfmount. Like polling changes are not fully processed?

nielash · 2024-02-08T07:21:19Z

That is odd. Was that the case before too? Or did my changes break something?

kapitainsky · 2024-02-08T07:22:54Z

For a moment ignore - maybe false alarm... I am trying to reproduce it now with -vv to have some data. But all works:) Maybe need another coffee before I proceed.

kapitainsky · 2024-02-08T08:09:26Z

Can we pass extra flags to mount -o port=xxxxx -o mountport=xxxxx localhost:volname /Users/username/path/to/mount ?

In similar way how fuse-t allows it? e.g.:

rclone mount remote: mountpoint -o nfc -o noatime

This way we could change nfs server features if needed. See this:

https://forum.rclone.org/t/macos-rclone-mount-new-fuse-t-released-old-issues-fixed/39502

nielash · 2024-02-08T08:28:33Z

Yes, -o or --option should work for this. (And for some reason, --fuse-flag is also supported, even though this doesn't use fuse...)

kapitainsky · 2024-02-08T08:34:38Z

Yes, -o or --option should work for this. (And for some reason, --fuse-flag is also supported, even though this doesn't use fuse...)

Indeed it works:

$ rclone nfsmount crypt: ~/Temp/mount --vfs-cache-mode=full --vfs-refresh --volname crypt_test -vv  -o nfc -o noatime
...
2024/02/08 08:33:51 DEBUG : Running mount command: ["mount" "-o" "port=54491" "-o" "mountport=54491" "-o" "nfc" "-o" "noatime" "localhost:crypt_test" "/Users/kptsky/Temp/mount"]

kapitainsky · 2024-02-08T08:44:24Z

-o nfc is not needed anymore thx to normalisation but I think -o noatime might be handy as like in FUSE-T we might have this issue:

File access and modification times cannot be set separately as it seems to be an issue with the NFS client which always modifies both. Can be reproduced with 'touch -m' and 'touch -a' commands

However I would not worry about it now. The key is that nfs mount options can be passed from rclone level. So whatever is needed can be used.

EDIT - I do not see this issue in Sonoma - so maybe Apple fixed it at last in nfs server. Before simple QickView in Finder would modify file modtime. Now it does not.

manselmi · 2024-02-08T12:14:42Z

…

So what fuse-t does is renaming localhost to fuse-t. Shame that it is not open source so we can't have a look how it is done

The fuse-t installer adds 127.0.0.1 fuse-t to /etc/hosts, which would allow it to specify fuse-t (as an alias for localhost) for the server name in its mount command.

kapitainsky · 2024-02-08T12:17:14Z

…
So what fuse-t does is renaming localhost to fuse-t. Shame that it is not open source so we can't have a look how it is done

The fuse-t installer adds 127.0.0.1 fuse-t to /etc/hosts, which would allow it to specify fuse-t (as an alias for localhost) for the server name in its mount command.

Good find. It would be problematic on the fly though as it requires sudo...

but we could add this to docs as a hint how to customise all nfsmount. Purely cosmetic but can be handy for some purists.

kapitainsky · 2024-02-08T12:26:01Z

@manselmi any chance you can try https://github.com/nielash/rclone/tree/nfsmount-fixes as well? It requires build it from source though.

Maybe this time we can avoid releasing crippled feature - which was the case with original nsfmount. At least on macOS

manselmi · 2024-02-08T15:03:54Z

@manselmi any chance you can try https://github.com/nielash/rclone/tree/nfsmount-fixes as well? It requires build it from source though.

Maybe this time we can avoid releasing crippled feature - which was the case with original nsfmount. At least on macOS

@kapitainsky I'm not sure what kind of testing you had in mind, so I compiled the branch from source like this:

go build -trimpath -ldflags "-s -X github.com/rclone/rclone/fs.Version=v9.9.9-test"

I then mounted my OneDrive remote and tried touching a new file into existence, which worked (it appeared in OneDrive) but took over 60 seconds to complete for some reason. I then added some text to the file locally, and this time it quickly synced to OneDrive. 👍

kapitainsky · 2024-02-08T15:12:24Z

Great. And thank you.

Simply by trying to use it you have a chance to notice something not working.

Before this change, writing files to an `nfsmount` via Finder on macOS would cause critical errors, rendering `nfsmount` effectively unusable on macOS. This change fixes the issue so that writes via Finder should be possible. The issue was primarily caused by the handler's HandleLimit being set to -1. -1 is the correct default for a NullAuthHandler, but not for a CachingHandler, which interprets -1 not as "no limit" but as "no cache". This change sets a high default of 1000000 by default, and gives the user control over it by repurposing the existing --vfs-cache-max-size flag. A minimum of 5 is enforced, as any lower than this will be insufficient to support directory listing.

Before this change, if a user unmounted externally (for example, via the Finder UI), rclone would not be aware of this and wait forever to exit -- effectively causing a deadlock that would require Ctrl+C to terminate. After this change, when the handler detects an external unmount, it calls a function which allows rclone to cleanly shutdown the VFS and exit.

Before this change, nfsmount ignored the --volname flag. After this change, the -- volname flag is respected, making it possible to set a custom volume name. macOS users should note that Finder will show the correct volume name in most places, but a notable exception is the sidebar, which will show "localhost". This seems to be a system limitation (at least without `sudo`), but see the discussion at rclone#7503 (comment) for some possible workarounds.

Before this change, writing files to an `nfsmount` via Finder on macOS would cause critical errors, rendering `nfsmount` effectively unusable on macOS. This change fixes the issue so that writes via Finder should be possible. The issue was primarily caused by the handler's HandleLimit being set to -1. -1 is the correct default for a NullAuthHandler, but not for a CachingHandler, which interprets -1 not as "no limit" but as "no cache". This change sets a high default of 1000000, and gives the user control over it with a new --nfs-cache-handle-limit flag (available in both `serve nfs` and `nfsmount`. A minimum of 5 is enforced, as any lower than this will be insufficient to support directory listing.

Before this change, if a user unmounted externally (for example, via the Finder UI), rclone would not be aware of this and wait forever to exit -- effectively causing a deadlock that would require Ctrl+C to terminate. After this change, when the handler detects an external unmount, it calls a function which allows rclone to cleanly shutdown the VFS and exit.

Before this change, nfsmount ignored the --volname flag. After this change, the -- volname flag is respected, making it possible to set a custom volume name. macOS users should note that Finder will show the correct volume name in most places, but a notable exception is the sidebar, which will show "localhost". This seems to be a system limitation (at least without `sudo`), but see the discussion at rclone#7503 (comment) for some possible workarounds.

Before this change, writing files to an `nfsmount` via Finder on macOS would cause critical errors, rendering `nfsmount` effectively unusable on macOS. This change fixes the issue so that writes via Finder should be possible. The issue was primarily caused by the handler's HandleLimit being set to -1. -1 is the correct default for a NullAuthHandler, but not for a CachingHandler, which interprets -1 not as "no limit" but as "no cache". This change sets a high default of 1000000, and gives the user control over it with a new --nfs-cache-handle-limit flag (available in both `serve nfs` and `nfsmount`. A minimum of 5 is enforced, as any lower than this will be insufficient to support directory listing.

Before this change, if a user unmounted externally (for example, via the Finder UI), rclone would not be aware of this and wait forever to exit -- effectively causing a deadlock that would require Ctrl+C to terminate. After this change, when the handler detects an external unmount, it calls a function which allows rclone to cleanly shutdown the VFS and exit.

Before this change, nfsmount ignored the --volname flag. After this change, the -- volname flag is respected, making it possible to set a custom volume name. macOS users should note that Finder will show the correct volume name in most places, but a notable exception is the sidebar, which will show "localhost". This seems to be a system limitation (at least without `sudo`), but see the discussion at #7503 (comment) for some possible workarounds.

nielash mentioned this issue Feb 5, 2024

new feature: VFS auto normalization #7072

Closed

nielash mentioned this issue Feb 12, 2024

nfsmount: fix writing files via Finder on macOS (and other bugs) - fixes #7503 #7628

Merged

5 tasks

nielash linked a pull request Feb 12, 2024 that will close this issue

nfsmount: fix writing files via Finder on macOS (and other bugs) - fixes #7503 #7628

Merged

5 tasks

nielash added bug VFS / mount OS: Mac Serving labels Feb 12, 2024

nielash closed this as completed in #7628 Feb 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nfsmount — not functional on macOS for writing files to OneDrive via Finder #7503

nfsmount — not functional on macOS for writing files to OneDrive via Finder #7503

scuba-tech commented Dec 12, 2023 •

edited

scuba-tech commented Dec 12, 2023

swiftbird07 commented Dec 25, 2023

scuba-tech commented Dec 25, 2023

swiftbird07 commented Dec 26, 2023

scuba-tech commented Dec 26, 2023 •

edited

kapitainsky commented Jan 24, 2024

nielash commented Jan 25, 2024

kapitainsky commented Jan 25, 2024

kapitainsky commented Jan 25, 2024

nielash commented Jan 25, 2024

kapitainsky commented Jan 25, 2024

nielash commented Feb 8, 2024

kapitainsky commented Feb 8, 2024 •

edited

kapitainsky commented Feb 8, 2024

nielash commented Feb 8, 2024

kapitainsky commented Feb 8, 2024 •

edited

nielash commented Feb 8, 2024

kapitainsky commented Feb 8, 2024

nielash commented Feb 8, 2024

kapitainsky commented Feb 8, 2024

nielash commented Feb 8, 2024

kapitainsky commented Feb 8, 2024

kapitainsky commented Feb 8, 2024 •

edited

nielash commented Feb 8, 2024

kapitainsky commented Feb 8, 2024 •

edited

kapitainsky commented Feb 8, 2024 •

edited

manselmi commented Feb 8, 2024

kapitainsky commented Feb 8, 2024 •

edited

kapitainsky commented Feb 8, 2024 •

edited

manselmi commented Feb 8, 2024

kapitainsky commented Feb 8, 2024

nfsmount — not functional on macOS for writing files to OneDrive via Finder #7503

nfsmount — not functional on macOS for writing files to OneDrive via Finder #7503

Comments

scuba-tech commented Dec 12, 2023 • edited

The associated forum post URL from https://forum.rclone.org

What is the problem you are having with rclone?

What is your rclone version (output from rclone version)

Which OS you are using and how many bits (e.g. Windows 7, 64 bit)

Which cloud storage system are you using? (e.g. Google Drive)

The command you were trying to run (e.g. rclone copy /tmp remote:tmp)

A log from the command with the -vv flag (e.g. output from rclone -vv copy /tmp remote:tmp)

How to use GitHub

scuba-tech commented Dec 12, 2023

swiftbird07 commented Dec 25, 2023

scuba-tech commented Dec 25, 2023

swiftbird07 commented Dec 26, 2023

scuba-tech commented Dec 26, 2023 • edited

kapitainsky commented Jan 24, 2024

nielash commented Jan 25, 2024

kapitainsky commented Jan 25, 2024

kapitainsky commented Jan 25, 2024

nielash commented Jan 25, 2024

kapitainsky commented Jan 25, 2024

nielash commented Feb 8, 2024

kapitainsky commented Feb 8, 2024 • edited

kapitainsky commented Feb 8, 2024

nielash commented Feb 8, 2024

kapitainsky commented Feb 8, 2024 • edited

nielash commented Feb 8, 2024

kapitainsky commented Feb 8, 2024

nielash commented Feb 8, 2024

kapitainsky commented Feb 8, 2024

nielash commented Feb 8, 2024

kapitainsky commented Feb 8, 2024

kapitainsky commented Feb 8, 2024 • edited

nielash commented Feb 8, 2024

kapitainsky commented Feb 8, 2024 • edited

kapitainsky commented Feb 8, 2024 • edited

manselmi commented Feb 8, 2024

kapitainsky commented Feb 8, 2024 • edited

kapitainsky commented Feb 8, 2024 • edited

manselmi commented Feb 8, 2024

kapitainsky commented Feb 8, 2024

scuba-tech commented Dec 12, 2023 •

edited

The associated forum post URL from `https://forum.rclone.org`

What is your rclone version (output from `rclone version`)

The command you were trying to run (e.g. `rclone copy /tmp remote:tmp`)

A log from the command with the `-vv` flag (e.g. output from `rclone -vv copy /tmp remote:tmp`)

scuba-tech commented Dec 26, 2023 •

edited

kapitainsky commented Feb 8, 2024 •

edited

kapitainsky commented Feb 8, 2024 •

edited

kapitainsky commented Feb 8, 2024 •

edited

kapitainsky commented Feb 8, 2024 •

edited

kapitainsky commented Feb 8, 2024 •

edited

kapitainsky commented Feb 8, 2024 •

edited

kapitainsky commented Feb 8, 2024 •

edited