New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
OneDrive Windows mount: changing file names renders them in capital letters and duplicates #6682
Comments
Could you write a short description of how to replicate the problem please? |
Just to clarify, the bug is present when building from this commit. I don't know (yet) if this is the commit causing the bug, or it is caused by an earlier commit.
The simple replication is to make a simple mount something like this on Windows:
and then open the Explorer in W: and click New/Folder - then you will see the new folder having the name "NEW FOLDER". Press F5 to refresh and you will see "New folder". The seemingly duplicate files are more difficult to reproduce, I don't have a bulletproof sequence yet, and guess the issues are related. |
Is it possible to reproduce this issue just with CMD? |
Tried, but not as simple as Perhaps you can recreate the sequence by replicating the actions performed to "Nueva carpeta"/"NUEVA CARPETA" in this snippet from the initial debug log: "Nueva carpeta" means "New Folder".
|
I have with the help of available betas narrowed the suspected changes down to the commits in this screenshot with some There were unfortunately no betas to separate between my 2 primary suspects (marked in orange): a947f29 and 1b0128e and a947f29 is the last I can build without a lot of hassle. Then I got the idea of reversing 1b0128e on top of a947f29 by changing go.mod to So the issue seems to come from the winfsp/cgofuse update from |
Great sleuthing :-)
I don't see anything unusual in that log - I looked through it very carefully. I reproduced this on my Windows VM. It did not reproduce with WinFSP 1.10.22006 however when I installed the latest version 1.12.22301 it does reproduce the problem:
I created a mount of a local directory
I then tried to create a new folder in explorer in Q: using the Explorer menu. This works but explorer shows it in UPPER CASE: However we can see that it actually created it in lower case:
And the CMD view agrees.
I suspect this is because rclone calls Line 166 in 98fa93f
If I comment that call out then Explorer then behaves sensibly. Maybe rclone shouldn't be doing that - I don't know. So I guess this is something to do with the case insensitive file handling, and it is likely to do with winfsp/cgofuse@9b9d107 (though I haven't bisected this yet). @billziss-gh do you have any thoughts on this? Is it a cgofuse or WinFSP bug? Or should rclone just not call |
As you know the Windows file system is case-insensitive by default. This means that an app may open a path Regardless of the path used to open a file, it is sometimes desirable in Windows to find the "real" path of a file, i.e. the path of the file with the correct case. This is called the "normalized" path in Windows parlance and Windows provides a special file system operation to get it. This file system operation is rather popular; lots of system components, filters, etc. inquire for the normalized path of a file that is being opened. For this reason this operation is built into the WinFsp driver. The driver follows a simple approach:
Native WinFsp file systems always supported path normalization, but FUSE did not. I recently added this capability in FUSE and cgofuse via the Now it is unclear to me why you should see a difference in behavior between WinFsp 1.10 and 1.12. I may have screwed something up when implementing the |
OK, so it looks like that rclone was taking option 1. with WinFSP 1.10 but now is taking option 2. after the FUSE/cgofuse Getpath changes. We didn't add any code to report the normalized path in rclone so I guess the driver is reporting the UPPERCASED path for us. So it looks like to fix this, rclone should implement I had a go with this. I wasn't sure whether
However all variants of that caused Explorer to give this new and exciting error message when I tried to create a directory. I couldn't find any docs for Any ideas @billziss-gh ? |
Assume that the actual path of a file is |
Before this fix, we told cgofuse/WinFSP that the backend was case insensitive but didn't implement the Getpath backend function to return the normalised case of a file. Resently cgofuse started implementing case insensitive files properly but since we hadn't implemented Getpath, the file names were taking the default of all in UPPER CASE. This patch implements Getpath for cgofuse which fixes the case problems. This problem came to light when we upgraded cgofuse and WinFSP (to 1.12) which had the code to implement Getpath. Fixes #6682
Thanks @billziss-gh - I was missing the leading It seems to work now and fixes the problem if you want to give it a try @ferferga v1.62.0-beta.6674.6de91e367.fix-6682-cmount-case-insensitive on branch fix-6682-cmount-case-insensitive (uploaded in 15-30 mins) |
Thanks @ferferga, very helpful. I used your version info to narrow down to commit 1b0128e in v1.59.0, which has then used by Nick to narrow down on the introduction of |
@ncw fantastic! BTW, I have sent you some private emails recently and wondering if you have received them. If you did, no problem :) |
@ncw Works great, and seems duplicated files are gone too! |
Thank you for testing and for all your help everyone :-) I've merged this to master now which means it will be in the latest beta in 15-30 minutes and released in v1.62 If we make a v1.61.2 I'll put it in that, but that isn't a definite! @billziss-gh I have one email from 7 December I haven't replied to yet were there others? My INBOX is a trashfire as usual (11,000 unread messages). Although the problem is me, I'm sure, rather than my INBOX - I just like to externalise the blame on technology which can't answer back ;-) |
@ncw no problem! There is nothing urgent, I just wanted to make sure that you did get them given that email can be an unreliable medium at times. |
The associated forum post URL from
https://forum.rclone.org
https://forum.rclone.org/t/onedrive-windows-mount-changing-file-names-renders-them-in-capital-letters-and-duplicates/35179
What is the problem you are having with rclone?
Changing file names renders them in capital letters and duplicates
What is your rclone version (output from
rclone version
)Which OS you are using and how many bits (e.g. Windows 7, 64 bit)
Windows 11 Pro 22H2 64 bits 22621.963
Which cloud storage system are you using? (e.g. Google Drive)
OneDrive
The command you were trying to run (e.g.
rclone copy /tmp remote:tmp
)Read the forum post associated for a better explanation of the issue
A log from the command with the
-vv
flag (e.g. output fromrclone -vv copy /tmp remote:tmp
)Read the forum post associated
Additional details
Although pointed in the forum post, I'll leave this commit found to contain the bug by @olefrost for ease of access: a947f29
How to use GitHub
The text was updated successfully, but these errors were encountered: