rclone cannot read an encfs mounted drive on Windows 7 #261

Closed
garypaduana opened this Issue Dec 24, 2015 · 16 comments

Projects

None yet

3 participants

@garypaduana

First mount the virtual drive using encfs

encfs --reverse h:/folder n:

Try to perform any operation with that drive using rclone and it will return with a read error. dir from the N:\ directory works without a problem.

N:\>rclone ls .
2015/12/24 12:00:39 Local file system at \\?\N:\: Failed to open directory: \\?\N:\: Readdir \\?\N:\: The system cannot find the path specified.

Transferred:            0 Bytes (   0.00 kByte/s)
Errors:                 1
Checks:                 0
Transferred:            0
Elapsed time:           0

N:\>

I'm using Windows 7 Professional x64, encfs4win, and Dokan 0.6.0

encfs.zip

DokanInstall_0.6.0.zip

@ncw
Owner
ncw commented Dec 29, 2015

Hmm looks like it might be related to @klauspost windows file name unification? Klaus?

@ncw ncw added the bug label Jan 2, 2016
@klauspost
Collaborator

@garypaduana - This is likely a Dokan bug. It is old and unmaintained, just as encfs4win, if you are using this version.

Have you tried some of the more recent EncFS software like this: http://encfsmp.sourceforge.net ?

@garypaduana

@klauspost Thank you for this suggestion but unfortunately it also does not work. It is possibly a step in the right direction, though, since drives mounted with EncfsMP are considered to be local drives and not removable storage.

Also, EncfsMP does not support the --reverse option, which would break the current backup workflow I'm using.

My current workaround is to share the folders over smb then run encfs and rclone from a linux VM.

Thanks
disks
encfsmp
rclone

@klauspost
Collaborator

Does it work, if you clone from a subdirectory?

\\?\R:\ does seem a bit strange. It could be a general problem with cloning from the drive root.

@klauspost klauspost self-assigned this Jan 2, 2016
@garypaduana

Yes, it does work when using EncfsMP, but not with Dokan.

EncfsMP:

screen shot 2016-01-02 at 12 50 18 pm

Dokan (with --reverse)

screen shot 2016-01-02 at 12 50 51 pm

@garypaduana

Also, I'm able to programmatically access the mounted drive from the root path as seen here with Groovy:

groovy

@klauspost
Collaborator

It must be programmatically accessible using a UNC path.

All paths are converted to UNC paths, otherwise we get into trouble with paths longer than 255 bytes.

I will look at the EncfsMP root problem tomorrow.

@klauspost klauspost added a commit to klauspost/rclone that referenced this issue Jan 4, 2016
@klauspost klauspost Add local file system option to disable UNC on Windows.
This will add an option to disable UNC conversion on Windows to deal with buggy file system implementations like EncFS.

Fixes #261
7ecd021
@klauspost
Collaborator

Really the file system drivers should be fixed.

However, since that probably isn't something you can do, I have added an option to disable UNC path conversion. See PR #271.

@klauspost klauspost added a commit to klauspost/rclone that referenced this issue Jan 4, 2016
@klauspost @klauspost klauspost + klauspost Add local file system option to disable UNC on Windows.
This will add an option to disable UNC conversion on Windows to deal with buggy file system implementations like EncFS.

Fixes #261
f106a7b
@garypaduana

Thank you very much for the fix! I look forward to using it.

@klauspost
Collaborator

@garypaduana - I have attached a build with the nounc option for you to test. I only have EncFSMP,

rclone-nounc-test.zip

Documentation is here: https://github.com/klauspost/rclone/blob/add-nounc-option/docs/content/local.md#long-paths-on-windows

@ncw ncw added this to the v1.27 milestone Jan 5, 2016
@klauspost klauspost added a commit that closed this issue Jan 5, 2016
@klauspost @ncw klauspost + Add local file system option to disable UNC on Windows.
This will add an option to disable UNC conversion on Windows to deal with buggy file system implementations like EncFS.

Fixes #261
3c31d71
@klauspost klauspost closed this in 3c31d71 Jan 5, 2016
@garypaduana

@klauspost I tried to use the fix this morning but unfortunately it doesn't seem to have worked. As you can see it is no longer using the UNC path but still cannot find the path specified in the encfs drive.

I'm able to access the mounted drive with every other software I frequently use (SyncBackPro, Amazon Cloud Drive Official Client, VLC, Filezilla FTP Server, SMB shares, various image viewers, 7zip, Search Everything, and so on). Is there anything else you'd like me to try?

Thank you for your continued effort but if you want to put this on hold I understand.

Does the additional colon at the end of the path have anything to do with this? e.g. N:\:

N:\>rclone ls .
2016/01/05 15:04:25 Local file system at N:\: Failed to open directory: N:\: Readdir N:\: The system cannot find the path specified.

Transferred:            0 Bytes (   0.00 kByte/s)
Errors:                 1
Checks:                 0
Transferred:            0
Elapsed time:           0

Trying again with a subdirectory:

N:\>cd "PjxNNGtyqk8EISnAH58A,fI,"

N:\PjxNNGtyqk8EISnAH58A,fI,>dir
 Volume in drive N is DOKAN

 Directory of N:\PjxNNGtyqk8EISnAH58A,fI,

01/02/2015  12:53 PM    <DIR>          2haehT9H7yiX1nj9aqo1lOOA
01/02/2015  12:53 PM    <DIR>          ejb0kh2oADu6vrJqiqcyq-Yn
01/02/2015  12:54 PM    <DIR>          -yKOSOU2DrPVaZ1i1-P7Ox3e
01/02/2015  12:55 PM    <DIR>          8hmzPOFgvBj,st4UCVbvP8Wr
01/02/2015  12:55 PM    <DIR>          qPrXtL3uqPmYk5-GO-Q7fHiP
01/02/2015  12:56 PM    <DIR>          4zrbvqYzJBzYcCsWxiT66W8r
01/02/2015  12:56 PM    <DIR>          YF23wJEzBDUbWhEqVaj4Upky
01/02/2015  12:57 PM    <DIR>          3,92vmHkguZXZgaGSmJd,fqy
               0 File(s)              0 bytes
               8 Dir(s)  363,137,863,680 bytes free

N:\PjxNNGtyqk8EISnAH58A,fI,>rclone ls .
2016/01/05 15:06:43 Local file system at N:\PjxNNGtyqk8EISnAH58A,fI,: Failed to open directory: N:\PjxNNGtyqk8EISnAH58A,fI,: Readdir N:\PjxNNGtyqk8EISnAH58A,fI,: The system cannot find the path specified.

Transferred:            0 Bytes (   0.00 kByte/s)
Errors:                 1
Checks:                 0
Transferred:            0
Elapsed time:           0
@klauspost klauspost reopened this Jan 6, 2016
@klauspost
Collaborator

The colon is added by the error printer, so it is not part of the requested path.

It seems like this check in Go file_windows.go is failing. That is the error returned. My guess is that Dokan/EncFS is returning the wrong information. I will create a test binary for you.

@klauspost
Collaborator

Here is a small test to run, that might give us something more to work with: test.zip

I have put the source code here if you would rather compile it yourself.

Please run it, and paste the output here, then hopefully we have something to work with.

@garypaduana
N:\>c:/users/gary/desktop/test.exe
Current dir: "N:\"
Main dir Stat:
Name "\"
Size 0
IsDir true
Mode.IsDir true
Mode.IsRegular false
Mode.String drwxrwxrwx
Mode.Perm: -rwxrwxrwx
2016/01/06 23:12:59 ioutil.ReadDirReaddir N:\: The system cannot find the path specified.

N:\>
@klauspost
Collaborator

ok, it is somewhere in the directory reading code, most likely Dokan returning a wrong value somewhere.

The only way I can see this triggering is if opening the directory as a file does NOT return an error.

Go first tries to open a file as a plain file, if that fails, it tries as a directory before failing. It seems like Dokan/encfs reports "success" when opening z:\ as a file, which it shouldn't, since it isn't a file.

There isn't much we can do from here. You can try filing a bug for Go, but I doubt they will include a workaround for (what I expect to be) a buggy filesystem driver. It may be fixed in DokanY, but I don't know if encfs4win has been compiled for DokanY.

Another workaround that could work, could be to share the drive, and access it as \\localmachine\share, though that is unlikely to make any difference.

@klauspost klauspost closed this Jan 7, 2016
@garypaduana

@klauspost I appreciate all the work you put into investigating this. I understand I'm using old software that hasn't been updated for years but only Dokan 0.6.0 is supported by encfs4win. As I mentioned earlier my workaround is to share the drive and run encfs and rclone inside an ubuntu VM. This works great and is really just a minor inconvenience.

Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment