Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dropbox and Box move not working if destination file exists #2569

Closed
brechmos opened this issue Sep 11, 2018 · 12 comments
Closed

Dropbox and Box move not working if destination file exists #2569

brechmos opened this issue Sep 11, 2018 · 12 comments
Assignees
@ncw
Labels
bug VFS / mount
Milestone
v1.44

Comments

@brechmos
Copy link

brechmos commented Sep 11, 2018 •
edited

What is the problem you are having with rclone?

I am trying to move a file on top of another one and I am getting input/output errors.

$ cd /tmp/db
$ touch a b  # Make two empty files.
$ mv a b
mv: cannot move 'a' to 'b': Input/output error

What is your rclone version (eg output from rclone -V)

rclone v1.43
- os/arch: linux/arm
- go version: go1.11

Which OS you are using and how many bits (eg Windows 7, 64 bit)

ODROID Arm :)

Which cloud storage system are you using? (eg Google Drive)

Box and Dropbox are the ones not working (see below for others that are).

The command you were trying to run (eg rclone copy /tmp remote:tmp)

$ cd /tmp/db
$ touch a b  # Make two empty files.
$ mv a b
mv: cannot move 'a' to 'b': Input/output error

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

This is the mounting:

$ ~/bin/rclone mount -v --debug-fuse  Dropbox: /tmp/db
2018/09/11 07:44:27 INFO  : Dropbox root '': poll-interval is not supported by this remote
2018/09/11 07:44:37 ERROR : Test/a: File.Rename error: move failed: to/conflict/file/..
2018/09/11 07:44:37 ERROR : Test/a: Dir.Rename error: move failed: to/conflict/file/..

The error above came when I did:

$ cd /tmp/db
$ touch a b  # Make two empty files.
$ mv a b
mv: cannot move 'a' to 'b': Input/output error

I am quite new at rclone, so there might be some helpful other flags but with the reasonably bare mount command I am seeing an error.

Oh... so, interesting. I just did the same thing with Mega and it worked:

$ ~/bin/rclone mount -v --debug-fuse  Mega: /tmp/db
2018/09/11 07:48:42 INFO  : mega root '': poll-interval is not supported by this remote
2018/09/11 07:48:56 INFO  : test/a: Copied (new)
2018/09/11 07:48:57 INFO  : test/b: Copied (new)

It also works with pCloud:

$ ~/bin/rclone mount -v --debug-fuse pCloud: /tmp/db
2018/09/11 07:50:05 INFO  : pcloud root '': poll-interval is not supported by this remote
2018/09/11 07:50:23 INFO  : test/a: Copied (new)
2018/09/11 07:50:23 INFO  : test/b: Copied (new)

It also worked with Google Drive.

Tried Box and it did not work.

Huh, so it seems to be only Box and Dropbox (of the ones I tested) that are giving me issues.

All of the above is based on:

rclone v1.43
- os/arch: linux/arm
- go version: go1.11

If you want me to test a PR, I am happy to do it.

Detailed log:

2018/09/11 17:40:20 DEBUG : fuse: <- Lookup [ID=0xc2 Node=0x1e Uid=1000 Gid=1000 Pid=15296] "a"                                                           
2018/09/11 17:40:20 DEBUG : test/: Lookup: name="a"                                                                                                       
2018/09/11 17:40:20 DEBUG : test/: >Lookup: node=<nil>, err=no such file or directory                                                                     
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xc2] Lookup error=ENOENT                                                                                        
2018/09/11 17:40:20 DEBUG : fuse: <- Create [ID=0xc3 Node=0x1e Uid=1000 Gid=1000 Pid=15296] "a" fl=OpenWriteOnly+OpenCreate+OpenNonblock+0x20000 mode=-rw-
rw-r-- umask=--------w-                                                                                                                                   
2018/09/11 17:40:20 DEBUG : test/: Create: name="a"                                                                                                       
2018/09/11 17:40:20 DEBUG : test/a: Open: flags=O_WRONLY|O_CREATE|0x20800                                                                                 
2018/09/11 17:40:20 DEBUG : test/a(0x163e0c0): Opening cached copy with flags=O_WRONLY|O_CREATE|0x20800                                                   
2018/09/11 17:40:20 DEBUG : test/a: >Open: fd=test/a (rw), err=<nil>                                                                                      
2018/09/11 17:40:20 DEBUG : test/: >Create: node=test/a, handle=&{test/a (rw)}, err=<nil>                                                                 
2018/09/11 17:40:20 DEBUG : test/a: Attr:                                                                                                                 
2018/09/11 17:40:20 DEBUG : test/a: >Attr: a=valid=1s ino=0 size=0 mode=-rw-rw-r--, err=<nil>                                                             
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xc3] Create {0x1b gen=55 valid=1m0s attr={valid=1s ino=486905183565473304 size=0 mode=-rw-rw-r--}} {0x1 fl=0}   
2018/09/11 17:40:20 DEBUG : fuse: <- Flush [ID=0xc4 Node=0x1b Uid=1000 Gid=1000 Pid=15296] 0x1 fl=0x0 lk=0x4cc929253ff572b                                
2018/09/11 17:40:20 DEBUG : &{test/a (rw)}: Flush:                                                                                                        
2018/09/11 17:40:20 DEBUG : test/a(0x163e0c0): RWFileHandle.Flush ignoring flush on unwritten handle                                                      
2018/09/11 17:40:20 DEBUG : &{test/a (rw)}: >Flush: err=<nil>                                                                                             
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xc4] Flush                                                                                                      
2018/09/11 17:40:20 DEBUG : fuse: <- Setattr [ID=0xc5 Node=0x1b Uid=1000 Gid=1000 Pid=15296] atime=2018-09-11 17:40:20.233985246 -0400 EDT atime=now mtime
=2018-09-11 17:40:20.233985246 -0400 EDT mtime=now handle=INVALID-0x0                                                                                     
2018/09/11 17:40:20 DEBUG : test/a: Setattr: a=Setattr [ID=0xc5 Node=0x1b Uid=1000 Gid=1000 Pid=15296] atime=2018-09-11 17:40:20.233985246 -0400 EDT atime
=now mtime=2018-09-11 17:40:20.233985246 -0400 EDT mtime=now handle=INVALID-0x0                                                                           
2018/09/11 17:40:20 DEBUG : test/a: >Setattr: err=<nil>                                                                                                   
2018/09/11 17:40:20 DEBUG : test/a: Attr:                                                                                                                 
2018/09/11 17:40:20 DEBUG : test/a: >Attr: a=valid=1s ino=0 size=0 mode=-rw-rw-r--, err=<nil>                                                             
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xc5] Setattr valid=1s ino=486905183565473304 size=0 mode=-rw-rw-r--                                             
2018/09/11 17:40:20 DEBUG : fuse: <- Flush [ID=0xc6 Node=0x1b Uid=1000 Gid=1000 Pid=15296] 0x1 fl=0x0 lk=0x4cc929253ff572b                                
2018/09/11 17:40:20 DEBUG : &{test/a (rw)}: Flush:                                                                                                        
2018/09/11 17:40:20 DEBUG : test/a(0x163e0c0): RWFileHandle.Flush ignoring flush on unwritten handle                                                      
2018/09/11 17:40:20 DEBUG : &{test/a (rw)}: >Flush: err=<nil>                                                                                             
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xc6] Flush                                                                                                      
2018/09/11 17:40:20 DEBUG : fuse: <- Release [ID=0xc7 Node=0x1b Uid=0 Gid=0 Pid=0] 0x1 fl=OpenWriteOnly+OpenNonblock+0x20000 rfl=0 owner=0x0              
2018/09/11 17:40:20 DEBUG : fuse: <- Lookup [ID=0xc8 Node=0x1e Uid=1000 Gid=1000 Pid=15296] "b"                                                           
2018/09/11 17:40:20 DEBUG : test/: Lookup: name="b"                                                                                                       
2018/09/11 17:40:20 DEBUG : test/: >Lookup: node=<nil>, err=no such file or directory                                                                     
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xc8] Lookup error=ENOENT                                                                                        
2018/09/11 17:40:20 DEBUG : &{test/a (rw)}: Release:                                                                                                      
2018/09/11 17:40:20 DEBUG : test/a(0x163e0c0): RWFileHandle.Release closing                                                                               
2018/09/11 17:40:20 DEBUG : test/a(0x163e0c0): close:                                                                                                     
2018/09/11 17:40:20 DEBUG : test/a: Couldn't find file - need to transfer                                                          
2018/09/11 17:40:20 DEBUG : fuse: <- Create [ID=0xc9 Node=0x1e Uid=1000 Gid=1000 Pid=15296] "b" fl=OpenWriteOnly+OpenCreate+OpenNonblock+0x20000 mode=-rw-
rw-r-- umask=--------w-
2018/09/11 17:40:20 DEBUG : test/: Create: name="b"
2018/09/11 17:40:20 DEBUG : test/b: Open: flags=O_WRONLY|O_CREATE|0x20800
2018/09/11 17:40:20 DEBUG : test/b(0x15be000): Opening cached copy with flags=O_WRONLY|O_CREATE|0x20800
2018/09/11 17:40:20 DEBUG : test/b: >Open: fd=test/b (rw), err=<nil>
2018/09/11 17:40:20 DEBUG : test/: >Create: node=test/b, handle=&{test/b (rw)}, err=<nil>
2018/09/11 17:40:20 DEBUG : test/b: Attr:
2018/09/11 17:40:20 DEBUG : test/b: >Attr: a=valid=1s ino=0 size=0 mode=-rw-rw-r--, err=<nil>
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xc9] Create {0x18 gen=56 valid=1m0s attr={valid=1s ino=486908482100357937 size=0 mode=-rw-rw-r--}} {0x1 fl=0}
2018/09/11 17:40:20 DEBUG : fuse: <- Setattr [ID=0xca Node=0x18 Uid=1000 Gid=1000 Pid=15296] atime=2018-09-11 17:40:20.253985452 -0400 EDT atime=now mtime
=2018-09-11 17:40:20.253985452 -0400 EDT mtime=now handle=INVALID-0x0
2018/09/11 17:40:20 DEBUG : test/b: Setattr: a=Setattr [ID=0xca Node=0x18 Uid=1000 Gid=1000 Pid=15296] atime=2018-09-11 17:40:20.253985452 -0400 EDT atime
=now mtime=2018-09-11 17:40:20.253985452 -0400 EDT mtime=now handle=INVALID-0x0
2018/09/11 17:40:20 DEBUG : test/b: >Setattr: err=<nil>
2018/09/11 17:40:20 DEBUG : test/b: Attr:
2018/09/11 17:40:20 DEBUG : test/b: >Attr: a=valid=1s ino=0 size=0 mode=-rw-rw-r--, err=<nil>
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xca] Setattr valid=1s ino=486908482100357937 size=0 mode=-rw-rw-r--
2018/09/11 17:40:20 DEBUG : fuse: <- Flush [ID=0xcb Node=0x18 Uid=1000 Gid=1000 Pid=15296] 0x1 fl=0x0 lk=0x4cc929253ff572b
2018/09/11 17:40:20 DEBUG : &{test/b (rw)}: Flush:
2018/09/11 17:40:20 DEBUG : test/b(0x15be000): RWFileHandle.Flush ignoring flush on unwritten handle
2018/09/11 17:40:20 DEBUG : &{test/b (rw)}: >Flush: err=<nil>
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xcb] Flush
2018/09/11 17:40:20 DEBUG : fuse: <- Release [ID=0xcc Node=0x18 Uid=0 Gid=0 Pid=0] 0x1 fl=OpenWriteOnly+OpenNonblock+0x20000 rfl=0 owner=0x0
2018/09/11 17:40:20 DEBUG : &{test/b (rw)}: Release:
2018/09/11 17:40:20 DEBUG : test/b(0x15be000): RWFileHandle.Release closing
2018/09/11 17:40:20 DEBUG : test/b(0x15be000): close:
2018/09/11 17:40:20 DEBUG : test/b: Couldn't find file - need to transfer
2018/09/11 17:40:20 DEBUG : fuse: <- Getattr [ID=0xcd Node=0x1e Uid=1000 Gid=1000 Pid=15339] 0x0 fl=0
2018/09/11 17:40:20 DEBUG : test/: Attr:
2018/09/11 17:40:20 DEBUG : test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xcd] Getattr valid=1s ino=14597221540505887712 size=0 mode=drwxrwxr-x
2018/09/11 17:40:20 DEBUG : fuse: <- Access [ID=0xce Node=0x1e Uid=1000 Gid=1000 Pid=15307] mask=0x4
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xce] Access
2018/09/11 17:40:20 DEBUG : fuse: <- Lookup [ID=0xcf Node=0x1e Uid=1000 Gid=1000 Pid=15342] ".git"
2018/09/11 17:40:20 DEBUG : test/: Lookup: name=".git"
2018/09/11 17:40:20 DEBUG : test/: >Lookup: node=<nil>, err=no such file or directory
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xcf] Lookup error=ENOENT
2018/09/11 17:40:20 DEBUG : fuse: <- Lookup [ID=0xd0 Node=0x1e Uid=1000 Gid=1000 Pid=15342] ".git"
2018/09/11 17:40:20 DEBUG : test/: Lookup: name=".git"
2018/09/11 17:40:20 DEBUG : test/: >Lookup: node=<nil>, err=no such file or directory
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xd0] Lookup error=ENOENT
2018/09/11 17:40:20 DEBUG : fuse: <- Lookup [ID=0xd1 Node=0x1e Uid=1000 Gid=1000 Pid=15342] "HEAD"
2018/09/11 17:40:20 DEBUG : test/: Lookup: name="HEAD"
2018/09/11 17:40:20 DEBUG : test/: >Lookup: node=<nil>, err=no such file or directory
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xd1] Lookup error=ENOENT
2018/09/11 17:40:20 DEBUG : fuse: <- Getattr [ID=0xd2 Node=0x1 Uid=1000 Gid=1000 Pid=15342] 0x0 fl=0
2018/09/11 17:40:20 DEBUG : /: Attr:
2018/09/11 17:40:20 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xd2] Getattr valid=1s ino=1 size=0 mode=drwxrwxr-x
2018/09/11 17:40:20 DEBUG : fuse: <- Lookup [ID=0xd3 Node=0x1 Uid=1000 Gid=1000 Pid=15342] ".git"
2018/09/11 17:40:20 DEBUG : /: Lookup: name=".git"
2018/09/11 17:40:20 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xd3] Lookup error=ENOENT
2018/09/11 17:40:20 DEBUG : fuse: <- Lookup [ID=0xd4 Node=0x1 Uid=1000 Gid=1000 Pid=15342] ".git"
2018/09/11 17:40:20 DEBUG : /: Lookup: name=".git"
2018/09/11 17:40:20 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xd4] Lookup error=ENOENT
2018/09/11 17:40:20 DEBUG : fuse: <- Lookup [ID=0xd5 Node=0x1 Uid=1000 Gid=1000 Pid=15342] "HEAD"
2018/09/11 17:40:20 DEBUG : /: Lookup: name="HEAD"
2018/09/11 17:40:20 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xd5] Lookup error=ENOENT
2018/09/11 17:40:20 INFO  : test/a: Copied (new)
2018/09/11 17:40:20 DEBUG : test/a: transferred to remote
2018/09/11 17:40:20 DEBUG : test/a(0x163e0c0): >close: err=<nil>
2018/09/11 17:40:20 DEBUG : &{test/a (rw)}: >Release: err=<nil>
2018/09/11 17:40:20 DEBUG : fuse: -> [ID=0xc7] Release
2018/09/11 17:40:21 INFO  : test/b: Copied (new)
2018/09/11 17:40:21 DEBUG : test/b: transferred to remote
2018/09/11 17:40:21 DEBUG : test/b(0x15be000): >close: err=<nil>
2018/09/11 17:40:21 DEBUG : &{test/b (rw)}: >Release: err=<nil>
2018/09/11 17:40:21 DEBUG : fuse: -> [ID=0xcc] Release
2018/09/11 17:40:21 DEBUG : fuse: <- Getattr [ID=0xd6 Node=0x18 Uid=1000 Gid=1000 Pid=15361] 0x0 fl=0
2018/09/11 17:40:21 DEBUG : test/b: Attr:
2018/09/11 17:40:21 DEBUG : test/b: >Attr: a=valid=1s ino=0 size=0 mode=-rw-rw-r--, err=<nil>
2018/09/11 17:40:21 DEBUG : fuse: -> [ID=0xd6] Getattr valid=1s ino=486908482100357937 size=0 mode=-rw-rw-r--
2018/09/11 17:40:21 DEBUG : fuse: <- Getattr [ID=0xd7 Node=0x1b Uid=1000 Gid=1000 Pid=15361] 0x0 fl=0
2018/09/11 17:40:21 DEBUG : test/a: Attr:
2018/09/11 17:40:21 DEBUG : test/a: >Attr: a=valid=1s ino=0 size=0 mode=-rw-rw-r--, err=<nil>
2018/09/11 17:40:21 DEBUG : fuse: -> [ID=0xd7] Getattr valid=1s ino=486905183565473304 size=0 mode=-rw-rw-r--
2018/09/11 17:40:21 DEBUG : fuse: <- Access [ID=0xd8 Node=0x18 Uid=1000 Gid=1000 Pid=15361] mask=0x2
2018/09/11 17:40:21 DEBUG : fuse: -> [ID=0xd8] Access
2018/09/11 17:40:21 DEBUG : fuse: <- Rename [ID=0xd9 Node=0x1e Uid=1000 Gid=1000 Pid=15361] from "a" to dirnode 0x1e "b"
2018/09/11 17:40:21 DEBUG : test/: Rename: oldName="a", newName="b", newDir=test/
2018/09/11 17:40:21 ERROR : test/a: File.Rename error: move failed: to/conflict/file/..
2018/09/11 17:40:21 ERROR : test/a: Dir.Rename error: move failed: to/conflict/file/..
2018/09/11 17:40:21 DEBUG : test/: >Rename: err=move failed: to/conflict/file/..
2018/09/11 17:40:21 DEBUG : fuse: -> [ID=0xd9] Rename error=EIO: move failed: to/conflict/file/..
2018/09/11 17:40:21 DEBUG : fuse: <- Getattr [ID=0xda Node=0x1e Uid=1000 Gid=1000 Pid=15406] 0x0 fl=0
2018/09/11 17:40:21 DEBUG : test/: Attr:
2018/09/11 17:40:21 DEBUG : test/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2018/09/11 17:40:21 DEBUG : fuse: -> [ID=0xda] Getattr valid=1s ino=14597221540505887712 size=0 mode=drwxrwxr-x
2018/09/11 17:40:21 DEBUG : fuse: <- Access [ID=0xdb Node=0x1e Uid=1000 Gid=1000 Pid=15394] mask=0x4
2018/09/11 17:40:21 DEBUG : fuse: -> [ID=0xdb] Access
2018/09/11 17:40:21 DEBUG : fuse: <- Lookup [ID=0xdc Node=0x1e Uid=1000 Gid=1000 Pid=15407] ".git"
2018/09/11 17:40:21 DEBUG : test/: Lookup: name=".git"
2018/09/11 17:40:21 DEBUG : test/: >Lookup: node=<nil>, err=no such file or directory
2018/09/11 17:40:21 DEBUG : fuse: -> [ID=0xdc] Lookup error=ENOENT
2018/09/11 17:40:21 DEBUG : fuse: <- Lookup [ID=0xdd Node=0x1e Uid=1000 Gid=1000 Pid=15407] ".git"
2018/09/11 17:40:21 DEBUG : test/: Lookup: name=".git"
2018/09/11 17:40:21 DEBUG : test/: >Lookup: node=<nil>, err=no such file or directory
2018/09/11 17:40:21 DEBUG : fuse: -> [ID=0xdd] Lookup error=ENOENT
2018/09/11 17:40:21 DEBUG : fuse: <- Lookup [ID=0xde Node=0x1e Uid=1000 Gid=1000 Pid=15407] "HEAD"
2018/09/11 17:40:21 DEBUG : test/: Lookup: name="HEAD"
2018/09/11 17:40:21 DEBUG : test/: >Lookup: node=<nil>, err=no such file or directory
2018/09/11 17:40:21 DEBUG : fuse: -> [ID=0xde] Lookup error=ENOENT
2018/09/11 17:40:21 DEBUG : fuse: <- Getattr [ID=0xdf Node=0x1 Uid=1000 Gid=1000 Pid=15407] 0x0 fl=0
2018/09/11 17:40:21 DEBUG : /: Attr:
2018/09/11 17:40:21 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2018/09/11 17:40:21 DEBUG : fuse: -> [ID=0xdf] Getattr valid=1s ino=1 size=0 mode=drwxrwxr-x
2018/09/11 17:40:21 DEBUG : fuse: <- Lookup [ID=0xe0 Node=0x1 Uid=1000 Gid=1000 Pid=15407] ".git"
2018/09/11 17:40:21 DEBUG : /: Lookup: name=".git"
2018/09/11 17:40:21 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2018/09/11 17:40:21 DEBUG : fuse: -> [ID=0xe0] Lookup error=ENOENT
2018/09/11 17:40:21 DEBUG : fuse: <- Lookup [ID=0xe1 Node=0x1 Uid=1000 Gid=1000 Pid=15407] ".git"
2018/09/11 17:40:21 DEBUG : /: Lookup: name=".git"
2018/09/11 17:40:21 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2018/09/11 17:40:21 DEBUG : fuse: -> [ID=0xe1] Lookup error=ENOENT
2018/09/11 17:40:21 DEBUG : fuse: <- Lookup [ID=0xe2 Node=0x1 Uid=1000 Gid=1000 Pid=15407] "HEAD"
2018/09/11 17:40:21 DEBUG : /: Lookup: name="HEAD"
2018/09/11 17:40:21 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2018/09/11 17:40:21 DEBUG : fuse: -> [ID=0xe2] Lookup error=ENOENT
@brechmos
Copy link
Author

brechmos commented Sep 11, 2018 •
edited

As a side note/question. I am confused why it keeps looking for ".git" and "HEAD". In the test directory there is no git stuff.

@brechmos brechmos mentioned this issue Sep 11, 2018
Closed
@ncw ncw added this to the v1.44 milestone Sep 12, 2018
@ncw
Copy link
Member

ncw commented Sep 12, 2018

rclone move works OK

$ date | rclone rcat dropbox:test/a
$ date | rclone rcat dropbox:test/b
$ rclone -vv moveto dropbox:test/a  dropbox:test/b
2018/09/12 08:02:00 DEBUG : rclone: Version "v1.43-DEV" starting with parameters ["rclone" "-vv" "moveto" "dropbox:test/a" "dropbox:test/b"]
2018/09/12 08:02:00 DEBUG : Using config file from "/home/ncw/.rclone.conf"
2018/09/12 08:02:03 DEBUG : a: Modification times differ by 3s: 2018-09-12 07:01:51 +0000 UTC, 2018-09-12 07:01:54 +0000 UTC
2018/09/12 08:02:03 DEBUG : a: DropboxHash = 4a24055121b9a51c6bf997a64484c472083a5491107224b95ddc49dcd426db1d (Dropbox root 'test')
2018/09/12 08:02:03 DEBUG : b: DropboxHash = d03c4e8d92c7fd479974fae89ca7d7081a8d1e7d48218489a93758af60967fff (Dropbox root 'test')
2018/09/12 08:02:03 DEBUG : a: DropboxHash differ
2018/09/12 08:02:04 INFO  : b: Deleted
2018/09/12 08:02:05 INFO  : a: Moved (server side)
2018/09/12 08:02:05 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:                 1 / 1, 100%
Transferred:            1 / 1, 100%
Elapsed time:        4.2s

2018/09/12 08:02:05 DEBUG : 4 go routines active
2018/09/12 08:02:05 DEBUG : rclone: Version "v1.43-DEV" finishing with parameters ["rclone" "-vv" "moveto" "dropbox:test/a" "dropbox:test/b"]

However mv in a mount doesn't

$ date | rclone rcat dropbox:test/a
$ date | rclone rcat dropbox:test/b
$ rclone mount -vv dropbox:test /mnt/tmp

$ mv /mnt/tmp/a  /mnt/tmp/b
mv: cannot move '/mnt/tmp/a' to '/mnt/tmp/b': Input/output error

Gives

2018/09/12 08:02:57 DEBUG : rclone: Version "v1.43-DEV" starting with parameters ["rclone" "mount" "-vv" "dropbox:test" "/mnt/tmp"]
2018/09/12 08:02:57 DEBUG : Using config file from "/home/ncw/.rclone.conf"
2018/09/12 08:02:57 DEBUG : Dropbox root 'test': Mounting on "/mnt/tmp"
2018/09/12 08:02:57 INFO  : Dropbox root 'test': poll-interval is not supported by this remote
2018/09/12 08:02:57 DEBUG : Adding path "vfs/forget" to remote control registry
2018/09/12 08:02:57 DEBUG : Adding path "vfs/refresh" to remote control registry
2018/09/12 08:02:57 DEBUG : : Root: 
2018/09/12 08:02:57 DEBUG : : >Root: node=/, err=<nil>
2018/09/12 08:02:57 DEBUG : /: Lookup: name=".Trash"
2018/09/12 08:02:58 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2018/09/12 08:02:58 DEBUG : /: Lookup: name=".Trash-502"
2018/09/12 08:02:58 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory
2018/09/12 08:03:10 DEBUG : /: Lookup: name="b"
2018/09/12 08:03:10 DEBUG : /: >Lookup: node=b, err=<nil>
2018/09/12 08:03:10 DEBUG : b: Attr: 
2018/09/12 08:03:10 DEBUG : b: >Attr: a=valid=1s ino=0 size=29 mode=-rw-rw-r--, err=<nil>
2018/09/12 08:03:10 DEBUG : /: Lookup: name="a"
2018/09/12 08:03:10 DEBUG : /: >Lookup: node=a, err=<nil>
2018/09/12 08:03:10 DEBUG : a: Attr: 
2018/09/12 08:03:10 DEBUG : a: >Attr: a=valid=1s ino=0 size=29 mode=-rw-rw-r--, err=<nil>
2018/09/12 08:03:10 DEBUG : /: Rename: oldName="a", newName="b", newDir=/
2018/09/12 08:03:10 ERROR : a: File.Rename error: move failed: to/conflict/file/...
2018/09/12 08:03:10 ERROR : a: Dir.Rename error: move failed: to/conflict/file/...
2018/09/12 08:03:10 DEBUG : /: >Rename: err=move failed: to/conflict/file/...
^C2018/09/12 08:03:14 DEBUG : rclone: Version "v1.43-DEV" finishing with parameters ["rclone" "mount" "-vv" "dropbox:test" "/mnt/tmp"]
$

The solution to this (and partially #1965) is to get VFS layer to use the operations.Move primitive rather than calling the Fs Move method directly.

@ncw
Copy link
Member

ncw commented Sep 12, 2018

@brechmos wrote:

As a side note/question. I am confused why it keeps looking for ".git" and "HEAD". In the test directory there is no git stuff.

That is likely your shell - do you have a prompt which shows special things when you are in a git repo?

@ncw
Copy link
Member

ncw commented Sep 12, 2018

@B4dM4n do you think using operations.Move is sensible here? That will mean moving a file will work reliably on all backends, but it will mean that those that don't support Move or Copy will be downloading and uploading the file which is probably unexpected...

@brechmos
Copy link
Author

That is likely your shell - do you have a prompt which shows special things when you are in a git repo?

D'oh, you are right. Sorry for the noise.

@B4dM4n
Copy link
Collaborator

B4dM4n commented Sep 17, 2018

I don't think operations.Move should be used in this case. Using mv on one filesystem should be fast operation and not upload the file again. This is the behavior most people will know and exists in all operating systems.

We could hide operations.Move behind a flag and otherwise use something like operations.MoveOnly to force a server side move.

@ncw
Copy link
Member

ncw commented Sep 18, 2018

We could hide operations.Move behind a flag and otherwise use something like operations.MoveOnly to force a server side move.

Excellent idea - thanks! I'll have a go at that.

@ncw
Copy link
Member

ncw commented Sep 20, 2018

See also forum thread which I think is the same issue but on drive a duplicate is created.

@ncw ncw self-assigned this Sep 20, 2018
ncw added a commit that referenced this issue Sep 20, 2018
@ncw
vfs: enable rename for nearly all remotes using server side Move or Copy
a9fc86f 
Before this change remotes without server side Move (eg swift, s3,
gcs) would not be able to rename files.

After it means nearly all remotes will be able to rename files on
rclone mount with the notable exceptions of b2 and yandex.

This changes checks to see if the remote can do Move or Copy then
calls `operations.Move` to do the actual move.  This will do a server
side Move or Copy but won't download and re-upload the file.

It also checks to see if the destination exists first which avoids
conflicts or duplicates.

Fixes #1965
Fixes #2569
@ncw
Copy link
Member

ncw commented Sep 20, 2018

I have attempted to fix this here

https://beta.rclone.org/branch/v1.43-096-ga9fc86f5-fix-2569-mount-move-beta/ (uploaded in 15-30 mins)

Feedback welcome :-)

@brechmos
Copy link
Author

@ncw I grabbed the branch in github and rebuilt.

I tested both Dropbox and Box. In both I can do the touch a b, mv a b and it appears to work fine. I added content and that works. I also tried mv across directories and that works too. I tried with content in files as well.

I also tried it with a cached mount of Dropbox and Box and they both seem to work.

Mega still works.

In this simple testing, it looks good!

If you have any ideas of edge cases you want me to try, I am happy to do that. Thanks for working on this!!

@ncw
Copy link
Member

ncw commented Sep 20, 2018

@brechmos Great - thanks for testing. I can't think of anything else at the moment!

I'll leave this open for the moment for feedback from the other issue that this fixes.

@ncw ncw closed this as completed in 59e14c2 Sep 29, 2018
@ncw
Copy link
Member

ncw commented Sep 29, 2018

I've merged this fix to master now, it will be in the latest beta in 15-30 mins an in v1.44.

@jimaan jimaan mentioned this issue May 18, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ncw ncw

Labels
bug VFS / mount
Projects
None yet
Milestone
v1.44
Development

No branches or pull requests
3 participants
@B4dM4n @ncw @brechmos