Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rclone complains when updating a file on OneDrive - overwriting/replacing #3702

Open
rclone-user opened this issue Nov 7, 2019 · 6 comments
Milestone

Comments

@rclone-user
Copy link

@rclone-user rclone-user commented Nov 7, 2019

I posted this on the rclone forum and was advised to create a bug report here. Below is all the relevant information.

Description
When I run rclone (in copy mode) it complains when the file already exists giving the following error message:
2019/11/03 00:08:08 ERROR : Business/Direct/DirectClients.xlsx: Failed to copy: nameAlreadyExists: An upload session already exists for this file (is it a OneNote file?)

It happens for any type of file e.g. text, shell script, Office type files and I believe that it happens every time. If the file does not exist on OneDrive then I have never seen a problem.

I looked on the forum and saw similar questions e.g. same error message when spaces or hyphens are in the filename but not the exact same issue as I have i.e. any existing file.

Please not that I may be a little slow in responding to answers over the next couple of days so please accept my apologies in advance.

rclone v1.50.1 (it was originally 1.48 but I have upgraded to the latest stable version and no change in behaviour.

os/arch: darwin/amd64
go version: go1.13.4

OS
MacOS
Darwin Kernel Version 17.7.0: Sun Jun 2 20:31:42 PDT 2019; root:xnu-4570.71.46~1/RELEASE_X86_64 x86_64

Cloud storage system
I am using OneDrive supplied as part with MS Exchange Business subscription.

Command
rclone copy -vv /Documents OneDriveCloud:iMac/Documents --checksum --retries 6 --exclude '[.]*' --log-file=/tmp/rclone-mac-docs.log

Logfile extract
9/11/03 00:08:07 ERROR : Business/CV/.DS_Store: Failed to copy: nameAlreadyExists: An upload session already exists for this file (is it a OneNote file?)

2019/11/03 00:08:07 NOTICE: Administration/iCloud/Links/iCloudBackupDestDirLnk: Can't follow symlink without -L/--copy-links

2019/11/03 00:08:08 ERROR : Business/Direct/DirectClients.xlsx: Failed to copy: nameAlreadyExists: An upload session already exists for this file (is it a OneNote file?)

2019/11/03 00:08:08 ERROR : Office/.DS_Store: Failed to copy: nameAlreadyExists: An upload session already exists for this file (is it a OneNote file?)

2019/11/03 00:08:09 INFO : Tmp/OutlookExport/Outlook/.DS_Store: Copied (new)

Other Points
I have tried to look for a case for the error nameAlreadyExists but cannot find it.
I also tried to find what an "upload session" is to see if I can remove existing ones but could not find any description.
I tried various flags e.g. --ignore-checksum and these reduce errors by reducing the number of files it has to copy.
The problem always seems to be with an existing file i.e. overwriting and MS Office ones seem to be a particular problem but I am unable to say if it is every file that needs to be overwritten but it is not just limited to MS Office files.

@rclone-user

This comment has been minimized.

Copy link
Author

@rclone-user rclone-user commented Nov 7, 2019

Here is a link to the forum, sorry for not adding it to the original post.
https://forum.rclone.org/t/rclone-complains-when-updating-a-file-on-onedrive/12769/9

@ncw ncw added this to the v1.51 milestone Nov 8, 2019
@ncw

This comment has been minimized.

Copy link
Collaborator

@ncw ncw commented Nov 8, 2019

@Cnly I'm not really sure what is going on here - can you help?

@rclone-user

This comment has been minimized.

Copy link
Author

@rclone-user rclone-user commented Nov 10, 2019

If it is any help. I mounted the OneDrive using rclone mount remote: command. I was then able to use cp srcfile destfile to copy a new file across and then updated the source file and copied it across again with issues. I do get a warning about extended attributes not supported.

I'd still be interested to know if the nameAlreadyExists error message comes from rclone or not.

If there is something else you want me to try let me know.

@ncw

This comment has been minimized.

Copy link
Collaborator

@ncw ncw commented Nov 12, 2019

The nameAlreadyExists error comes from OneDrive.

Here is a simplistic attempt by me to reproduce your problem. Does this reproduce the problem for you? If not can you make it reproduce the problem?

$ cd /tmp/
$ echo hello > file.txt
$ rclone copy -vv file.txt TestOneDrive:testFiles
2019/11/12 13:40:01 DEBUG : rclone: Version "v1.50.1-035-gc21a4fee-beta" starting with parameters ["rclone" "copy" "-vv" "file.txt" "TestOneDrive:testFiles"]
2019/11/12 13:40:01 DEBUG : Using config file from "/home/ncw/.rclone.conf"
2019/11/12 13:40:03 DEBUG : file.txt: Need to transfer - File not found at Destination
2019/11/12 13:40:04 DEBUG : file.txt: Starting multipart upload
2019/11/12 13:40:05 DEBUG : file.txt: Uploading segment 0/6 size 6
2019/11/12 13:40:06 DEBUG : file.txt: SHA-1 = f572d396fae9206628714fb2ce00f72e94f2258f OK
2019/11/12 13:40:06 INFO  : file.txt: Copied (new)
2019/11/12 13:40:06 INFO  : 
Transferred:   	         6 / 6 Bytes, 100%, 2 Bytes/s, ETA 0s
Errors:                 0
Checks:                 0 / 0, -
Transferred:            1 / 1, 100%
Elapsed time:        2.8s

2019/11/12 13:40:06 DEBUG : 10 go routines active
2019/11/12 13:40:06 DEBUG : rclone: Version "v1.50.1-035-gc21a4fee-beta" finishing with parameters ["rclone" "copy" "-vv" "file.txt" "TestOneDrive:testFiles"]

$ rclone lsf TestOneDrive:testFiles
file.txt

$ echo hellohello > file.txt

$ rclone copy -vv file.txt TestOneDrive:testFiles
2019/11/12 13:40:28 DEBUG : rclone: Version "v1.50.1-035-gc21a4fee-beta" starting with parameters ["rclone" "copy" "-vv" "file.txt" "TestOneDrive:testFiles"]
2019/11/12 13:40:28 DEBUG : Using config file from "/home/ncw/.rclone.conf"
2019/11/12 13:40:28 DEBUG : file.txt: Sizes differ (src 11 vs dst 6)
2019/11/12 13:40:28 DEBUG : file.txt: Starting multipart upload
2019/11/12 13:40:29 DEBUG : file.txt: Uploading segment 0/11 size 11
2019/11/12 13:40:30 DEBUG : file.txt: SHA-1 = 60e4cc679a5c273df21513400ccc98dcf88f3df1 OK
2019/11/12 13:40:30 INFO  : file.txt: Copied (replaced existing)
2019/11/12 13:40:30 INFO  : 
Transferred:   	        11 / 11 Bytes, 100%, 8 Bytes/s, ETA 0s
Errors:                 0
Checks:                 0 / 0, -
Transferred:            1 / 1, 100%
Elapsed time:        1.3s

2019/11/12 13:40:30 DEBUG : 8 go routines active
2019/11/12 13:40:30 DEBUG : rclone: Version "v1.50.1-035-gc21a4fee-beta" finishing with parameters ["rclone" "copy" "-vv" "file.txt" "TestOneDrive:testFiles"]

$ rclone lsl TestOneDrive:testFiles
       11 2019-11-12 13:40:22.000000000 file.txt
@rclone-user

This comment has been minimized.

Copy link
Author

@rclone-user rclone-user commented Nov 12, 2019

@ncw

This comment has been minimized.

Copy link
Collaborator

@ncw ncw commented Nov 12, 2019

I looked to see if it was something to do with versions but no luck there. I will try some other things tonight and if I spot any trend I’ll email you again.

Thanks! If we can reproduce this it will make it much much easier to fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.