Disable destination checks for copy and sync command

[manishgupta24]

What is your current rclone version (output from rclone version)?

1.49.3

What problem are you are trying to solve?

Copy the files to the destination without performing any checks.

How do you think rclone should be changed to solve that?

Add a new flag --no-check which will skip all the destination checks before copying the file.

[ncw]

Thanks for making this issue. Do you want to have a go at implementing it?

[ivandeex]

I believe that naming this flag --force is more in line with general practice, unless it prevents other rclone subcommands from having like-named flag due to implementation details.

[ncw]

I see what you mean. The major use of this flag would be to reduce checks at the expense of transfers, so how about --force-transfer?

[ivandeex]

I have in mind right that plus the ongoing chunker effort and rclone cleanup [--force] as a way to garbage-collect dead temporary chunks and take into account put/copy/cleanup running in parallel (and rclone about to see their size, btw). I just don't know whether cobra cli library used in rclone allows to use same flag name in different subcommands for internally different purposes. If it does, I'd prefer one name (rclone sync --force, rclone cleanup --force etc is perfectly ok since for user it's "force whatever this command does"). If it can't, so let it be --force-transfer.

[ncw]

I made a prototype of this here

I decided to give the flag a long name --no-check-dest as it is quite a difficult flag to use properly.

https://beta.rclone.org/branch/v1.49.5-236-gc7a2db45-fix-3616-force-beta/ (uploaded in 15-30 mins)

Here is the user docs.

Let me know what you think

--no-check-dest

The --no-check-dest can be used with move or copy and it causes
rclone not to check the destination at all when copying files.

This means that:

• the destination is not listed minimising the API calls
• files are always transferred
• this can cause duplicates on remotes which allow it (eg Google Drive)
• --retries 1 is recommended otherwise you'll transfer everything again on a retry

This flag is useful to minimise the transactions if you know that none
of the files are on the destination.

This is a specialized flag which should be ignored by most users!

[ncw]

@manishgupta24 - did you get a chance to try the beta above with --no-check-dest?

[manishgupta24]

@ncw. I will check it today and will post the results. Thanks

[ncw]

I updated the beta to include the latest changes

https://beta.rclone.org/branch/v1.49.5-245-gbb561583-fix-3616-force-beta/ (uploaded in 15-30 mins)

I note that this includes af05e29 which fixes --files-from which I think you were having a problem with so that without the --no-check-dest flag might be enough to solve your problem.

[ciaw]

Hi, +1 for this feature.

Thanks!

Can't wait to see it in one of the main release versions.

[ncw]

@ciaw - I'm awaiting feedback from user testing - fancy giving this a go and reporting back? I rebased the patch on master.

https://beta.rclone.org/branch/v1.50.2-085-g96d0e9a5-fix-3616-force-beta/ (uploaded in 15-30 mins)

[wogam]

@ciaw - I'm awaiting feedback from user testing - fancy giving this a go and reporting back? I rebased the patch on master.

https://beta.rclone.org/branch/v1.50.2-085-g96d0e9a5-fix-3616-force-beta/ (uploaded in 15-30 mins)

I've used this build to upload around 2000 files and haven't encountered any errors. Smaller files upload dramatically faster in congested directories.

[ncw]

Thanks for testing.

I've merged this to master now which means it will be in the latest beta in 15-30 mins and released in v1.51