Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Rclone copy --files-from tag takes some time before staring the migration #3248
I am trying to upload a folder using copy command and I am passing a list of files using
The wait time depends on the number of files I am trying to migrate with the
I tested it with a folder containing around 5k files and the initial wait time was around 45 minutes.
I looked into the code and this might be happening because Rclone creates a DirTree for the list of files provided with the tag.
My question is why does this take more time as compared to the Folder Walk using
Which backend are you using? Google drive is particularly slow with
Which version of rclone are you using?
With --files-from rclone finds all the files first before starting the transfers as you've noted.
In the latest beta I changed the algorithm so that it will now traverse directories to find the files from list, unless you use --no-traverse when it will use the 1.47 behaviour.
So give the latest beta a go and see if that fixes your problem.
My Source is a local Windows Directory and My destination is Google Cloud Storage bucket.
How is it different from not passing a list of files using --files-from tag? By default the rclone traverses the directory given as argument with copy command.
You still pass the list of files with