Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upload speed to google drive #2368

Closed
NguyenKhong opened this issue Jun 30, 2018 · 8 comments
Closed

Upload speed to google drive #2368

NguyenKhong opened this issue Jun 30, 2018 · 8 comments

Comments

@NguyenKhong
Copy link

What is the problem you are having with rclone?

I do not know about golang, but I found the issue of rclone's upload speed, it's located at line 1236 in the transport.go file under the net/http library. I tried changing to pconn.bw = bufio.NewWriterSize(persistConnWriter{pconn}, 2097152) with a 2MBytes buffer instead of 4K. The speed varies considerably from a few hundred KBytes/s to MBytes/s

What is your rclone version (eg output from rclone -V)

rclone v1.41

  • os/arch: windows/amd64
  • go version: go1.10

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 7, 64 bit

Which cloud storage system are you using? (eg Google Drive)

Google Drive and my webdav

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Rclone copy and sync

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

@ncw
Copy link
Member

ncw commented Jul 1, 2018

Can you show me which rclone command line you are using to measure the speed?

And show a before and after please?

Did you try increasing --buffer-size and --drive-chunk-size ?

That piece of code is in the Go standard library so I can''t patch it in the rclone source. It might be that we can find a work-around in the rclone code.

If not though, then you'll need to make a simple example with a before and after comparison and make an issue on the go bugtracker.

@NguyenKhong
Copy link
Author

Sorry for not giving you example.
The following is a comparison picture

  • Google drive
    googledrive

  • My webdav use tcptrack to monitor
    mywebdav

@ncw
Copy link
Member

ncw commented Jul 1, 2018

Do you get better performance if you increase --buffer-size from the default 16M?

@NguyenKhong
Copy link
Author

NguyenKhong commented Jul 2, 2018

I tried changing the --buffer-size from 8M to 16M, 32M, 128M but it still does not increase performance.

@ncw
Copy link
Member

ncw commented Jul 2, 2018

Thanks for the tests and the screnshots.

The bit of code you changed is in the go standard library.

If you can make a stand alone go program to demonstrate the problem they you can report a go issue.

@hazcod
Copy link

hazcod commented Jul 13, 2018

Referencing: golang/go#22618

Fix: https://golang.org/cl/76410

@ncw
Copy link
Member

ncw commented Jul 13, 2018

Thanks for pointing that out @hazcod - that is exactly the fix @NguyenKhong was looking for.

@Animosity022
Copy link
Collaborator

This looks to be a stale issue. If you have a new question, please feel free to ask in the forums and we are happy to help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants