Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Resumable upload for big files with 2 Legged oauth. 401 error #28

Open
unlimit opened this Issue · 6 comments

3 participants

unlimit Bob Aman Steve Bazyl
unlimit

Hi!

I faced with upload problem for big files and 2 legged oauth.
I using small chunks (10MB) for uploading.
After 10 minutes of successful uploading, google responds with 401 error.

For small files resumable upload works fine.

Bob Aman
Admin

Check your access token and make sure it didn't expire mid upload?

unlimit

@sporkmonger, I using 2 Legged Oauth.
From my logs I see that every API PUT request using right 'oauth_timestamp'.
It seems the problem comes from google backend.

http://code.google.com/p/google-api-python-client/issues/detail?can=2&start=0&num=100&q=&colspec=ID%20Type%20Status%20Priority%20Milestone%20Owner%20Summary&groupby=&sort=&id=231

But in my case I get 401 error after 10 minutes.

Steve Bazyl
Admin

Would help to know which API you're uploading to -- storage? Drive?

@sporkmonger -- It shouldn't be a token expiry problem since it (supposedly) is only checked on the initial request to start the upload session. That said, the comments from Joe on the other thread imply there might be something to it.

@unlimit -- Also worth noting that chunking is discouraged. The preferred approach is to upload in one continuous chunk and retry/resume if it fails. Of course there are cases where chunking is necessary, so you still can. But try to do a continuous upload if possible.

unlimit

@sqrrrl, I using drive API, google-api-ruby-client version: 0.6.2

I tested upload in one continuous chunk, got the same 401 error.

I using chunk uploads in order to prevent RAM memory issues when uploading Big files.
What is the best way for uploading 10Gb file? In case of one continuous chunk, will it be loaded completely in memory before uploading?

Steve Bazyl
Admin

It doesn't need to read the entire file into memory. I haven't tested files that big, but a while back I tried ~ 1gb file uploaded to Drive and the process didn't grow past a few mb.

BTW, what API are you uploading to?

unlimit

client.execute!(
:api_method => service.files.insert,
:body_object => file,
:media => media,
:parameters => {'uploadType' => 'resumable', 'alt' => 'json'})

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.