HttpError 400 on uploading 0 bytes files, 416 on downloading 0 bytes files

[lampvux]

It seems like the bug #241 and @Aran-Fey's problem happened again when i trying to download and upload 0 bytes file using google drive api.

    credentials = google.oauth2.credentials.Credentials(**token)
    services_account = googleapiclient.discovery.build(API_SERVICE_NAME, API_VERSION, credentials=credentials)
    fd= BytesIO()
    media = googleapiclient.http.MediaIoBaseUpload(fd=fd, chunksize=128, mimetype='text/plain', resumable=True)
    body = {'name' : 'foo.txt'}
    services_account.files().insert(body=body, media_body=media, fields='id').execute()

throws

    `googleapiclient.errors.HttpError: <HttpError 400 when requesting https://www.googleapis.com/upload/drive/v2/files?fields=id&alt=json&uploadType=resumable
returned "Bad Request">`

[busunkim96]

Hi @9kmmr, could you let us know which version of the API (V2 or V3) you are using?

[grant]

@busunkim96 The error states Drive V2. (It's generally recommended to use V3)

@9kmmr It looks like the API request is bad. Here are more samples for uploading files:
https://developers.google.com/drive/api/v2/manage-uploads

[lampvux]

@busunkim96 It is V2
@grant I tried with no-empty file and it worked

[SurferJeffAtGoogle]

@9kmmr Is using Drive V3 API possible in your environment?

[SurferJeffAtGoogle]

@9kmmr Please re-open this bug if it's still an issue for you.

[rlunaro]

same error, using python. V3 of the drives, python 3.8 and windows 64 bit

[gkowalski-google]

I'm still seeing this issue with google-api-python-client 1.7.10. Example code can be found here in the Forseti repo. A googleapiclient.errors.HttpError 416 exception is raised on the call to next_chunk() when downloading a 0 byte file. I see several closed issues related to this issue. What's the best way to fix this?

[rlunaro]

Well, in fact I didn't had to deal with that. For my concrete example, I've decided to avoid zero size files and that's all. El mar., ene. 14, 2020 a 23:27, Gregg Kowalski<notifications@github.com> escribió: I'm still seeing this issue with google-api-python-client 1.7.10. Example code can be found here in the Forseti repo. A googleapiclient.errors.HttpError 416 exception is raised on the call to next_chunk() when downloading a 0 byte file. I see several closed issues related to this issue. What's the best way to fix this? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[gkowalski-google]

Unfortunately there are cases when we are exporting data from GCP to GCS, and some times those files are 0 bytes in size so it's not something we can avoid.

[efiop]

We are running into this as well. Ignoring 416 does the job, but looks like a hack. We are using v2 though (google-api-python-client-1.11.0), as we can't migrate easily to v3. @SurferJeffAtGoogle Looks like other people are having this issue as well, should we reopen this issue? Or is it fixed in v3 and older ones won't be fixed?

[SurferJeffAtGoogle]

We confirmed that other clients fetch 0-bytes files without error, so this must be an issue with the python client.

[busunkim96]

The initial POST to begin the resumable upload succeeds. The PUT fails with invalid argument, and the content-range seems the most suspect at the moment content-range: bytes 0--1/0.

content_range = "bytes %d-%d/%s"  % (self.resumable_progress, chunk_end, size)

send: b'PUT /upload/drive/v2/files?fields=id&alt=json&uploadType=resumable&upload_id=ABg5....HTTP/1.1\r\nHost: www.googleapis.com\r\ncontent-range: bytes 0--1/0\r\ncontent-length: 0\r\nauthorization: Bearer ya...\r\nuser-agent: Python-httplib2/0.18.1 (gzip)\r\naccept-encoding: gzip, deflate\r\n\r\n'
sendIng a read()able
reply: 'HTTP/1.1 400 Bad Request\r\n'
header: Content-Type: text/plain; charset=utf-8
header: X-GUploader-UploadID: ABg5...
header: Content-Length: 37
header: Date: Tue, 20 Oct 2020 00:34:55 GMT
header: Server: UploadServer
Traceback (most recent call last):
  File "zero_byte_file.py", line 49, in <module>
    main()
  File "zero_byte_file.py", line 45, in main
    service.files().insert(body=body, media_body=media, fields="id").execute()
  File "/usr/local/google/home/busunkim/bugs/api-client-638/env/lib/python3.8/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/google/home/busunkim/bugs/api-client-638/env/lib/python3.8/site-packages/googleapiclient/http.py", line 871, in execute
    _, body = self.next_chunk(http=http, num_retries=num_retries)
  File "/usr/local/google/home/busunkim/bugs/api-client-638/env/lib/python3.8/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/google/home/busunkim/bugs/api-client-638/env/lib/python3.8/site-packages/googleapiclient/http.py", line 1061, in next_chunk
    return self._process_response(resp, content)
  File "/usr/local/google/home/busunkim/bugs/api-client-638/env/lib/python3.8/site-packages/googleapiclient/http.py", line 1092, in _process_response
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://www.googleapis.com/upload/drive/v2/files?fields=id&alt=json&uploadType=resumable returned "Bad Request">
(env) busunkim@busunkim:~/bugs/api-client-638$ 

[busunkim96]

The empty upload succeeds once the Content-Range header is removed from the PUT request. This is also what the Java client appears to do (https://github.com/googleapis/google-api-java-client/blob/326370e0ec4351aaea21da271692b2fb9585a28c/google-api-client/src/main/java/com/google/api/client/googleapis/media/MediaHttpDownloader.java#L242-L255)

[efiop]

Thank you so much @busunkim96 ! 🙏

[efiop]

@busunkim96 @SurferJeffAtGoogle Upload works flawlessly now, but downloading still fails with 416 as discussed in this ticket, so looks like it shouldn't be closed. Or am I missing something?

EDIT: here is the exc https://travis-ci.com/github/iterative/PyDrive2/jobs/403036538#L557

[busunkim96]

Ah, I missed the download part of this. Re-opening.

[efiop]

Thank you so much for a quick ⚡ ⚡ ⚡ fix @busunkim96 ! 🙏