Transfer invalid / hash mismatch error after oauth2client refresh on download

[thobrla]

When downloading a file, bytes from an oauth2client refresh request are inappropriately considered as part of the request.

This problem occurs with downloads (via cp or rsync) that run over the duration of an access token's expiration (typically 1 hour as of this issue's posting). It can also occur for sequential commands that use a cached access token stored in gsutil_state_dir (~/.gsutil/credstore by default). When the access token expires, gcs_json_media erroneously considers the results of the POST as part of the downloaded bytes.

This will result in one of two errors:

1. For small files, the oauth2client bytes will exceed the size of the file, resulting in: Service Exception: Transfer invalid cannot have start index greater than total size
2. For larger files, the MD5 won't match (since the data incorrectly md5 signature computed for local file (...) doesn't match cloud-supplied digest

This bug was likely introduced with this commit in gsutil 4.15. It occurs for user or service account credentials, but not for GCE credentials which use a different refresh path.

There is no sure-fire workaround for downloads that run over an hour. For sequential commands running less than an hour, removing ~/.gsutil/credstore between commands should prevent this issue.

[thobrla]

Until 4.16 is released, you can use a prerelease version:

Standalone installs: gsutil update gs://prerelease/gsutil_4.16pre_oauth2.tar.gz

Cloud SDK installs:

gcloud components repositories add https://storage.googleapis.com/prerelease/cloudsdk-prerelease-components/gsutil_4.16pre_gcoauth2/components-2.json
gcloud components update

[linar-jether]

After updating to the prerelease version, gsutil shows this error instead:

Traceback (most recent call last):
  File "/usr/local/bin/../share/google/google-cloud-sdk/bin/bootstrapping/gsutil.py", line 70, in <module>
    bootstrapping.CheckUpdates()
TypeError: CheckUpdates() takes exactly 1 argument (0 given)

[thobrla]

4.16 has been released, so you should use gcloud components repositories remove to remove the prerelease, and just run gcloud components update to get 4.16.

[linar-jether]

Had to remove gsutil and install again to get 4.16, but still getting that error

ServiceException: Transfer invalid (possible encoding error: Cannot have start index greater than total size)

[linar-jether]

These are the versions installed after updating to the latest
│ Name │ Version │ Size │
│ Cloud Storage Command Line Tool │ 4.16 │ 2.6 MiB │
│ Cloud Storage Command Line Tool (Platform Specific) │ 4.15 │ < 1 MiB │

[thobrla]

I'm curious about that platform-specific version. If you run gsutil version -l, does it show 4.16?

[linar-jether]

Yes this is after removing the alpha repository and updating (apparently
was already updated before)

gsutil version shows:

gsutil version: 4.16
checksum: 784a06805a228e6aa7d244e4477ccdca (OK)
boto version: 2.38.0
python version: 2.7.11 |Anaconda 2.4.1 (64-bit)| (default, Dec 6 2015,
18:08:32
) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
OS: Linux 3.16.0-4-amd64
multiprocessing available: True
using cloud sdk: True
config path: /etc/boto.cfg
gsutil path: /usr/local/share/google/google-cloud-sdk/platform/gsutil/gsutil
compiled crcmod: False
installed via package manager: False
editable install: False
On Jan 14, 2016 19:34, "thobrla" notifications@github.com wrote:

I'm curious about that platform-specific version. If you run gsutil
version -l, does it show 4.16?

—
Reply to this email directly or view it on GitHub
#310 (comment)
.

[thobrla]

Can you provide a gsutil -D ... log of the failure to gs-team@google.com? It's possible your case is a different bug with the same symptom, or a corner case was missed in the original fix.