Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Content-Type for Google Cloud Storage API calls #361

Merged
merged 2 commits into from
Aug 5, 2016

Conversation

grosskur
Copy link
Contributor

@grosskur grosskur commented Aug 3, 2016

The Google Cloud Storage team rolled out a change today to their service that checks the Content-Type header more strictly against the GS object metadata. Previously, the Buildkite agent would use the Google API library in the normal way, which would allow GS to sniff the content type and fill in the metadata header. But since we also set the Content-Type header when uploading, this unfortunately results in errors like:

Failed to PUT file "foo" (googleapi: Error 400: Content-Type specified in the upload (text/plain; charset=utf-8) does not match Content-Type specified in metadata (application/json). If it was a simple upload (uploadType=media), the Content-Type was specified as a bare header. If it was a multipart upload (uploadType=multipart), then the Content-Type was specified in the second part of the multipart. If it was a resumable upload (uploadType=resumable), then the Content-Type was specified with the X-Upload-Content-Type header with the start of the resumable session. For more information, see https://cloud.google.com/storage/docs/json_api/v1/how-tos/upload., invalid) (Attempt 1/10 Retrying in 1s)

The Google Cloud Storage team recommends fixing this by updating to the latest Google API client library and disabling Content-Type auto-sniffing with:

Media(file, googleapi.ContentType(""))

See googleapis/google-api-go-client#134 for background.

* Add github.com/golang/protobuf
* Bump golang.org/x/{net,oauth2}
* Add google.golang.org/grpc
* Bump google.golang.org/{api,cloud}
The Google Cloud Storage team now recommends using

    Media(file, googleapi.ContentType(""))

to bypass auto-sniffing. See googleapis/google-api-go-client#134 for
background.
@keithpitt
Copy link
Member

Nice! Thanks for fixing this for us! I'll merge it in now.

Are you using the 3.0 beta series to 2.1.x?

@keithpitt keithpitt merged commit 2e9bab6 into buildkite:master Aug 5, 2016
@grosskur
Copy link
Contributor Author

grosskur commented Aug 7, 2016

Thanks! We're now using the latest master (3.0 beta series).

@keithpitt
Copy link
Member

@grosskur cool cool, I just pushed out a new beta with your changes: https://github.com/buildkite/agent/releases/tag/v3.0-beta.7

Let me know if it's all good now!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants