Add support for JSON API headers and query string parameters

[mlhamel]

Description

The XML-Api and the Json-API are supporting a large set of parameters as described at https://cloud.google.com/storage/docs/xml-api/reference-headers.

Into those parameters you can find very useful features like x-goog-metageneration which let you control how to deal with versions and also blocking rewriting of blobs in a bucket.

These API used to be available in the AppEngine library for google cloud storage as you can see at https://cloud.google.com/appengine/docs/standard/python/googlecloudstorageclient/functions#open

Proposition

Adds support for extended options on different operations like upload_from_string and delete, something like:

def upload_from_string(self, data, content_type='text/plain', client=None,
                           predefined_acl=None, options=None):

Which would like you pick and sets of options from https://cloud.google.com/storage/docs/xml-api/reference-headers

[tseaver]

Docs for the JSON API headers / query string parameters. Where they differ, we would be following the latter, because we never directly invoke the XML API (Blob.public_url and Blob.generate_signed_url create URLs against the XML API endpoint, but don't invoke them).

[tseaver]

Of the "standard" HTTP headers, I can see that there might be a usecase for supplying:

• Cache-Control (override default 3600 value; only supported for fetching blobs accessible to all anonymous users).
• Range (download partial content)
• If-Match / If-None-Match (check an ETag)

AFAICT, the JSON API only supports one "extension" header which is not covered / superseded by existing google-cloud-storage usage:

• X-HTTP-Method-Override (smuggle PATCH requests through obtuse firewalls)

[tseaver]

Of the the query string parameters supported by the JSON API, we might want to support:

• prettyPrint (remove whitespace from JSON responses)
• quotaUser (enforce per-user quotas from a server-side application even in cases when the user's IP address is unknown)
• userIp (enforce per-user quotas when calling the API from a server-side application.)

The others are ether superseded by the library, or are tied to deprecated authentication schemes / JSON-P.

[tseaver]

With the release of google-cloud-core 1.0.0, the Connection class now exposes an extra_headers property, which can be used to pass whatever additional headers are needed (but not on a per-request basis:

client._connection.extra_headers['X-Goog-Foo'] = 'Bar'

[mlhamel]

Thanks for the updates I'm guessing it means we can now close this issue.

[tseaver]

@mlhamel I don't think the new Connection.extra_headers support is exactly what this issue wanted: it is kind of an "always on" configuration, rather than one which takes effect per-request.

[daniellehanks]

Any update on this? I stumbled upon this needing to specify an If-None-Match header. Basically my use case is my API is a thin layer over GCS (for storing profile images), and I only want to actually read the image data and serve it to the client if the actual content is different from their cache. They will specify an If-None-Match header to my API (as they won't know anything about GCS or generation numbers, kind of the point), so I just need to essentially pass this along to GCS. I don't want to either do the extra round tripping of reading the current file to check the etag and then use the generation number from that in a subsequent request, or not use the client library.