Pangeo Hub access to requester pays data

[choldgraf]

Description

The Pangeo Hub does not currently have "requester pays" access to data stored in the cloud. This is a feature that allows the requesting hub (the 2i2c Pangeo Hub in this case) to pay the costs of accessing cloud data. We should activate this!

Value / benefit

Without this feature Pangeo Hub users won't be able to access many of the datasets that are commonly used in that workflow.

Implementation details

The documentation about "Requester Pays Access" is here: https://cloud.google.com/storage/docs/requester-pays

that includes instructions for what we'll need to do.

Completion criteria

The way that @rabernat originally discovered that this access does not exist, and probably the completion criteria to consider this resolved, is that the following code should work:

Run the first few cells of this notebook, specifically

from intake import open_catalog
cat = open_catalog("https://raw.githubusercontent.com/pangeo-data/pangeo-datastore/master/intake-catalogs/ocean.yaml")
ds  = cat["sea_surface_height"].to_dask()

If access is not there, you'll get this result:

OSError: Forbidden: https://storage.googleapis.com/download/storage/v1/b/pangeo-cmems-duacs/o/.zmetadata?alt=media
Caller does not have serviceusage.services.use access to the Google Cloud project.

Tasks to complete

No response

Updates

No response

[sgibson91]

Picking this up today

[sgibson91]

I moved this back into the "In Progress" column for now since #662 documents the manual steps but this may already be automated as part of the helm chart. I need to verify that that config works and document how to use that instead.