BigQuery: Allow loading data from a list of URLs

[jeremywadsack]

In the BigQuery API documentation, a load job can be given a list or source URIs. This PR updates the client to allow you to send an array of Storage objects, or URLs (and String or URI), when loading data into a table or dataset.

The initial commit adds some table-reference acceptance tests that were missing, but make it complete with the table acceptance tests.

I noticed that there are very few load/load_job acceptance tests for the dataset. Would you like me to also fill that in to cover all the various cases we have for table and table-reference?

[blowmage]

Can you mix files and URLs?

[jeremywadsack]

@blowmage Local files? Not as far as I could tell. To upload local files you send a media-upload request to the /upload URI. This can be a multipart upload (which I believe is what the library does), but the documentation says that you cannot upload multiple files:

Wildcards and comma separated lists are not supported when you load files from a local data source. Files must be loaded individually.

Additionally:

The body of the request is formatted as a multipart/related content type [RFC2387] and contains exactly two parts. The parts are identified by a boundary string, and the final boundary string is followed by two hyphens.

Each part of the multipart request needs an additional Content-Type header:

Metadata part: Must come first, and Content-Type must match one of the accepted metadata formats.
Media part: Must come second, and Content-Type must match one the method's accepted media MIME types.

That said, I haven't tried to see what the API does if you use a media-upload request and set configuration.load.sourceUris[].

---

On the CI failure, I could use some guidance. I don't know how to address a doc test failure, "This code example is not yet mocked."

[blowmage]

@jeremywadsack The doctest failure is because you added a new code example labeled "Pass a list of google-cloud-storage files:", but didn't configure the documentation mocks for it in support/doctest_helper.rb. You can probably copy an existing configuration for Google::Cloud::Bigquery::Table#load_job in that file. Be sure to run bundle exec rake ci in the directory to see what other failures you may have before updating the PR.

[jeremywadsack]

@blowmage ba48571 adds doctest mocks (thanks for the pointer). That also reminded me I should document the example for the other three methods.

I ran rake ci locally and it passed before and after these changes, so that commit is a bit of a shot-in-the-dark.

[jeremywadsack]

@blowmage @quartzmo Let me know if there's anything else you want me to update. If this is good, I can squash/rebase.

[quartzmo]

I pulled this branch and ran rake ci:a (includes acceptance tests), passes for me locally.

[quartzmo]

LGTM!

[jeremywadsack]

Sqashed and rebased off master. I left 58c5960 as a separate commit because it was covering specs that were missing, unrelated to adding this support.

[quartzmo]

I'll merge when green (just a formality...)

[blowmage]

Thank you for the rebase/squash!

[quartzmo]

Thank you @jeremywadsack, really appreciate your contributions!

[jeremywadsack]

I really appreciate both of your responsiveness to issues and pull requests. I realize this is a really huge repository covering most of the GCP products. It's nice to have things moved through so quickly to releases.