Skip to content

BigQuery Job Updaters and EncryptionConfiguration#2022

Merged
quartzmo merged 21 commits into
masterfrom
bigquery
Mar 26, 2018
Merged

BigQuery Job Updaters and EncryptionConfiguration#2022
quartzmo merged 21 commits into
masterfrom
bigquery

Conversation

@quartzmo
Copy link
Copy Markdown
Member

@quartzmo quartzmo commented Mar 26, 2018 •
edited
Loading

This PR merges into master the bigquery topic branch that collected the changes from #1973, #2003, #2014, #2015 and #2016.

tswast added 21 commits March 26, 2018 11:41
@tswast @quartzmo
BigQuery: add EncryptionConfiguration class
4010dab 
This adds partial support for customer-managed encryption keys.
@tswast @quartzmo
encryption_configuration: fix line lengths
afd820b 
Also adds
- acceptence test for creating and modifying a table.
- additional unit tests for EncryptionConfiguration methods.
@tswast @quartzmo
Move encryption.rb to encryption_configuration
82986f1 
Use block table updater to set encryption_configuration in create_table.
@tswast @quartzmo
Update encryption_configuration in docstring to use updater block for…
4abd35d 
… create_table.
@tswast @quartzmo
Remove leftover changes from LoadJob
54214a0
@tswast @quartzmo
BigQuery: Refactor load_job to use LoadJob::Updater block
9475203 
This adds a block to load_job so that new properties can be added to
LoadJob without needing to add arguments to the load_job function.
@tswast @quartzmo
BigQuery yield LoadJob::Updater for dataset.load and table.load
c27b21a 
This will allow additional properties to be added to LoadJob without
needing to add additional arguments to the `dataset.load` and
`table.load` methods.

Also added a `ensure_job_succeeded! job` method to get the line counts
down enough to make rubocop happy with the change.
@tswast @quartzmo
BigQuery: Remove unused service.query_config method
0d7b57f
@tswast @quartzmo
[WorkInProgress] BigQuery: QueryJob::Updater class for additional que…
e4b20a9 
…ry job options
@tswast @quartzmo
Move create_disposition and write_disposition to Convert module.
c0bc83d
@tswast @quartzmo
Add QueryJob::Updater block to query methods.
6df6210
@tswast @quartzmo
Add remaining properties to QueryJob::Updater.
0f4e365
@tswast @quartzmo
Mark private methods @Private.
13668ca
@tswast @quartzmo
Use options for common args, block for uncommon args.
5aff470
@tswast @quartzmo
Add ExtractJob::Updater for setting options.
1b80f20 
Also move some shared conversion code to the Convert class.
@tswast @quartzmo
Add missing ## to doc comments
7d9da5d
@tswast @quartzmo
Pass in block to extract_job from extract.
c26cfe6
@tswast @quartzmo
BigQuery: Add CopyJob::Updater class.
ee950b2
@tswast @quartzmo
Add missing ## to doc comments
45f45fe
@tswast @quartzmo
Pass in block to copy_job from copy
6143fbd
@tswast @quartzmo
BigQuery: Add encryption properties to jobs classes. (#2016)
89bba16 
* Add a helper to create an EncryptionConfiguration in project.
* Rename encryption_configuration property to more concise encryption.
* Add acceptance tests for encryption
* Add imports to job files that depend on encryption.
@quartzmo quartzmo added the api: bigquery Issues related to the BigQuery API. label Mar 26, 2018
@quartzmo quartzmo self-assigned this Mar 26, 2018
@quartzmo quartzmo requested review from blowmage and tswast March 26, 2018 21:36
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Mar 26, 2018
@quartzmo
Copy link
Copy Markdown
Member Author

quartzmo commented Mar 26, 2018 •
edited
Loading

@tswast @jeremywadsack There were a number of conflicts between #1973 and #2000 that I needed to manually resolve while rebasing this branch on master. All tests are passing (including acceptance tests), but PTAL just to be sure the Load Job code still looks good.

tswast
tswast approved these changes Mar 26, 2018
View reviewed changes
Copy link
Copy Markdown
Contributor

@tswast tswast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Load job changes look good to me.

Comment thread google-cloud-bigquery/lib/google/cloud/bigquery/table.rb
yield updater if block_given?

job_gapi = updater.to_gapi
if local_file? files

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

blowmage
blowmage approved these changes Mar 26, 2018
View reviewed changes
Copy link
Copy Markdown
Contributor

@blowmage blowmage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for taking care of the conflicts, @quartzmo!

@quartzmo quartzmo merged commit 544137d into master Mar 26, 2018
@tswast tswast deleted the bigquery branch March 26, 2018 22:20
@jeremywadsack
Copy link
Copy Markdown
Contributor

jeremywadsack commented Mar 27, 2018

Looks like this preserves the changes in 2000.

quartzmo added a commit to quartzmo/google-cloud-ruby that referenced this pull request Mar 28, 2018
@quartzmo
Add tests for BigQuery Job Updaters
b6f9be1 
Add both unit and acceptance tests to improve coverage.

[refs googleapis#2022]
@quartzmo quartzmo mentioned this pull request Mar 28, 2018
Merged
quartzmo added a commit that referenced this pull request Mar 28, 2018
@quartzmo
Add tests for BigQuery Job Updaters
43a69fb 
Add both unit and acceptance tests to improve coverage.

[refs #2022]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tswast tswast tswast approved these changes

+2 more reviewers

@blowmage blowmage blowmage approved these changes

@jeremywadsack jeremywadsack jeremywadsack left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

@quartzmo quartzmo

Labels

api: bigquery Issues related to the BigQuery API. cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@quartzmo @jeremywadsack @blowmage @tswast @googlebot