Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add AvroOptions to configure AVRO external data #994

Merged
merged 6 commits into from Sep 30, 2021
Merged

Conversation

tswast
Copy link
Contributor

@tswast tswast commented Sep 29, 2021

Also:

  • Unify ExternalConfig class to use _properties for everything. This does
    result in more code, but it should make maintenance easier as it aligns with
    our other mutable resource classes.
  • Adds bigtable_options, csv_options, and google_sheets_options
    properties. This aligns with parquet_options.

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

In response to internal request. 馃

Also:

* Unify `ExternalConfig` class to use `_properties` for everything. This does
  result in more code, but it should make maintenance easier as it aligns with
  our other mutable resource classes.
* Adds `bigtable_options`, `csv_options`, and `google_sheets_options`
  properties. This aligns with `parquet_options`.
@tswast tswast requested a review from a team September 29, 2021 18:50
@tswast tswast requested a review from a team as a code owner September 29, 2021 18:50
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Sep 29, 2021
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Sep 29, 2021
Copy link
Contributor

@plamut plamut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing much to add, except for the type hint duplication nit.

A few lines are missing from the coverage, please just address that.

google/cloud/bigquery/external_config.py Outdated Show resolved Hide resolved
@tswast tswast requested a review from plamut September 30, 2021 16:00
plamut
plamut approved these changes Sep 30, 2021
Copy link
Contributor

@plamut plamut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. You can address a nit if you want.

"""Additional properties to set if ``sourceFormat`` is set to
GOOGLE_SHEETS.
Copy link
Contributor

@plamut plamut Sep 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(nit) Does this fit into a single line? (IIRC the summary in the docs only grabs the first line)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not within 79 characters, which is what I've got VS Code configured to wrap at.

Looks like Sphinx did the right thing. Probably because there's no blank line.

Screen Shot 2021-09-30 at 11 52 47 AM

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that surprised me, but even better then!

@tswast tswast merged commit 1a9431d into main Sep 30, 2021
13 checks passed
@tswast tswast deleted the avro-options branch September 30, 2021 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants