Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add AvroOptions to configure AVRO external data #994

Merged
merged 6 commits into from Sep 30, 2021
Merged

feat: add AvroOptions to configure AVRO external data #994

merged 6 commits into from Sep 30, 2021

Conversation

@tswast
Copy link
Contributor

@tswast tswast commented Sep 29, 2021

Also:

  • Unify ExternalConfig class to use _properties for everything. This does
    result in more code, but it should make maintenance easier as it aligns with
    our other mutable resource classes.
  • Adds bigtable_options, csv_options, and google_sheets_options
    properties. This aligns with parquet_options.

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

In response to internal request. 馃

Also:

* Unify `ExternalConfig` class to use `_properties` for everything. This does
  result in more code, but it should make maintenance easier as it aligns with
  our other mutable resource classes.
* Adds `bigtable_options`, `csv_options`, and `google_sheets_options`
  properties. This aligns with `parquet_options`.
@tswast tswast requested review from as code owners Sep 29, 2021
@tswast tswast requested a review from stephaniewang526 (assigned from googleapis/api-bigquery) Sep 29, 2021
@google-cla google-cla bot added the cla: yes label Sep 29, 2021
Copy link
Contributor

@plamut plamut left a comment

Nothing much to add, except for the type hint duplication nit.

A few lines are missing from the coverage, please just address that.

Loading

google/cloud/bigquery/external_config.py Outdated Show resolved Hide resolved
Loading
@tswast tswast requested a review from plamut Sep 30, 2021
plamut
plamut approved these changes Sep 30, 2021
Copy link
Contributor

@plamut plamut left a comment

LGTM. You can address a nit if you want.

Loading

"""Additional properties to set if ``sourceFormat`` is set to
GOOGLE_SHEETS.
Copy link
Contributor

@plamut plamut Sep 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(nit) Does this fit into a single line? (IIRC the summary in the docs only grabs the first line)

Loading

Copy link
Contributor Author

@tswast tswast Sep 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not within 79 characters, which is what I've got VS Code configured to wrap at.

Looks like Sphinx did the right thing. Probably because there's no blank line.

Screen Shot 2021-09-30 at 11 52 47 AM

Loading

Copy link
Contributor

@plamut plamut Sep 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that surprised me, but even better then!

Loading

@tswast tswast merged commit 1a9431d into main Sep 30, 2021
13 checks passed
Loading
@tswast tswast deleted the avro-options branch Sep 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants