Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: raise error for unknown properties in job config #446

Merged
merged 6 commits into from Jan 13, 2021

Conversation

cguardia
Copy link
Contributor

@cguardia cguardia commented Dec 22, 2020

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Simple feature to raise warning if unknown property is set on job_config instances.

Fixes #303 馃

@cguardia cguardia requested review from a team and tswast and removed request for a team December 22, 2020 09:51
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Dec 22, 2020
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Dec 22, 2020
def __setattr__(self, name, value):
"""Override to be able to warn if an uknown property is being set"""
if not name.startswith("_") and name not in type(self).__dict__:
warnings.warn("Property {} is unknown for {}.".format(name, type(self)))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, I think we can go a step further and throw AttributeError, just like object() instances do.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. Done.

@cguardia cguardia changed the title feature: warn about unknown properties in job config feature: raise error for unknown properties in job config Jan 8, 2021
Copy link
Contributor

@tswast tswast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@tswast
Copy link
Contributor

tswast commented Jan 8, 2021

    def __setattr__(self, name, value):
        """Override to be able to raise error if an unknown property is being set"""
        if not name.startswith("_") and name not in type(self).__dict__:
>           raise AttributeError("Property {} is unknown for {}.".format(name, type(self)))
E           AttributeError: Property labels is unknown for <class 'google.cloud.bigquery.job.query.QueryJobConfig'>.

@tswast
Copy link
Contributor

tswast commented Jan 8, 2021

It appears you've uncovered a bug in the client. I don't see a setter for labels!

@tswast
Copy link
Contributor

tswast commented Jan 8, 2021

Nevermind, there is a setter:

Perhaps this logic needs to be adjusted in order to account for properties defined in the superclass?

@@ -659,6 +659,12 @@ def __init__(self, job_type, **kwargs):
for prop, val in kwargs.items():
setattr(self, prop, val)

def __setattr__(self, name, value):
"""Override to be able to raise error if an unknown property is being set"""
if not name.startswith("_") and name not in type(self).__dict__:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try using hasattr to see if that accounts for superclass properties.

Suggested change
if not name.startswith("_") and name not in type(self).__dict__:
if not name.startswith("_") and not hasattr(self, name):

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, but we have to use type(self), because if not the properties are looked up on the instance.

@cguardia cguardia requested a review from tswast January 12, 2021 19:29
@tswast
Copy link
Contributor

tswast commented Jan 13, 2021

One of our system tests actually had a mistake, which this change caught. There is no "destination" property on LoadJobConfig https://googleapis.dev/python/bigquery/latest/generated/google.cloud.bigquery.job.LoadJobConfig.html

@plamut
Copy link
Contributor

plamut commented Jan 13, 2021

Re: labels error - as discussed offline, it's actually a bug in the base _AsyncJob.

@tswast tswast merged commit 1526e39 into googleapis:master Jan 13, 2021
evansd added a commit to ebmdatalab/openprescribing that referenced this pull request Dec 2, 2022
As far as I can tell, this has never been a valid config value here. But
prior to the merging of [this PR][1] in v2.7.0, invalid attributes were
silently ignored.

[1]: googleapis/python-bigquery#446
evansd added a commit to ebmdatalab/openprescribing that referenced this pull request Dec 2, 2022
As far as I can tell, this has never been a valid config value here. But
prior to the merging of [this PR][1] in v2.7.0, invalid attributes were
silently ignored.

[1]: googleapis/python-bigquery#446
evansd added a commit to ebmdatalab/openprescribing that referenced this pull request Dec 2, 2022
As far as I can tell, this has never been a valid config value here. But
prior to the merging of [this PR][1] in v2.7.0, invalid attributes were
silently ignored.

[1]: googleapis/python-bigquery#446
evansd added a commit to ebmdatalab/openprescribing that referenced this pull request Dec 2, 2022
As far as I can tell, this has never been a valid config value here. But
prior to the merging of [this PR][1] in v2.7.0, invalid attributes were
silently ignored.

[1]: googleapis/python-bigquery#446
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Resource objects such as *JobConfig do not error or warn when setting unknown properties
3 participants