Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[query] Fix bad error message from mt.make_table() with missing keys #8275

Merged
merged 3 commits into from
Mar 10, 2020

Conversation

tpoterba
Copy link
Contributor

@tpoterba tpoterba commented Mar 9, 2020

Fixes #8222

@@ -4050,12 +4050,16 @@ def make_table(self, separator='.') -> Table:
if not (len(self.col_key) == 1 and self.col_key[0].dtype == hl.tstr):
raise ValueError("column key must be a single field of type str")

col_key_field = list(self.col_key)[0]
col_keys = [k[col_key_field] for k in self.col_key.collect()]
col_keys = self.col_key[0].collect()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

simpler and results in less data collected.


duplicates = [k for k, count in Counter(col_keys).items() if count > 1]
if duplicates:
raise ValueError(f"column keys must be unique, found duplicates: {', '.join(duplicates)}")
for c in col_keys:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could avoid the second scan by reusing the counter:

counts = Counter(col_keys)
duplicates = [k for k, count in counts.items() if count > 1]
if duplicates:
  ...
if counts[None] > 0:
  ...

But if there is more than one missing key, won't this throw the duplicate key error?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good call, fixed.

@danking danking merged commit cebecfb into hail-is:master Mar 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

MT.make_table throws a bad error if a key is NA
3 participants