Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add partition support to BigQuery #2058

Merged
merged 4 commits into from
Aug 5, 2016

Conversation

dwmclary
Copy link
Contributor

@dwmclary dwmclary commented Aug 5, 2016

This adds the ability to create a partitioned table. It would be nice to expose a table.list_partitions() method, but currently there is not an API-based way of returning partitions. So, for now, it simply allows users to interact with the current partitioning API.

@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Aug 5, 2016
@dwmclary dwmclary changed the title Add partition support Add partition support to BigQuery Aug 5, 2016
@tseaver tseaver added the api: bigquery Issues related to the BigQuery API. label Aug 5, 2016
partitioned = None
if "timePartitioning" in self._properties:
partitioned = self._properties.get('timePartitioning').get('type')
return partitioned

This comment was marked as spam.

@tseaver
Copy link
Contributor

tseaver commented Aug 5, 2016

The docs for creating partitioned tables point to a query for listing partitions, Could we use that to implement `table.list_partitions'?

with self.assertRaises(ValueError):
table.partitioning_type = "HASH"
with self.assertRaises(ValueError):
table.partition_expiration = "NEVER"

This comment was marked as spam.

@tseaver
Copy link
Contributor

tseaver commented Aug 5, 2016

Thank you for the patch!

@tseaver tseaver self-assigned this Aug 5, 2016
@dwmclary
Copy link
Contributor Author

dwmclary commented Aug 5, 2016

I've got a method I didn't include for list_partitions that does generate and run that query. What I'm not sure about is the best way to fake a partitioned table for testing said method.

@tseaver
Copy link
Contributor

tseaver commented Aug 5, 2016

What I'm not sure about is the best way to fake a partitioned table for testing said method.

Assuming the method looks something like:

    def list_partitions(self):
        query = self._client.query(
            'SELECT partition_id from [%s.%s$__PARTITIONS_SUMMARY__]' %
                (self.dataset_name, self.name))
        query.run()
        return [row[0] for row in query.rows]

I think I would just mock up the query method on the _Client mock to return an instance of a mocked _Query class: it would then have a run method which would populate its rows attribute.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the BigQuery API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants