Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CREATE TABLE with PARTITION BY is unsupported #152

Open
oleksii-karpukhin-w3w opened this issue Feb 23, 2023 · 5 comments
Open

CREATE TABLE with PARTITION BY is unsupported #152

oleksii-karpukhin-w3w opened this issue Feb 23, 2023 · 5 comments

Comments

@oleksii-karpukhin-w3w
Copy link

Any attempt to create a table with a partition column[s] specified produces this error.
For example:
CREATE TABLE test_events_date (ID date) PARTITION BY ID

Result:

BigQuery error in query operation: Error processing job 'test:bqjob_r3d26e65599bb2335_000001867f166412_1': failed to analyze: INVALID_ARGUMENT: CREATE TABLE with PARTITION BY is
unsupported [at 1:41]

PS: used docker image ghcr.io/goccy/bigquery-emulator:0.2.13

@leonasdev
Copy link

It seems that any feature related to PARTITIONED TABLES has issues, such as the _PARTITIONTIME pseudo column.

@Shuanglu
Copy link

Encounter something similar googleapi: Error 400: failed to analyze: INVALID_ARGUMENT: Unrecognized name: _PARTITIONTIME [at 44:23], jobInternalError

@prismec
Copy link

prismec commented Dec 19, 2023

The same is true for me. Are there any updates on this issue ?

@prismec
Copy link

prismec commented Dec 19, 2023

At the moment, I can create a partitioned table using the client libraries. At least, I won't get an error (what already helps in tests and local emulation!). The emulator, however, accepts queries that would not work with BigQuery for the schema.

Here's an example. Given the following table definition

{
    "type": "TABLE",
    "schema": {
      "fields": [
        {
          "mode": "REQUIRED",
          "name": "value",
          "type": "DATE"
        }
      ]
    },
    "timePartitioning": {
      "field": "value",
      "type": "DAY",
      "expirationMs": 315569260000,
      "requirePartitionFilter": true
    }
  }

BigQuery would reject a query that does not contain the value field within its' WHERE clause, where the emulator accepts such a query.

I guess it is a lot of work to support all the partitioning features of BigQuery, but I could imagine that it's easier to check if the query is valid according to requirePartitionFilter fields of the schema. This would at least help to ensure that queries are well formed. Would this be a first step into the direction of better partitioning support ?

@prismec
Copy link

prismec commented Mar 5, 2024

Are there any updates on this topic ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants