Skip to content

Bug: BigQuery adapter hangs against BigQuery emulator; CLUSTER BY rejected #473

@cofin

Description

@cofin

Adapter: sqlspec.adapters.bigquery
SQLSpec version: 0.47.0
Emulator: Google's official BigQuery emulator (e.g. via pytest-databases[bigquery])

Summary

The SQLSpec BigQuery adapter does not interoperate with the BigQuery emulator:

  1. Without a workaround, schema creation includes CLUSTER BY which the emulator does not accept.
  2. After removing the CLUSTER BY clause, the client path hangs indefinitely on subsequent operations (writes block past any reasonable timeout — observed at 180s+).

The driver already detects the emulator via detect_emulator() (sqlspec/adapters/bigquery/driver.py), but the events store DDL at sqlspec/adapters/bigquery/events/store.py:78 hard-codes CLUSTER BY channel, status, available_at regardless of that flag.

Reproduction

Using a BigQueryConfig pointed at the BigQuery emulator, attempt any DDL/DML cycle:

from sqlspec.adapters.bigquery import BigQueryConfig

config = BigQueryConfig(
    connection_config={
        "project": project,
        "dataset_id": dataset,
        "credentials": credentials,
        "client_options": client_options,
        "use_query_cache": False,
    }
)
# 1. CREATE TABLE ... CLUSTER BY (status) → emulator rejects
# 2. After dropping CLUSTER BY, subsequent INSERT/SELECT operations hang.

Expected Behavior

  • BigQuery adapter should detect emulator endpoints and avoid features the emulator does not support, OR provide a configuration flag to opt out.
  • Adapter should never block indefinitely waiting on a response from the emulator — surface a timeout error instead.

Suggested Fix Direction

  • Have the events store DDL consult the driver's _using_emulator flag (or accept a cluster_by_enabled switch on the config) and omit CLUSTER BY when the emulator is in use.
  • Add a connection-level timeout so emulator unresponsiveness surfaces as an error rather than a hang.

File References

  • sqlspec/adapters/bigquery/driver.py:142 (_using_emulator = detect_emulator(connection) — already detected)
  • sqlspec/adapters/bigquery/events/store.py:78 (return " CLUSTER BY channel, status, available_at" — emulator-unsafe)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions