Skip to content

Error Creating Table with JSON Column #1282

@waltaskew

Description

@waltaskew

Environment details

  • OS type and version: Linux, Debian
  • Python version: Python 3.13.7
  • pip version: pip 25.3
  • sqlalchemy-bigquery version: 1.16.0

Steps to reproduce

  1. Define a table with a JSON column
  2. Try to create it with alembic, create_all or such

Code example

Table("tbl", MetaData(), Column("stuff", JSON()))

Stack trace

Traceback (most recent call last):
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/visitors.py", line 134, in _compiler_dispatch
    meth = getter(visitor)
AttributeError: 'BigQueryTypeCompiler' object has no attribute 'visit_JSON'. Did you mean: 'visit_BLOB'?

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/compiler.py", line 6769, in visit_create_table
    processed = self.process(
        create_column, first_pk=column.primary_key and not first_pk
    )
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/compiler.py", line 932, in process
    return obj._compiler_dispatch(self, **kwargs)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/visitors.py", line 138, in _compiler_dispatch
    return meth(self, **kw)  # type: ignore  # noqa: E501
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/compiler.py", line 6800, in visit_create_column
    text = self.get_column_specification(column, first_pk=first_pk)
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/sqlalchemy_bigquery/base.py", line 665, in get_column_specification
    colspec = super(BigQueryDDLCompiler, self).get_column_specification(
        column, **kwargs
    )
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/compiler.py", line 7026, in get_column_specification
    + self.dialect.type_compiler_instance.process(
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        column.type, type_expression=column
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/compiler.py", line 977, in process
    return type_._compiler_dispatch(self, **kw)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/visitors.py", line 136, in _compiler_dispatch
    return visitor.visit_unsupported_compilation(self, err, **kw)  # type: ignore  # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/compiler.py", line 982, in visit_unsupported_compilation
    raise exc.UnsupportedCompilationError(self, element) from err
sqlalchemy.exc.UnsupportedCompilationError: Compiler <sqlalchemy_bigquery.base.BigQueryTypeCompiler object at 0x7f9e0285ae90> can't render element of type JSON (Background on this error at: https://sqlalche.me/e/20/l7de)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/_pytest/runner.py", line 353, in from_call
    result: TResult | None = func()
                             ~~~~^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/_pytest/runner.py", line 245, in <lambda>
    lambda: runtest_hook(item=item, **kwds),
            ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/_pytest/logging.py", line 850, in pytest_runtest_call
    yield
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/_pytest/capture.py", line 900, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 139, in _multicall
    teardown.throw(exception)
    ~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/_pytest/skipping.py", line 268, in pytest_runtest_call
    return (yield)
            ^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/_pytest/runner.py", line 179, in pytest_runtest_call
    item.runtest()
    ~~~~~~~~~~~~^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/_pytest/python.py", line 1720, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/pluggy/_hooks.py", line 512, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 167, in _multicall
    raise exception
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/pluggy/_callers.py", line 121, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/_pytest/python.py", line 166, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/tests/unit/test_table_options.py", line 62, in test_table_with_json_columns
    setup_table(
    ~~~~~~~~~~~^
        faux_conn,
        ^^^^^^^^^^
        "some_table",
        ^^^^^^^^^^^^^
        sqlalchemy.Column("some_stuff", sqlalchemy.JSON),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/tests/unit/conftest.py", line 100, in setup_table
    metadata.create_all(connection.engine)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/schema.py", line 5928, in create_all
    bind._run_ddl_visitor(
    ~~~~~~~~~~~~~~~~~~~~~^
        ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 3252, in _run_ddl_visitor
    conn._run_ddl_visitor(visitorcallable, element, **kwargs)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 2459, in _run_ddl_visitor
    ).traverse_single(element)
      ~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/visitors.py", line 661, in traverse_single
    return meth(obj, **kw)
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/ddl.py", line 984, in visit_metadata
    self.traverse_single(
    ~~~~~~~~~~~~~~~~~~~~^
        table,
        ^^^^^^
    ...<2 lines>...
        _is_metadata_operation=True,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/visitors.py", line 661, in traverse_single
    return meth(obj, **kw)
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/ddl.py", line 1022, in visit_table
    )._invoke_with(self.connection)
      ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/ddl.py", line 321, in _invoke_with
    return bind.execute(self)
           ~~~~~~~~~~~~^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
    return meth(
        self,
        distilled_parameters,
        execution_options or NO_OPTIONS,
    )
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/ddl.py", line 187, in _execute_on_connection
    return connection._execute_ddl(
           ~~~~~~~~~~~~~~~~~~~~~~~^
        self, distilled_params, execution_options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1527, in _execute_ddl
    compiled = ddl.compile(
        dialect=dialect, schema_translate_map=schema_translate_map
    )
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/elements.py", line 311, in compile
    return self._compiler(dialect, **kw)
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/ddl.py", line 76, in _compiler
    return dialect.ddl_compiler(dialect, self, **kw)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/compiler.py", line 886, in __init__
    self.string = self.process(self.statement, **compile_kwargs)
                  ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/compiler.py", line 932, in process
    return obj._compiler_dispatch(self, **kwargs)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/visitors.py", line 138, in _compiler_dispatch
    return meth(self, **kw)  # type: ignore  # noqa: E501
  File "/usr/local/google/home/waltaskew/src/python-bigquery-sqlalchemy/venv/lib/python3.13/site-packages/sqlalchemy/sql/compiler.py", line 6779, in visit_create_table
    raise exc.CompileError(
    ...<2 lines>...
    ) from ce
sqlalchemy.exc.CompileError: (in table 'some_table', column 'some_stuff'): Compiler <sqlalchemy_bigquery.base.BigQueryTypeCompiler object at 0x7f9e0285ae90> can't render element of type JSON

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the googleapis/python-bigquery-sqlalchemy API.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions