Skip to content

BUG: to_sql with ADBC driver fails when schema exists without tables #59841

Open
@WillAyd

Description

@WillAyd

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

This is hard to create a reproducer for given sqlite does not have schemas and is not affected by the issue. However, with the proper database setup running code like:

with adbc_driver_postgresql.dbapi.connect() as conn:
    df.to_sql("table", schema="schema_with_no_tables")

Will produce the following traceback:

File ~/micromamba/envs/youtube-demo/lib/python3.12/site-packages/pandas/util/_decorators.py:333, in deprecate_nonkeyword_arguments.<locals>.decorate.<locals>.wrapper(*args, **kwargs)
    327 if len(args) > num_allow_args:
    328     warnings.warn(
    329         msg.format(arguments=_format_argument_list(allow_args)),
    330         FutureWarning,
    331         stacklevel=find_stack_level(),
    332     )
--> 333 return func(*args, **kwargs)

File ~/micromamba/envs/youtube-demo/lib/python3.12/site-packages/pandas/core/generic.py:3087, in NDFrame.to_sql(self, name, con, schema, if_exists, index, index_label, chunksize, dtype, method)
   2889 """
   2890 Write records stored in a DataFrame to a SQL database.
   2891 
   (...)
   3083 [(1,), (None,), (2,)]
   3084 """  # noqa: E501
   3085 from pandas.io import sql
-> 3087 return sql.to_sql(
   3088     self,
   3089     name,
   3090     con,
   3091     schema=schema,
   3092     if_exists=if_exists,
   3093     index=index,
   3094     index_label=index_label,
   3095     chunksize=chunksize,
   3096     dtype=dtype,
   3097     method=method,
   3098 )

File ~/micromamba/envs/youtube-demo/lib/python3.12/site-packages/pandas/io/sql.py:842, in to_sql(frame, name, con, schema, if_exists, index, index_label, chunksize, dtype, method, engine, **engine_kwargs)
    837     raise NotImplementedError(
    838         "'frame' argument should be either a Series or a DataFrame"
    839     )
    841 with pandasSQL_builder(con, schema=schema, need_transaction=True) as pandas_sql:
--> 842     return pandas_sql.to_sql(
    843         frame,
    844         name,
    845         if_exists=if_exists,
    846         index=index,
    847         index_label=index_label,
    848         schema=schema,
    849         chunksize=chunksize,
    850         dtype=dtype,
    851         method=method,
    852         engine=engine,
    853         **engine_kwargs,
    854     )

File ~/micromamba/envs/youtube-demo/lib/python3.12/site-packages/pandas/io/sql.py:2386, in ADBCDatabase.to_sql(self, frame, name, if_exists, index, index_label, schema, chunksize, dtype, method, engine, **engine_kwargs)
   2381 # pandas if_exists="append" will still create the
   2382 # table if it does not exist; ADBC is more explicit with append/create
   2383 # as applicable modes, so the semantics get blurred across
   2384 # the libraries
   2385 mode = "create"
-> 2386 if self.has_table(name, schema):
   2387     if if_exists == "fail":
   2388         raise ValueError(f"Table '{table_name}' already exists.")

File ~/micromamba/envs/youtube-demo/lib/python3.12/site-packages/pandas/io/sql.py:2422, in ADBCDatabase.has_table(self, name, schema)
   2419 if not schema_record:
   2420     continue
-> 2422 for table_record in schema_record["db_schema_tables"]:
   2423     if table_record["table_name"] == name:
   2424         return True

TypeError: 'NoneType' object is not iterable

Issue Description

The pandas code does not handle well the case where schema_record["db_schema_tables"] may be None

Expected Behavior

No error

Installed Versions

2.2.2

Metadata

Metadata

Labels

BugIO SQLto_sql, read_sql, read_sql_query

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions