Open
Description
What happened?
After upgrading existing Ibis code that uses Table.union
to 10.4.0, I started receiving error messages about string lengths not matching, e.g.:
Traceback (most recent call last):
| File "/Users/.../.venv/lib/python3.11/site-packages/ibis/expr/operations/relations.py", line 343, in __init__
| missing_from_left = right.schema - left.schema
| ~~~~~~~~~~~~~^~~~~~~~~~~~~
| File "/Users/.../.venv/lib/python3.11/site-packages/ibis/common/collections.py", line 247, in __sub__
| common_keys = self._check_conflict(other)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/Users/.../.venv/lib/python3.11/site-packages/ibis/common/collections.py", line 212, in _check_conflict
| raise ConflictingValuesError(conflicts)
| ibis.common.exceptions.ConflictingValuesError: Conflicting values for keys:
| `customer_id`: string(50) != string(4000)
In this scenario I am trying to union tables created via backend.sql(query_str)
where each source table has a different length for the column in Snowflake, e.g.
query 1:
select customer_id from some_table1
query 2:
select customer_id from some_table2
Is this a bug, or expected behavior after the implementation of #9382?
I can understand preference of the error to force explicit casting, but wondering if there is a way to keep the old upscaling behavior for the cases when it doesn't matter if the smaller types are kept. To work around this I'd have to always cast a few hundred columns to strings of a fixed length.
What version of ibis are you using?
10.4.0
What backend(s) are you using, if any?
Snowflake
Relevant log output
Code of Conduct
- I agree to follow this project's Code of Conduct
Metadata
Metadata
Assignees
Type
Projects
Status
backlog