Different type mapping from postgres arrow and arrow_stream #864
Description
What language are you using?
Python
What version are you using?
0.4.4
What database are you using?
PostgreSQL
What dataframe are you using?
Arrow / Arrow Streaming
Can you describe your bug?
When I query postgres using "arrow" the postgres integer[] type is mapped to ids: large_list<item: int32> but when I switch to "arrow_stream" I get:
`Result::unwrap()` on an `Err` value: ConnectorX(NoConversionRule("Int4Array(true)", "connectorx::destinations::arrowstream::typesystem::ArrowTypeSystem"))`
What are the steps to reproduce the behavior?
reader: pyarrow.lib.RecordBatchReader = cx.read_sql(
credentials,
query=query,
return_type="arrow",
protocol="binary",
)
Works
reader: pyarrow.lib.RecordBatchReader = cx.read_sql(
credentials,
query=query,
return_type="arrow_stream",
protocol="binary",
batch_size=BATCH_SIZE,
)
Fails
If possible, please include a minimal simple example including:
Database setup if the error only happens on specific data or data type
CREATE TABLE test (
id integer,
values integer[]
PRIMARY KEY (id)
);
Example query / code
select * from test
What is the error?
ConnectorX(NoConversionRule("Int4Array(true)", "connectorx::destinations::arrowstream::typesystem::ArrowTypeSystem"))