Parquet FLOAT vs. DOUBLE type issue

[prrao87]

Kùzu version: 0.0.10

I've tried upgrading to the latest version 0.0.10 that was just released, and noticed that there's an issue with parquet reading FLOAT vs DOUBLE data type, which wasn't present in the previous version. I think the default behaviour should be more accepting of both data types, as the data in parquet can come from many different sources? The parquet file attached here came from Polars with snappy compression.

MRE

import duckdb

conn_1 = duckdb.connect()
tbl = conn_1.sql("SELECT * FROM 'cities.parquet'")

print(tbl)

import kuzu

db = kuzu.Database(f"./test")
conn = kuzu.Connection(db)

def create_city_node_table(conn) -> None:
    conn.execute(
        """
        CREATE NODE TABLE
            City(
                id INT64,
                city STRING,
                state STRING,
                country STRING,
                lat FLOAT,
                lon FLOAT,
                population INT64,
                PRIMARY KEY (id)
            )
        """
    )

create_city_node_table(conn)
conn.execute(f"COPY City FROM 'cities.parquet';")

$ python test.py
┌───────┬────────────┬─────────┬─────────┬─────────┬───────────┬────────────┐
│  id   │    city    │  state  │ country │   lat   │    lng    │ population │
│ int64 │  varchar   │ varchar │ varchar │ double  │  double   │   int32    │
├───────┼────────────┼─────────┼─────────┼─────────┼───────────┼────────────┤
│     1 │ Airdrie    │ Alberta │ Canada  │ 51.2917 │ -114.0144 │      61581 │
│     2 │ Beaumont   │ Alberta │ Canada  │ 53.3572 │ -113.4147 │      17396 │
│     3 │ Blackfalds │ Alberta │ Canada  │ 52.3833 │    -113.8 │       9328 │
│     4 │ Brooks     │ Alberta │ Canada  │ 50.5642 │ -111.8989 │      14451 │
│     5 │ Calgary    │ Alberta │ Canada  │   51.05 │ -114.0667 │    1239220 │
└───────┴────────────┴─────────┴─────────┴─────────┴───────────┴────────────┘

Traceback (most recent call last):
  File "/code/kuzudb-study/test.py", line 31, in <module>
    conn.execute(f"COPY City FROM 'cities.parquet';")
  File "/code/kuzudb-study/.venv/lib/python3.11/site-packages/kuzu/connection.py", line 90, in execute
    self._connection.execute(
RuntimeError: Binder exception: Column `lat` type mismatch. Expected FLOAT but got DOUBLE.

Example file

cities.parquet.zip

As can be seen, the parquet output from DuckDB is perfectly fine, and is of type DOUBLE. I think the correct behaviour here should be that Kùzu coerces the data type to FLOAT without the user having to worry about it?

[ray6080]

Hi @prrao87 , yeah this is a known issue, I totally agree with you. We introduced more rigid rules for data type checks during the binding phase to fix some type mismatch bugs, but not able to finish the casting part yet. Before the next major release, we will add back more flexible data type castings. Sorry for the inconvenience caused on your side.

[semihsalihoglu-uw]

Labeling with both bug and feature as it's not very clear to me which one is more accurate.

[andyfengHKU]

Solved in #3887