Datafusion timestamp type doesn't respect delta lake schema

[Veiasai]

Environment

WSL2 Ubuntu 22.04

Delta-rs version:
0.17.1

Binding:

Environment:

• Cloud provider:
• OS: Ubuntu 22.04
• Other:

---

Bug

What happened:
the timestamp in delta table definition is macro, and in parquet it is nanos. (Spark behavior)
Then, when datafusion reads it, the schema seems to be inferred on parquet raw data, rather than from delta lake definition.

Field { name: "exchange_time", data_type: Timestamp(Nanosecond, None), nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }

What you expected to happen:

How to reproduce it:

More details:

[ion-elgreco]

This is a limitation of datafusion, please set the correct spark conf : SparkSession.config("spark.sql.parquet.outputTimestampType", "TIMESTAMP_MICROS")