Cannot query CSV file with non-standard extension from DataFusion CLI / Python bindings

[andygrove]

Describe the bug

I have CSV files with extension .tbl and it is not possible to query them through SQL. This is quite the barrier to running TPC-H benchmarks with the generated files.

To Reproduce

Querying a CSV file with .csv extension works

$ echo bob > customer.csv
$ datafusion-cli

DataFusion CLI v23.0.0
❯ CREATE EXTERNAL TABLE customer STORED AS CSV LOCATION 'customer.csv';
0 rows in set. Query took 0.008 seconds.
❯ SELECT * FROM customer;
+----------+
| column_1 |
+----------+
| bob      |
+----------+
1 row in set. Query took 0.001 seconds.

😄

Cannot query CSV file with .tbl extension

$ echo bob > customer.tbl
$ datafusion-cli

DataFusion CLI v23.0.0
❯ CREATE EXTERNAL TABLE customer STORED AS CSV LOCATION 'customer.tbl';
0 rows in set. Query took 0.001 seconds.
❯ SELECT * FROM customer;
0 rows in set. Query took 0.001 seconds.

😢

Expected behavior

No response

Additional context

No response

[andygrove]

@alamb fyi, would be great if we could resolve this to help with benchmarking. I am not sure how to work around this from Python right now.

Using low-level Rust APIs it is possible to work around this by registering a file extension (with the object store, IIRC).

[tustvold]

I think this may be a duplicate of #1736

[alamb]

Marking this as a good first issue (though I agree it is likely the same as #1736 this description I think is cleaner)

To anyone who finds this ticket, here is the suggested fix: #1736 (comment)