Read CSV format text from stdin or memory

[alamb]

Note: migrated from original JIRA: https://issues.apache.org/jira/browse/ARROW-12306

Hello,

I'm building a command line tool that can run SQL queries on text files (csv, json-line ..) . But the CsvExec in datafusion can only read csv text from files currently. I have checked its inner implantation the csv reader in arrow, anything impl Read could be a valid input.

 

Should this feature ( read csv from stdin) come with datafusion ? Or I just make it into my own crate.

[alamb]

Comment from Andrew Lamb(alamb) @ 2021-04-09T18:18:23.459+0000:

[~heymind] I think having the CsvExec be able to read from stdin (aka anything that impl `Read`) is a good idea


Perhaps you an extend CsvFile https://github.com/apache/arrow/blob/master/rust/datafusion/src/datasource/csv.rs#L50 so that it can take either a `Path` or something that implements `Read` and then pass that information down to the `CsvExec`

Comment from Siwei(heymind) @ 2021-04-11T06:57:39.254+0000:

Ok.I will do it.