Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scan_csvdoes not support a list of datatypes in schema_overrides #17813

Open
2 tasks done
Tracked by #13040
nameexhaustion opened this issue Jul 23, 2024 · 1 comment
Open
2 tasks done
Tracked by #13040
Labels
A-io-csv Area: reading/writing CSV files accepted Ready for implementation bug Something isn't working P-low Priority: low python Related to Python Polars
Milestone

Comments

@nameexhaustion
Copy link
Collaborator

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

pl.scan_csv("https://j.mp/iriscsv", schema_overrides=4 * [pl.String]).collect()

Fails with TypeError: expected 'schema_overrides' dict, found 'list'

Log output

No response

Issue description

Needs to be fixed to allow dispatching read_csv to scan_csv

Expected behavior

Same output as read_csv:

pl.read_csv("https://j.mp/iriscsv", schema_overrides=4 * [pl.String])

shape: (150, 5)
┌──────────────┬─────────────┬──────────────┬─────────────┬───────────┐
│ sepal_lengthsepal_widthpetal_lengthpetal_widthspecies   │
│ ---------------       │
│ strstrstrstrstr       │
╞══════════════╪═════════════╪══════════════╪═════════════╪═══════════╡
│ 5.13.51.40.2setosa    │
│ 4.931.40.2setosa    │
│ 4.73.21.30.2setosa    │
│ 4.63.11.50.2setosa    │
│ 53.61.40.2setosa    │
│ …            ┆ …           ┆ …            ┆ …           ┆ …         │
│ 6.735.22.3virginica │
│ 6.32.551.9virginica │
│ 6.535.22virginica │
│ 6.23.45.42.3virginica │
│ 5.935.11.8virginica │
└──────────────┴─────────────┴──────────────┴─────────────┴───────────┘

Installed versions

1.2.1

@nameexhaustion nameexhaustion added bug Something isn't working python Related to Python Polars accepted Ready for implementation P-low Priority: low A-io-csv Area: reading/writing CSV files labels Jul 23, 2024
@nameexhaustion nameexhaustion added this to the 2.0.0 milestone Jul 25, 2024
@alonme
Copy link
Contributor

alonme commented Sep 1, 2024

I am taking a look 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-io-csv Area: reading/writing CSV files accepted Ready for implementation bug Something isn't working P-low Priority: low python Related to Python Polars
Projects
Status: Ready
Development

No branches or pull requests

2 participants