Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PERF: read_csv GH#44106 #44192

Merged
merged 2 commits into from
Oct 26, 2021
Merged

PERF: read_csv GH#44106 #44192

merged 2 commits into from
Oct 26, 2021

Conversation

jbrockmendel
Copy link
Member

@jbrockmendel jbrockmendel commented Oct 26, 2021

Using the example in #44106 I time 188.4s on master and 3.9s on this PR.

@jreback jreback added IO CSV read_csv, to_csv Performance Memory or execution speed performance labels Oct 26, 2021
@jreback jreback added this to the 1.3.5 milestone Oct 26, 2021
@jreback
Copy link
Contributor

jreback commented Oct 26, 2021

can you add a release note for 1.3.5

@jreback jreback merged commit f9444ea into pandas-dev:master Oct 26, 2021
@jreback
Copy link
Contributor

jreback commented Oct 26, 2021

@meeseeksdev backport 1.3.x

@lumberbot-app
Copy link

lumberbot-app bot commented Oct 26, 2021

Something went wrong ... Please have a look at my logs.

@jreback
Copy link
Contributor

jreback commented Oct 26, 2021

should have asked if we have an asv that covers this?

@jbrockmendel jbrockmendel deleted the perf-44106 branch October 26, 2021 21:01
@jbrockmendel
Copy link
Member Author

should have asked if we have an asv that covers this?

not that i know of. @haghshenas want to have a go at implementing a benchmark for this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
IO CSV read_csv, to_csv Performance Memory or execution speed performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PERF: read_csv() very slow after updating to version 1.3.4 for CSV file with large number of columns
2 participants