Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix for make_timeseries performance regression #10428

Merged
merged 5 commits into from Jul 28, 2023
Merged

Fix for make_timeseries performance regression #10428

merged 5 commits into from Jul 28, 2023

Conversation

j-bennet
Copy link
Contributor

Related to coiled/benchmarks#910.

  • Closes #xxxx
  • Tests added / passed
  • Passes pre-commit run --all-files

@j-bennet j-bennet requested review from phofl and fjetter July 25, 2023 17:13
@j-bennet j-bennet mentioned this pull request Jul 25, 2023
Closed
phofl
phofl reviewed Jul 25, 2023
View reviewed changes
dask/dataframe/io/demo.py Outdated Show resolved Hide resolved
phofl
phofl reviewed Jul 25, 2023
View reviewed changes
dask/dataframe/io/demo.py Outdated Show resolved Hide resolved
@j-bennet
Remove unnecesary check.
34b56d2
@j-bennet
Index.copy() is a shallow copy by default while astype(..., copy=Fals…
cb47974 
…e) will return exactly the same object. The latter can create problems, so better avoid it. Sharing indexes between two different objects is generally not a great idea, and pandas will move away from that soon.
@j-bennet j-bennet requested a review from phofl July 25, 2023 21:06
@j-bennet
Copy link
Contributor Author

@phofl I have a smaller reproduction example which confirms astype, as I suspected. Going to push the change here, and re-trigger the bencmarks ci.

@j-bennet
Copy link
Contributor Author

j-bennet commented Jul 26, 2023 •
edited

This fixes the regression.

image

Confirmed by this PR in benchmarks: coiled/benchmarks#914.

phofl
phofl approved these changes Jul 27, 2023
View reviewed changes
Copy link
Collaborator

@phofl phofl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx! lgtm.

@j-bennet
Copy link
Contributor Author

The failing test is unrelated.

@hendrikmakait hendrikmakait merged commit 1615724 into dask:main Jul 28, 2023
23 of 24 checks passed
@j-bennet j-bennet deleted the j-bennet/make-timeseries-regression branch July 28, 2023 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@phofl phofl phofl approved these changes

@fjetter fjetter Awaiting requested review from fjetter

Assignees
No one assigned
Labels
dataframe io
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@j-bennet @phofl @hendrikmakait