Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update filter usage to be more efficient #618

Merged
merged 1 commit into from
Apr 30, 2024
Merged

Update filter usage to be more efficient #618

merged 1 commit into from
Apr 30, 2024

Conversation

abelsiqueira
Copy link
Member

@abelsiqueira abelsiqueira commented Apr 30, 2024

Pull request details

Describe the changes made in this pull request

This improves the filter function based on investigations on performance and online comments.
A summary seems to be:

  • Use filter in the DataFrames syntax (filter(:column => action, df)) instead of subset (might not be true always).
  • When possible, use the "implicit" functions (don't know actual name), i.e., instead of :column => x -> x == value, use :column => ==(value). The ==(value) generates a function that checks for equality to value.
  • Use view=true, when possible. This we knew.

List of related issues or pull requests

Closes #ISSUE_NUMBER

Collaboration confirmation

As a contributor I confirm

  • I read and followed the instructions in README.dev.md
  • The documentation is up to date with the changes introduced in this Pull Request (or NA)
  • Tests are passing
  • Lint is passing

@abelsiqueira abelsiqueira added the benchmark PR only - Run benchmark on PR label Apr 30, 2024
Copy link
Contributor

github-actions bot commented Apr 30, 2024

Benchmark Results

1e267eb... 338010a... 1e267eb.../338010a09f3ad1...
energy_problem/create_model 0.108 ± 0.0054 h 0.108 ± 0.0048 h 0.997
energy_problem/input_and_constructor 27.2 ± 0.58 s 16.7 ± 1.2 s 1.63
time_to_load 2.62 ± 0.02 s 2.7 ± 0.024 s 0.97
1e267eb... 338010a... 1e267eb.../338010a09f3ad1...
energy_problem/create_model 0.749 G allocs: 31.2 GB 0.738 G allocs: 31 GB 1.01
energy_problem/input_and_constructor 0.249 G allocs: 4.34 GB 16.3 M allocs: 0.829 GB 5.24
time_to_load 0.153 k allocs: 14.5 kB 0.153 k allocs: 14.5 kB 1

Benchmark Plots

A plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR.
Go to "Actions"->"Benchmark a pull request"->[the most recent run]->"Artifacts" (at the bottom).

Copy link

codecov bot commented Apr 30, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (1e267eb) to head (338010a).

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #618   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           15        15           
  Lines          642       639    -3     
=========================================
- Hits           642       639    -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@abelsiqueira
Copy link
Member Author

Small improvement in time, but decent improvement in memory. Noticed as part of #611, so had to do it separately.

Copy link
Member

@datejada datejada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The benchmark looks really good. Thanks for the changes and improvement in the code!

@datejada datejada merged commit 179f064 into main Apr 30, 2024
7 checks passed
@datejada datejada deleted the better-filter branch April 30, 2024 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
benchmark PR only - Run benchmark on PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants