Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve speed of outcome count per threshold #545

Merged
merged 1 commit into from
Apr 14, 2023

Conversation

gabegma
Copy link
Contributor

@gabegma gabegma commented Apr 12, 2023

Description:

I tested and it’s actually faster to call compute_on_dataset_split() for Outcomes, so we don't need to wait for the tasks to be scheduled. This is the case because we call Outcomes many times with different thresholds. See the difference in the profiling logs.
logs_compute_on_ds_split.txt
logs_get_task.txt

Checklist:

You should check all boxes before the PR is ready. If a box does not apply, check it to acknowledge it.

  • ISSUE NUMBER. You linked the issue number (Ex: Resolve #XXX).
  • PRE-COMMIT. You ran pre-commit on all commits, or else, you
    ran pre-commit run --all-files at the end.
  • USER CHANGES. The changes are added to CHANGELOG.md and the documentation, if they impact
    our users.
  • DEV CHANGES.
    • Update the documentation if this PR changes how to develop/launch on the app.
    • Update the README files and our wiki for any big design decisions, if relevant.
    • Add unit tests, docstrings, typing and comments for complex sections.

@gabegma gabegma self-assigned this Apr 12, 2023
@gabegma gabegma marked this pull request as ready for review April 12, 2023 21:32
@gabegma gabegma force-pushed the ggm/improve-speed-outcome-per-threshold branch from 952129f to d6dc25a Compare April 13, 2023 20:37
@gabegma gabegma changed the base branch from main to ggm/refactor-validation-modules April 13, 2023 20:38
@gabegma gabegma force-pushed the ggm/refactor-validation-modules branch from 601875c to b87c32e Compare April 14, 2023 17:31
@gabegma gabegma force-pushed the ggm/improve-speed-outcome-per-threshold branch from d6dc25a to 274f69d Compare April 14, 2023 17:33
Copy link
Contributor

@JosephMarinier JosephMarinier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤯

Base automatically changed from ggm/refactor-validation-modules to main April 14, 2023 17:54
@gabegma gabegma force-pushed the ggm/improve-speed-outcome-per-threshold branch from 274f69d to 0f4cc88 Compare April 14, 2023 19:40
@gabegma gabegma enabled auto-merge April 14, 2023 19:40
@gabegma gabegma merged commit f546770 into main Apr 14, 2023
@gabegma gabegma deleted the ggm/improve-speed-outcome-per-threshold branch April 14, 2023 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants