Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add profiler runs on regression #1798

Merged
merged 5 commits into from
Mar 17, 2023

Conversation

ddyurchenko
Copy link
Collaborator

@ddyurchenko ddyurchenko commented Mar 14, 2023

What does this PR do?

This PR adds profiler runs for benchmarks if regression was detected. This should add more confidence in results and allow to see what caused change in performance.

Motivation

Describe how to test/QA your changes

Reviewer's Checklist

  • If known, an appropriate milestone has been selected; otherwise the Triage milestone is set.
  • Changed code has unit tests for its functionality.
  • If this interacts with the agent in a new way, a system test has been added.

@ddyurchenko ddyurchenko force-pushed the ddyurchenko/add-profiler-run-on-regression branch from 433d4a5 to 898b449 Compare March 14, 2023 13:00
@pr-commenter
Copy link

pr-commenter bot commented Mar 14, 2023

Benchmarks

Comparing candidate commit 843d685 in PR branch ddyurchenko/add-profiler-run-on-regression with baseline commit 96b9f30 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 18 metrics, 0 unstable metrics.

@ddyurchenko ddyurchenko marked this pull request as ready for review March 14, 2023 17:00
@ddyurchenko ddyurchenko requested a review from a team as a code owner March 14, 2023 17:00
felixge
felixge previously approved these changes Mar 14, 2023
Copy link
Member

@felixge felixge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but the bash is getting a bit wild in terms of duplications. Extracting some stuff into a simple bash functions might help a lot with keeping the code readable and maintainable.

Not a blocker for merging this, but if you have some time ...

.gitlab/scripts/run-benchmarks-with-profiler.sh Outdated Show resolved Hide resolved
.gitlab/scripts/analyze-results.sh Outdated Show resolved Hide resolved
@felixge felixge added this to the v1.49.0 milestone Mar 14, 2023
@ddyurchenko ddyurchenko force-pushed the ddyurchenko/add-profiler-run-on-regression branch from 8f52069 to d4fa189 Compare March 15, 2023 13:27
Copy link
Member

@felixge felixge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. 🚀

@felixge felixge merged commit 10e9cce into main Mar 17, 2023
@felixge felixge deleted the ddyurchenko/add-profiler-run-on-regression branch March 17, 2023 14:43
ddyurchenko added a commit that referenced this pull request Mar 17, 2023
* Add profiler runs on regression

* Add uploading to DD backend

* Remove duplication of code in bash scripts

---------

Co-authored-by: Felix Geisendörfer <felix@datadoghq.com>
ddyurchenko added a commit that referenced this pull request Mar 17, 2023
* Add profiler runs on regression

* Add uploading to DD backend

* Remove duplication of code in bash scripts

---------

Co-authored-by: Felix Geisendörfer <felix@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants