Skip to content

Fix Perf tests AB mode #4656

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Jun 27, 2024
Merged

Fix Perf tests AB mode #4656

merged 3 commits into from
Jun 27, 2024

Conversation

pb8o
Copy link
Contributor

@pb8o pb8o commented Jun 27, 2024 •
edited
Loading

Changes

Fix perf tests when running in AB mode

Reason

After the build sharing changes, this mode was working because we were still building Firecracker (unintentionally). After we removed that capability in 467c90d this mode started failing.

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

  • If a specific issue led to this PR, this PR closes the issue.
  • The description of changes is clear and encompassing.
  • Any required documentation changes (code and docs) are included in this
    PR.
  • API changes follow the Runbook for Firecracker API changes.
  • User-facing changes are mentioned in CHANGELOG.md.
  • All added/changed functionality is tested.
  • New TODOs link to an issue.
  • Commits meet
    contribution quality standards.
  • This functionality cannot be added in rust-vmm.

@pb8o pb8o added Type: Bug Indicates an unexpected problem or unintended behavior Priority: Medium Indicates than an issue or pull request should be resolved ahead of issues or pull requests labelled Status: Awaiting author Indicates that an issue or pull request requires author action python Pull requests that update Python code labels Jun 27, 2024
@pb8o pb8o requested a review from roypat June 27, 2024 09:09
@pb8o pb8o self-assigned this Jun 27, 2024
@pb8o pb8o added Status: Awaiting review Indicates that a pull request is ready to be reviewed and removed Status: Awaiting author Indicates that an issue or pull request requires author action labels Jun 27, 2024
bchalios
bchalios reviewed Jun 27, 2024
View reviewed changes
@codecov Codecov
Copy link

codecov bot commented Jun 27, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.08%. Comparing base (3666267) to head (f2d622e).

Additional details and impacted files 
@@           Coverage Diff           @@
##             main    #4656   +/-   ##
=======================================
  Coverage   82.08%   82.08%           
=======================================
  Files         255      255           
  Lines       31257    31257           
=======================================
  Hits        25656    25656           
  Misses       5601     5601
Flag Coverage Δ
4.14-c5n.metal 79.57% <ø> (ø)
4.14-m5n.metal 79.56% <ø> (+<0.01%) ⬆️
4.14-m6a.metal 78.78% <ø> (ø)
4.14-m6g.metal 76.60% <ø> (ø)
4.14-m6i.metal 79.55% <ø> (+<0.01%) ⬆️
4.14-m7g.metal 76.60% <ø> (ø)
5.10-c5n.metal 82.09% <ø> (ø)
5.10-m5n.metal 82.07% <ø> (+<0.01%) ⬆️
5.10-m6a.metal 81.38% <ø> (+<0.01%) ⬆️
5.10-m6g.metal 79.37% <ø> (ø)
5.10-m6i.metal 82.07% <ø> (ø)
5.10-m7g.metal 79.37% <ø> (ø)
6.1-c5n.metal 82.09% <ø> (ø)
6.1-m5n.metal 82.07% <ø> (-0.01%) ⬇️
6.1-m6a.metal 81.38% <ø> (-0.01%) ⬇️
6.1-m6g.metal 79.37% <ø> (+<0.01%) ⬆️
6.1-m6i.metal 82.06% <ø> (-0.01%) ⬇️
6.1-m7g.metal 79.36% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

roypat
roypat previously approved these changes Jun 27, 2024
View reviewed changes
bchalios
bchalios previously approved these changes Jun 27, 2024
View reviewed changes
pb8o added 3 commits June 27, 2024 12:56
@pb8o
fix(ci): set retry list
c4a1be5 
In commit f5906ee we moved the
initialization of the list but forgot to change it here.

Fixes: f5906ee

Signed-off-by: Pablo Barbáchano <pablob@amazon.com>
@pb8o
fix(ab): only convert to commit ids when decorating metrics
adf7164 
This matches the directory names of the shared build.

Fixes: 0c55e10

Signed-off-by: Pablo Barbáchano <pablob@amazon.com>
@pb8o
fix(ab): fix shared build location
f2d622e 
In 0c55e10 we introduced shared builds
but that change was not effective until
467c90d at which point it broke AB
tests.

Fix commands so that AB tests work again.

Fixes: 0c55e10

Signed-off-by: Pablo Barbáchano <pablob@amazon.com>
@pb8o pb8o dismissed stale reviews from bchalios and roypat via f2d622e June 27, 2024 10:56
@pb8o pb8o merged commit 6d6caf4 into firecracker-microvm:main Jun 27, 2024
@pb8o pb8o deleted the fix-ab-perf branch June 27, 2024 11:50
This was referenced Sep 29, 2025
Merged
Closed
Closed
@dependabot dependabot bot mentioned this pull request Oct 13, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bchalios bchalios bchalios approved these changes

+1 more reviewer

@roypat roypat roypat approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@pb8o pb8o

Labels

Priority: Medium Indicates than an issue or pull request should be resolved ahead of issues or pull requests labelled python Pull requests that update Python code Status: Awaiting review Indicates that a pull request is ready to be reviewed Type: Bug Indicates an unexpected problem or unintended behavior

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@pb8o @bchalios @roypat