Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[YUNIKORN-2500] Use GetPreemptableResource, GetRemainingGuaranteedRes… #830

Closed
wants to merge 9 commits into from

Conversation

manirajv06
Copy link
Contributor

…ource in Preemption flow

What is this PR for?

Use GetPreemptableResource, GetRemainingGuaranteedResource instead of old methods in preemption flow. In addition, there are some other changes to ensure the whole flow works as earlier.

What type of PR is it?

  • - Improvement

Todos

  • - Task

What is the Jira issue?

https://issues.apache.org/jira/browse/YUNIKORN-2500

How should this be tested?

Screenshots (if appropriate)

Questions:

  • - The licenses files need update.
  • - There is breaking changes for older versions.
  • - It needs documentation.

Copy link

codecov bot commented Mar 29, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.17%. Comparing base (b07f264) to head (f7b1454).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #830      +/-   ##
==========================================
- Coverage   79.35%   79.17%   -0.18%     
==========================================
  Files          82       82              
  Lines       11362    11330      -32     
==========================================
- Hits         9016     8971      -45     
- Misses       2019     2033      +14     
+ Partials      327      326       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@manirajv06
Copy link
Contributor Author

Ran the e2e test in my local setup to verify the changes. It is running with out any failures.

Copy link
Contributor

@craigcondit craigcondit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-1. I know that there is a spreadsheet of behavioral cases that has been created for this to document how the preemption logic should behave in a variety of circumstances. I would absolutely like to see a PR consisting only of test cases for each of the identified scenarios. We can disable the failing ones for now, but that would give us a baseline for how "compliant" the existing preemption logic is. Then YUNIKORN-2500 can be rebased on top of that, re-enabling the tests that it fixes. That will ensure we don't regress compared to the existing behavior.

@codecov-commenter
Copy link

codecov-commenter commented May 2, 2024

Codecov Report

Attention: Patch coverage is 94.69027% with 6 lines in your changes missing coverage. Please review.

Project coverage is 78.22%. Comparing base (5e3535c) to head (1738928).

Files Patch % Lines
pkg/scheduler/objects/preemption.go 92.59% 3 Missing and 1 partial ⚠️
pkg/common/resources/resources.go 95.74% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #830      +/-   ##
==========================================
+ Coverage   78.01%   78.22%   +0.21%     
==========================================
  Files          97       97              
  Lines       12119    12177      +58     
==========================================
+ Hits         9455     9526      +71     
+ Misses       2354     2346       -8     
+ Partials      310      305       -5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@manirajv06
Copy link
Contributor Author

-1. I know that there is a spreadsheet of behavioral cases that has been created for this to document how the preemption logic should behave in a variety of circumstances. I would absolutely like to see a PR consisting only of test cases for each of the identified scenarios. We can disable the failing ones for now, but that would give us a baseline for how "compliant" the existing preemption logic is. Then YUNIKORN-2500 can be rebased on top of that, re-enabling the tests that it fixes. That will ensure we don't regress compared to the existing behavior.

PR #847 covers all the different test cases but skipped for now. Once the same has been merged, this pr will fix all those test cases with necessary changes.

Copy link
Contributor

@craigcondit craigcondit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, would like Wilfred's input as well before commit.

Copy link
Contributor

@wilfred-s wilfred-s left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

preemption side looks good
StrictlyGreaterThanOnlyExisting needs clarification

pkg/common/resources/resources.go Outdated Show resolved Hide resolved
pkg/common/resources/resources.go Outdated Show resolved Hide resolved
pkg/common/resources/resources.go Outdated Show resolved Hide resolved
@@ -816,6 +874,29 @@ func ComponentWiseMinPermissive(left, right *Resource) *Resource {
return out
}

// ComponentWiseMinOnlyExisting Returns a new Resource with the smallest value for resource type
// existing only in left but not vice versa.
func ComponentWiseMinOnlyExisting(left, right *Resource) *Resource {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the 3rd version of ComponentWiseMin... need to think about a refactor in a follow up jira

Copy link
Contributor

@pbacsko pbacsko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two coverage related comments + one about logging.

pkg/common/resources/resources.go Show resolved Hide resolved
pkg/scheduler/objects/preemption.go Show resolved Hide resolved
pkg/scheduler/objects/preemption.go Show resolved Hide resolved
Copy link
Contributor

@wilfred-s wilfred-s left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
@pbacsko I think we're OK, please have a last check

Copy link
Contributor

@pbacsko pbacsko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants