Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

koordlet: add nri server for runtimehooks #1501

Merged
merged 10 commits into from Aug 14, 2023

Conversation

kangclzjc
Copy link
Contributor

Ⅰ. Describe what this PR does

Koordinator as a QoS based scheduling system for hybrid workloads orchestration on Kubernetes and its runtime hooks support two working modes for different scenarios: Standalone and Proxy. However, both of them have some constraints in 2023/02/07 meeting discussion. We'd like to integrate NRI framework to address these constraints based on community recommend mechanism.

This PR will implement a NRI plugin in koordlet. Currently, it co-exist with standalone and proxy mode. Koodlet will start an NRI plugin to subscribe pod/container lifecycle events from container runtime (e.g. containerd, crio), and then koordlet NRI plugin will call runtime hooks to adjust pod resources or OCI spec.

Ⅱ. Does this pull request fix one issue?

fix #1277

Ⅲ. Describe how to verify it

  1. Create BE pod with batch_resource hook enable
  2. Check pod/container level cpu.cfs_quota_us and memory.limit_in_bytes values

Ⅳ. Special notes for reviews

  1. Some errors may need to pass to containerd based on PluginFailurePolicy, this may be done in the future
  2. Other hook points may be added in the future

V. Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests
  • All checks passed in make test

@kangclzjc kangclzjc force-pushed the dev-nri-0727 branch 5 times, most recently from 83a3535 to ae53333 Compare July 27, 2023 10:07
pkg/koordlet/runtimehooks/config.go Outdated Show resolved Hide resolved
pkg/koordlet/runtimehooks/nri/server.go Outdated Show resolved Hide resolved
pkg/koordlet/runtimehooks/nri/server.go Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Jul 27, 2023

Codecov Report

Patch coverage: 72.38% and project coverage change: +0.81% 🎉

Comparison is base (091cd40) 64.04% compared to head (aa59513) 64.85%.
Report is 17 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1501      +/-   ##
==========================================
+ Coverage   64.04%   64.85%   +0.81%     
==========================================
  Files         340      349       +9     
  Lines       35056    35802     +746     
==========================================
+ Hits        22452    23220     +768     
+ Misses      10938    10864      -74     
- Partials     1666     1718      +52     
Flag Coverage Δ
unittests 64.85% <72.38%> (+0.81%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
pkg/koordlet/runtimehooks/nri/server.go 54.83% <54.83%> (ø)
pkg/koordlet/runtimehooks/runtimehooks.go 70.47% <58.33%> (-3.60%) ⬇️
...oordlet/runtimehooks/protocol/container_context.go 50.21% <90.76%> (+18.07%) ⬆️
pkg/koordlet/runtimehooks/config.go 100.00% <100.00%> (ø)
pkg/koordlet/runtimehooks/protocol/pod_context.go 25.35% <100.00%> (+16.80%) ⬆️

... and 62 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@saintube
Copy link
Member

@koordinator-bot
Copy link

@saintube: GitHub didn't allow me to request PR reviews from the following users: PTAL.

Note that only koordinator-sh members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @zwzhang0107 @hormes @eahydra @FillZpp @ZiMengSheng @jasonliu747 @ZYecho @honpey PTAL

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@hormes
Copy link
Member

hormes commented Jul 27, 2023

Welcome to participate in Koordinator's technical co-construction, and jointly promote the evolution of the framework of community standards(nri).

@kangclzjc kangclzjc force-pushed the dev-nri-0727 branch 8 times, most recently from 173aed8 to c0289fa Compare July 28, 2023 03:01
pkg/koordlet/runtimehooks/runtimehooks.go Outdated Show resolved Hide resolved
pkg/koordlet/runtimehooks/nri/server.go Outdated Show resolved Hide resolved
Signed-off-by: Kang.Zhang <kang.zhang@intel.com>
Signed-off-by: Kang.Zhang <kang.zhang@intel.com>
Signed-off-by: Kang.Zhang <kang.zhang@intel.com>
Signed-off-by: Kang.Zhang <kang.zhang@intel.com>
Copy link
Member

@saintube saintube left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
@kangclzjc Thanks for your contributions!

@koordinator-bot koordinator-bot bot added the lgtm label Aug 9, 2023
@zwzhang0107 zwzhang0107 changed the title add nri server koordlet: add nri server for runtimehooks Aug 10, 2023
pkg/koordlet/runtimehooks/runtimehooks.go Outdated Show resolved Hide resolved
pkg/koordlet/runtimehooks/nri/server.go Outdated Show resolved Hide resolved
pkg/koordlet/runtimehooks/nri/server.go Outdated Show resolved Hide resolved
Signed-off-by: Kang.Zhang <kang.zhang@intel.com>
@koordinator-bot koordinator-bot bot removed the lgtm label Aug 10, 2023
Signed-off-by: Kang.Zhang <kang.zhang@intel.com>
@zwzhang0107 zwzhang0107 added this to the v1.3 milestone Aug 10, 2023
Signed-off-by: Kang.Zhang <kang.zhang@intel.com>
Signed-off-by: Kang.Zhang <kang.zhang@intel.com>
Signed-off-by: Kang.Zhang <kang.zhang@intel.com>
@zwzhang0107
Copy link
Contributor

/lgtm
/approve

@saintube
Copy link
Member

/assign @hormes @FillZpp

@hormes
Copy link
Member

hormes commented Aug 14, 2023

/approve

@koordinator-bot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hormes, zwzhang0107

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@koordinator-bot koordinator-bot bot merged commit 5f2538f into koordinator-sh:main Aug 14, 2023
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[proposal] NRI on Koordinator
5 participants