Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

koordlet: support numa topology reporting #1516

Conversation

saintube
Copy link
Member

@saintube saintube commented Jul 31, 2023

Ⅰ. Describe what this PR does

  • Add the node NUMA info collection in the NodeInfoCollector.
  • Add the NUMA topology reporting in the NRT state informer.
  • Fix a bug where a prom metric is not registered.
  • Refactor the internal LocalCPUInfo to save the cpu maps instead of the cpu counts.

Ⅱ. Does this pull request fix one issue?

Fixes #1503.

Ⅲ. Describe how to verify it

Ⅳ. Special notes for reviews

V. Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests
  • All checks passed in make test

@codecov
Copy link

codecov bot commented Jul 31, 2023

Codecov Report

Patch coverage: 80.00% and project coverage change: +0.15% 🎉

Comparison is base (53efaaf) 64.62% compared to head (66c0920) 64.78%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1516      +/-   ##
==========================================
+ Coverage   64.62%   64.78%   +0.15%     
==========================================
  Files         346      347       +1     
  Lines       35123    35355     +232     
==========================================
+ Hits        22699    22905     +206     
- Misses      10756    10768      +12     
- Partials     1668     1682      +14     
Flag Coverage Δ
unittests 64.78% <80.00%> (+0.15%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
pkg/koordlet/metriccache/metric_types.go 58.13% <ø> (ø)
pkg/koordlet/util/kubelet/kubelet.go 56.28% <0.00%> (ø)
pkg/koordlet/util/system/system_file.go 62.85% <0.00%> (-3.81%) ⬇️
pkg/koordlet/metrics/common.go 74.57% <70.00%> (+13.35%) ⬆️
...advisor/collectors/nodeinfo/node_info_collector.go 80.32% <77.77%> (ø)
...statesinformer/impl/states_noderesourcetopology.go 69.56% <81.51%> (+3.45%) ⬆️
pkg/koordlet/util/meminfo.go 82.47% <81.69%> (-5.34%) ⬇️
...collectors/noderesource/node_resource_collector.go 65.43% <100.00%> (ø)
...et/qosmanager/plugins/resctrl/resctrl_reconcile.go 76.22% <100.00%> (ø)
pkg/koordlet/util/cpuinfo.go 72.99% <100.00%> (+0.19%) ⬆️

... and 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@saintube saintube force-pushed the koordlet-support-numa-topo-management branch from d9a41b7 to 7a9dcb4 Compare August 2, 2023 03:53
@koordinator-bot koordinator-bot bot added size/XXL and removed size/L labels Aug 2, 2023
@saintube saintube changed the title [WIP] koordlet: support numa topology reporting koordlet: support numa topology reporting Aug 2, 2023
pkg/koordlet/util/meminfo.go Outdated Show resolved Hide resolved
if err != nil {
return 0, err
}
usage := int64(memInfo.MemTotal - memInfo.MemAvailable)
return usage, nil
}

type NUMAInfo struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is socket id can be identified?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm afraid not. There is no info about the socket and core relations under the numa info directory. If we want to look up the socket ID of the NUMA node, we need to check the cpu info instead.

pkg/koordlet/util/meminfo.go Outdated Show resolved Hide resolved
Signed-off-by: saintube <saintube@foxmail.com>
Copy link
Member

@eahydra eahydra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!
/lgtm

@hormes
Copy link
Member

hormes commented Aug 3, 2023

/approve

@koordinator-bot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hormes

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@koordinator-bot koordinator-bot bot merged commit 4c274b5 into koordinator-sh:main Aug 3, 2023
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[proposal] koordlet identifies and reports the NUMA stats
4 participants