Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

koordlet: CPI collector for Interference Detection #622

Conversation

songtao98
Copy link
Contributor

@songtao98 songtao98 commented Sep 16, 2022

Signed-off-by: songtao98 songtao2603060@gmail.com

Ⅰ. Describe what this PR does

This PR is the first part of koordinator interference detection support, which implements CPI metric collect and store feature. Koordlet collects CPI once a minute in a 10 second time window and store it in metric cache on per container level for future usage.
Other codes will be commited continuously with other PRs.

Ⅱ. Does this pull request fix one issue?

Ⅲ. Describe how to verify it

Ⅳ. Special notes for reviews

V. Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests
  • All checks passed in make test

@songtao98
Copy link
Contributor Author

/hold

pkg/util/perf.go Outdated Show resolved Hide resolved
pkg/util/perf.go Outdated Show resolved Hide resolved
pkg/util/perf.go Outdated Show resolved Hide resolved
pkg/koordlet/metricsadvisor/collector.go Outdated Show resolved Hide resolved
pkg/koordlet/metricsadvisor/collector.go Outdated Show resolved Hide resolved
pkg/util/perf.go Outdated Show resolved Hide resolved
pkg/koordlet/metriccache/api.go Outdated Show resolved Hide resolved
pkg/koordlet/metriccache/api.go Outdated Show resolved Hide resolved
pkg/koordlet/metriccache/api.go Outdated Show resolved Hide resolved
pkg/koordlet/metriccache/api.go Outdated Show resolved Hide resolved
@koordinator-bot koordinator-bot bot added size/XL and removed size/L labels Sep 29, 2022
@songtao98 songtao98 force-pushed the feature_interference_collect_metrics branch 2 times, most recently from 385b85a to 6e8b7d9 Compare September 29, 2022 08:24
pkg/koordlet/metricsadvisor/interference_collector.go Outdated Show resolved Hide resolved
apis/slo/v1alpha1/interference_metrics.go Outdated Show resolved Hide resolved
apis/slo/v1alpha1/interference_metrics.go Outdated Show resolved Hide resolved
pkg/features/koordlet_features.go Outdated Show resolved Hide resolved
pkg/koordlet/metriccache/storage_tables.go Outdated Show resolved Hide resolved
pkg/util/perf.go Outdated Show resolved Hide resolved
pkg/util/stat.go Outdated Show resolved Hide resolved
pkg/util/stat.go Outdated Show resolved Hide resolved
pkg/util/perf.go Outdated Show resolved Hide resolved
pkg/util/perf.go Outdated Show resolved Hide resolved
pkg/util/perf.go Outdated Show resolved Hide resolved
@songtao98 songtao98 changed the title Interference detection support Interference detection support: CPI collector Sep 29, 2022
@songtao98 songtao98 force-pushed the feature_interference_collect_metrics branch from 0dd3f3d to b6fe121 Compare September 29, 2022 16:45
@koordinator-bot koordinator-bot bot added size/L and removed size/XL labels Sep 29, 2022
@songtao98 songtao98 force-pushed the feature_interference_collect_metrics branch 4 times, most recently from ec7e4b9 to 989a88e Compare October 10, 2022 09:28
@codecov
Copy link

codecov bot commented Oct 10, 2022

Codecov Report

Base: 68.74% // Head: 68.73% // Decreases project coverage by -0.00% ⚠️

Coverage data is based on head (7798b3e) compared to base (24b5d4c).
Patch coverage: 71.23% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #622      +/-   ##
==========================================
- Coverage   68.74%   68.73%   -0.01%     
==========================================
  Files         205      207       +2     
  Lines       23356    23726     +370     
==========================================
+ Hits        16055    16308     +253     
- Misses       6200     6289      +89     
- Partials     1101     1129      +28     
Flag Coverage Δ
unittests 68.73% <71.23%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
pkg/koordlet/metriccache/storage_tables.go 57.14% <ø> (ø)
pkg/koordlet/metricsadvisor/collector.go 9.26% <0.00%> (-0.27%) ⬇️
pkg/util/perf/perf_linux.go 47.50% <47.50%> (ø)
pkg/koordlet/metriccache/metric_cache.go 57.72% <76.71%> (+5.67%) ⬆️
pkg/util/stat.go 81.08% <76.92%> (-2.26%) ⬇️
...dlet/metricsadvisor/performance_collector_linux.go 82.14% <82.14%> (ø)
pkg/koordlet/metriccache/storage.go 93.47% <100.00%> (+1.07%) ⬆️
pkg/koordlet/metricsadvisor/config.go 100.00% <100.00%> (ø)
pkg/util/container.go 69.23% <100.00%> (+1.04%) ⬆️
pkg/scheduler/plugins/elasticquota/controller.go 60.00% <0.00%> (-12.31%) ⬇️
... and 1 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@koordinator-bot koordinator-bot bot added size/XL and removed size/L labels Oct 10, 2022
@songtao98 songtao98 force-pushed the feature_interference_collect_metrics branch from b36210a to 03ed56b Compare October 10, 2022 14:00
@songtao98 songtao98 force-pushed the feature_interference_collect_metrics branch from 03ed56b to b0fe218 Compare October 10, 2022 14:03
@songtao98 songtao98 force-pushed the feature_interference_collect_metrics branch 17 times, most recently from c2cfed0 to 3d3fbd0 Compare October 26, 2022 11:31
@songtao98 songtao98 changed the title Interference detection support: CPI collector koordlet: CPI collector for Interference Detection Oct 26, 2022
@songtao98 songtao98 force-pushed the feature_interference_collect_metrics branch from 3d3fbd0 to da851da Compare October 26, 2022 13:12
Signed-off-by: songtao98 <songtao2603060@gmail.com>
@songtao98 songtao98 force-pushed the feature_interference_collect_metrics branch from da851da to 7798b3e Compare October 27, 2022 07:31
@zwzhang0107
Copy link
Contributor

/lgtm

Copy link
Member

@jasonliu747 jasonliu747 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@koordinator-bot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by: jasonliu747

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@koordinator-bot koordinator-bot bot merged commit 0dfffbd into koordinator-sh:main Oct 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants