Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

ovs: increase cpu limit to 2 cores #3530

Merged
merged 1 commit into from
Jan 4, 2024

Conversation

zhangzujian
Copy link
Member

@zhangzujian zhangzujian commented Dec 15, 2023

Pull Request

What type of this PR

Examples of user facing changes:

  • Features
  • Bug fixes
  • Docs
  • Tests

Which issue(s) this PR fixes

In some scenarios ovn-controller and ovs-vswitchd requires more cpu resources.

ovs-vswitchd logs:

2023-12-13T03:08:30.098Z|18849|ovs_rcu(urcu8)|WARN|blocked 1052 ms waiting for main to quiesce
2023-12-13T03:08:31.050Z|18850|ovs_rcu(urcu8)|WARN|blocked 2004 ms waiting for main to quiesce
2023-12-13T03:08:33.047Z|18851|ovs_rcu(urcu8)|WARN|blocked 4001 ms waiting for main to quiesce
2023-12-13T03:08:37.046Z|18852|ovs_rcu(urcu8)|WARN|blocked 8000 ms waiting for main to quiesce
2023-12-13T03:08:45.046Z|18853|ovs_rcu(urcu8)|WARN|blocked 16000 ms waiting for main to quiesce
2023-12-13T03:08:45.124Z|115041|timeval|WARN|Unreasonably long 16306ms poll interval (6568ms user, 258ms system)
2023-12-13T03:08:45.124Z|115042|timeval|WARN|faults: 284 minor, 0 major
2023-12-13T03:08:45.124Z|115043|timeval|WARN|context switches: 111 voluntary, 2576 involuntary
2023-12-13T03:08:45.135Z|115044|rconn|WARN|br-int<->unix#3839: connection dropped (Broken pipe)
2023-12-13T03:08:45.154Z|115045|bridge|WARN|could not open network device a20024808c24_h (No such device)
2023-12-13T03:08:45.209Z|115046|bridge|WARN|could not open network device 9d941e6ad898_h (No such device)
2023-12-13T03:08:45.210Z|115047|bridge|INFO|bridge br-int: added interface 68d3fa2561d3_h on port 7665
2023-12-13T03:08:45.211Z|115048|bridge|INFO|bridge br-int: added interface 5598ef96024a_h on port 7666
2023-12-13T03:08:45.217Z|115049|bridge|WARN|could not open network device bfce8cbd8aca_h (No such device)
2023-12-13T03:08:45.225Z|115050|bridge|WARN|could not open network device edf02cfe62a6_h (No such device)
2023-12-13T03:08:45.234Z|115051|bridge|WARN|could not open network device d2ccb8652c7d_h (No such device)
2023-12-13T03:08:45.240Z|115052|bridge|WARN|could not open network device 934a261104fc_h (No such device)
2023-12-13T03:08:45.245Z|115053|bridge|WARN|could not open network device bc5179c88860_h (No such device)
2023-12-13T03:08:45.246Z|115054|bridge|INFO|bridge br-int: added interface f99ff40f9c56_h on port 7667
2023-12-13T03:08:45.252Z|115055|bridge|WARN|could not open network device b376701ee9af_h (No such device)
2023-12-13T03:08:45.253Z|115056|bridge|INFO|bridge br-int: added interface 809191aaeb10_h on port 7668
2023-12-13T03:08:45.258Z|115057|bridge|WARN|could not open network device 56514f9dba75_h (No such device)
2023-12-13T03:08:45.259Z|115058|bridge|INFO|bridge br-int: added interface 3267b851338a_h on port 7669
2023-12-13T03:08:45.262Z|115059|bridge|WARN|could not open network device eef4bfc5cc37_h (No such device)
2023-12-13T03:08:45.263Z|115060|bridge|INFO|bridge br-int: added interface 44fe398e70a3_h on port 7670
2023-12-13T03:08:45.267Z|115061|bridge|WARN|could not open network device 0ea33c15a877_h (No such device)
2023-12-13T03:08:45.270Z|115062|bridge|WARN|could not open network device d3a9848478c4_h (No such device)
2023-12-13T03:08:45.271Z|115063|bridge|INFO|bridge br-int: added interface 88df172fe48c_h on port 7671
2023-12-13T03:08:45.307Z|115064|bridge|WARN|could not open network device 0ef94202ed4c_h (No such device)
2023-12-13T03:08:45.312Z|115065|bridge|WARN|could not open network device 6245e67c7126_h (No such device)
2023-12-13T03:08:45.316Z|115066|bridge|WARN|could not open network device 77004835159f_h (No such device)
2023-12-13T03:08:45.322Z|115067|bridge|WARN|could not open network device a0d65ea736cb_h (No such device)
2023-12-13T03:08:45.322Z|115068|bridge|INFO|bridge br-int: added interface eddd70d71ad6_h on port 7672
2023-12-13T03:08:45.327Z|115069|bridge|WARN|could not open network device 14909ba70381_h (No such device)
2023-12-13T03:08:45.329Z|115070|bridge|WARN|could not open network device 60e1d4b46a21_h (No such device)
2023-12-13T03:08:45.333Z|115071|bridge|WARN|could not open network device aa6c97210d57_h (No such device)
2023-12-13T03:08:45.337Z|115072|bridge|WARN|could not open network device bc4d4b06afc9_h (No such device)
2023-12-13T03:08:45.338Z|115073|bridge|INFO|bridge br-int: added interface 7fc34d8a7224_h on port 7673
2023-12-13T03:08:45.341Z|115074|bridge|WARN|could not open network device 8456430e2246_h (No such device)
2023-12-13T03:08:45.345Z|115075|bridge|WARN|could not open network device 8e1f8918e3d2_h (No such device)
2023-12-13T03:08:45.349Z|115076|bridge|WARN|could not open network device ffd13cb3c95f_h (No such device)
2023-12-13T03:08:45.350Z|115077|bridge|INFO|bridge br-int: added interface bdbf09c022c2_h on port 7674
2023-12-13T03:08:45.353Z|115078|bridge|WARN|could not open network device b22b1a7b1886_h (No such device)
2023-12-13T03:08:45.356Z|115079|bridge|WARN|could not open network device cc37efbecc00_h (No such device)
2023-12-13T03:08:45.359Z|115080|bridge|WARN|could not open network device 14c65a670a52_h (No such device)
2023-12-13T03:08:45.362Z|115081|bridge|WARN|could not open network device cf6cab8e626b_h (No such device)
2023-12-13T03:08:45.365Z|115082|rconn|WARN|br-int<->unix#3840: connection dropped (Broken pipe)
2023-12-13T03:08:46.421Z|18854|ovs_rcu(urcu8)|WARN|blocked 1000 ms waiting for main to quiesce
2023-12-13T03:08:47.421Z|18855|ovs_rcu(urcu8)|WARN|blocked 2000 ms waiting for main to quiesce
2023-12-13T03:08:49.421Z|18856|ovs_rcu(urcu8)|WARN|blocked 4001 ms waiting for main to quiesce
2023-12-13T03:08:53.420Z|18857|ovs_rcu(urcu8)|WARN|blocked 8000 ms waiting for main to quiesce
2023-12-13T03:09:01.422Z|18858|ovs_rcu(urcu8)|WARN|blocked 16001 ms waiting for main to quiesce

ovn-controller logs:

2023-12-13T02:59:17.005Z|09353|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connected
2023-12-13T02:59:27.014Z|09354|rconn|ERR|unix:/var/run/openvswitch/br-int.mgmt: no response to inactivity probe after 5 seconds, disconnecting
2023-12-13T02:59:28.014Z|09355|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connecting...
2023-12-13T02:59:29.015Z|09356|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connection timed out
2023-12-13T02:59:29.015Z|09357|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: waiting 2 seconds before reconnect
2023-12-13T02:59:31.015Z|09358|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connecting...
2023-12-13T02:59:33.016Z|09359|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connection timed out
2023-12-13T02:59:33.016Z|09360|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: waiting 4 seconds before reconnect
2023-12-13T02:59:37.017Z|09361|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connecting...
2023-12-13T02:59:41.018Z|09362|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connection timed out
2023-12-13T02:59:41.018Z|09363|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: continuing to retry connections in the background but suppressing further logging
2023-12-13T03:01:05.401Z|09364|lflow_cache|INFO|Detected cache inactivity (last active 30060 ms ago): trimming cache
2023-12-13T03:01:10.537Z|09365|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connected
2023-12-13T03:01:20.538Z|09366|rconn|ERR|unix:/var/run/openvswitch/br-int.mgmt: no response to inactivity probe after 5 seconds, disconnecting
2023-12-13T03:01:21.538Z|09367|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connecting...
2023-12-13T03:01:22.543Z|09368|rconn|INFO|unix:/var/run/openvswitch/br-int.mgmt: connection timed out

cpu usage:

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
1987678 root      10 -10  838020 166416   6184 R  48.5   1.0   2:04.62 ovs-vswitchd
1997569 root      10 -10  714584 464556   6972 R  48.5   2.8   6:51.72 ovn-controller

WHAT

馃[deprecated] Generated by Copilot at 6b9c85a

Increased cpu limit for kube-ovn-cni container in charts/values.yaml to enhance CNI performance and stability.

馃[deprecated] Generated by Copilot at 6b9c85a

kube-ovn-cni needs
more cpu cores to attach
pods in autumn wind

HOW

馃[deprecated] Generated by Copilot at 6b9c85a

  • Increase the cpu limit for the kube-ovn-cni container to 2 cores (link). This improves the performance and stability of the CNI plugin, which attaches pods to the OVN network. The change is based on user feedback and OVN documentation recommendations.

@zhangzujian zhangzujian added bug Something isn't working need backport labels Dec 15, 2023
dist/images/install.sh Outdated Show resolved Hide resolved
@zhangzujian zhangzujian marked this pull request as draft December 15, 2023 10:04
@zhangzujian zhangzujian deleted the fix-res-limit branch December 15, 2023 10:57
@zhangzujian zhangzujian restored the fix-res-limit branch January 4, 2024 02:33
@zhangzujian zhangzujian reopened this Jan 4, 2024
Signed-off-by: zhangzujian <zhangzujian.7@gmail.com>
@zhangzujian zhangzujian marked this pull request as ready for review January 4, 2024 02:34
@zhangzujian zhangzujian merged commit 83ca99d into kubeovn:master Jan 4, 2024
15 checks passed
@zhangzujian zhangzujian deleted the fix-res-limit branch January 4, 2024 02:35
zhangzujian added a commit that referenced this pull request Jan 4, 2024
Signed-off-by: zhangzujian <zhangzujian.7@gmail.com>
zhangzujian added a commit that referenced this pull request Jan 4, 2024
Signed-off-by: zhangzujian <zhangzujian.7@gmail.com>
bobz965 pushed a commit that referenced this pull request Jan 5, 2024
Signed-off-by: zhangzujian <zhangzujian.7@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working need backport
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants