Skip to content

fix #51135 make CFS quota period configurable #63437

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
k8s-github-robot merged 1 commit into from
Sep 1, 2018
Merged

fix #51135 make CFS quota period configurable #63437

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cmd/kubeadm/app/util/config/testdata/conversion/master/internal.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ ComponentConfigs:
CacheAuthorizedTTL: 5m0s
CacheUnauthorizedTTL: 30s
CPUCFSQuota: true
CPUCFSQuotaPeriod: 0s
CPUManagerPolicy: none
CPUManagerReconcilePeriod: 10s
CgroupDriver: cgroupfs
Expand Down
1 change: 1 addition & 0 deletions cmd/kubeadm/app/util/config/testdata/conversion/master/v1alpha2.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ kubeletConfiguration:
containerLogMaxSize: 10Mi
contentType: application/vnd.kubernetes.protobuf
cpuCFSQuota: true
cpuCFSQuotaPeriod: 0s
cpuManagerPolicy: none
cpuManagerReconcilePeriod: 10s
enableControllerAttachDetach: true
Expand Down
1 change: 1 addition & 0 deletions cmd/kubeadm/app/util/config/testdata/conversion/master/v1alpha3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ containerLogMaxFiles: 5
containerLogMaxSize: 10Mi
contentType: application/vnd.kubernetes.protobuf
cpuCFSQuota: true
cpuCFSQuotaPeriod: 0s
cpuManagerPolicy: none
cpuManagerReconcilePeriod: 10s
enableControllerAttachDetach: true
Expand Down
1 change: 1 addition & 0 deletions cmd/kubeadm/app/util/config/testdata/defaulting/master/defaulted.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ containerLogMaxFiles: 5
containerLogMaxSize: 10Mi
contentType: application/vnd.kubernetes.protobuf
cpuCFSQuota: true
cpuCFSQuotaPeriod: 0s
cpuManagerPolicy: none
cpuManagerReconcilePeriod: 10s
enableControllerAttachDetach: true
Expand Down
1 change: 1 addition & 0 deletions cmd/kubelet/app/options/options.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -530,6 +530,7 @@ func AddKubeletConfigFlags(mainfs *pflag.FlagSet, c *kubeletconfig.KubeletConfig

fs.StringVar(&c.ResolverConfig, "resolv-conf", c.ResolverConfig, "Resolver configuration file used as the basis for the container DNS resolution configuration.")
fs.BoolVar(&c.CPUCFSQuota, "cpu-cfs-quota", c.CPUCFSQuota, "Enable CPU CFS quota enforcement for containers that specify CPU limits")
fs.DurationVar(&c.CPUCFSQuotaPeriod.Duration, "cpu-cfs-quota-period", c.CPUCFSQuotaPeriod.Duration, "Sets CPU CFS quota period value, cpu.cfs_period_us, defaults to Linux Kernel default")

@feiskyer feiskyer Jun 22, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

defaults to Linux Kernel default (100ms) for clear?

@szuecs szuecs Jun 22, 2018

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the former comment from ligitt. Cobra will render the default automatically.

@dixudx dixudx Jul 4, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should stop adding new flags to kubelet. Add directly to KubeletConfiguration instead?

@derekwaynecarr derekwaynecarr Aug 21, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i would prefer to not have the flag and just use via KubeletConfiguration; we should note that dynamically changing the field value will not impact currently running pods (only new pods)

fs.BoolVar(&c.EnableControllerAttachDetach, "enable-controller-attach-detach", c.EnableControllerAttachDetach, "Enables the Attach/Detach controller to manage attachment/detachment of volumes scheduled to this node, and disables kubelet from executing any attach/detach operations")
fs.BoolVar(&c.MakeIPTablesUtilChains, "make-iptables-util-chains", c.MakeIPTablesUtilChains, "If true, kubelet will ensure iptables utility rules are present on host.")
fs.Int32Var(&c.IPTablesMasqueradeBit, "iptables-masquerade-bit", c.IPTablesMasqueradeBit, "The bit of the fwmark space to mark packets for SNAT. Must be within the range [0, 31]. Please match this parameter with corresponding parameter in kube-proxy.")
Expand Down
1 change: 1 addition & 0 deletions cmd/kubelet/app/server.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -708,6 +708,7 @@ func run(s *options.KubeletServer, kubeDeps *kubelet.Dependencies, stopCh <-chan
ExperimentalCPUManagerReconcilePeriod: s.CPUManagerReconcilePeriod.Duration,
ExperimentalPodPidsLimit: s.PodPidsLimit,
EnforceCPULimits: s.CPUCFSQuota,
CPUCFSQuotaPeriod: s.CPUCFSQuotaPeriod.Duration,
},
s.FailSwapOn,
devicePluginEnabled,
Expand Down
7 changes: 7 additions & 0 deletions pkg/features/kube_features.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,12 @@ const (
// Alternative container-level CPU affinity policies.
CPUManager utilfeature.Feature = "CPUManager"

// owner: @szuecs
// alpha: v1.12
//
// Enable nodes to change CPUCFSQuotaPeriod
CPUCFSQuotaPeriod utilfeature.Feature = "CustomCPUCFSQuotaPeriod"

// owner: @derekwaynecarr
// beta: v1.10
//
Expand Down Expand Up @@ -408,6 +414,7 @@ var defaultKubernetesFeatureGates = map[utilfeature.Feature]utilfeature.FeatureS
ExpandInUsePersistentVolumes: {Default: false, PreRelease: utilfeature.Alpha},
AttachVolumeLimit: {Default: false, PreRelease: utilfeature.Alpha},
CPUManager: {Default: true, PreRelease: utilfeature.Beta},
CPUCFSQuotaPeriod: {Default: false, PreRelease: utilfeature.Alpha},
ServiceNodeExclusion: {Default: false, PreRelease: utilfeature.Alpha},
MountContainers: {Default: false, PreRelease: utilfeature.Alpha},
VolumeScheduling: {Default: true, PreRelease: utilfeature.Beta},
Expand Down
1 change: 1 addition & 0 deletions pkg/kubelet/apis/config/helpers_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,7 @@ var (
"Authorization.Webhook.CacheAuthorizedTTL.Duration",
"Authorization.Webhook.CacheUnauthorizedTTL.Duration",
"CPUCFSQuota",
"CPUCFSQuotaPeriod.Duration",
"CPUManagerPolicy",
"CPUManagerReconcilePeriod.Duration",
"QOSReserved[*]",
Expand Down
2 changes: 2 additions & 0 deletions pkg/kubelet/apis/config/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -220,6 +220,8 @@ type KubeletConfiguration struct {
// cpuCFSQuota enables CPU CFS quota enforcement for containers that
// specify CPU limits
CPUCFSQuota bool
// CPUCFSQuotaPeriod sets the CPU CFS quota period value, cpu.cfs_period_us, defaults to 100ms

@paskal paskal Jul 21, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm super confused about this one and can't figure this one out by myself: cpu.cfs_period_us default limit called "100ms" but actually is microseconds:

cpu.cfs_period_us: the length of a period (in microseconds)

In the change I'm commenting author clearly acted under assumptions that milliseconds are used in k8s and now it's all over the test cases and comments. But I wonder where that 1000x disperancy between k8s value and Linux value comes from? And does it exist, or k8s default is actually a microsecond, same as linux? Unless we multiple that default value of QuotaPeriod by 1000 somewhere, converted to time.Duration it's 100 microseconds and not milliseconds.

cc @vishh @sjenning and especially @hjacobs: I hope you all have some context about it and can tell me why I'm wrong and k8s code comments are right.

@szuecs szuecs Jul 22, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes cfs_period_us is in µs, that's why it has _us suffix
Maybe https://go.dev/play/p/65fG1OmLFYN helps to understand.
As far as I remember, I think the time.Duration is only used if you want to set it via kubelet flag/config.

@paskal paskal Jul 22, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment in that MR says "ms". In contrast, I think it better be "microseconds". I can send MR to change it as well as explicit millisecond mentions in the current code, whereas it should be microseconds. I'll ping you in it.

In this MR, you wrote milliseconds duration in multiple places in defaults and tests CPUCFSQuotaPeriod: metav1.Duration{Duration: 100 * time.Millisecond},, I wonder if that should be changed to microseconds to match the actual default value of 100 microseconds?

Also, the discussion which led to this MR has multiple people talking about changing the value from 100 milliseconds to 5/25 milliseconds for testing. Still, it doesn't make sense unless everyone were altering the source code in microseconds, thinking it's milliseconds. @hjacobs even did a talk about it talking about the default value as if it was in milliseconds. Your code introducing milliseconds and not microseconds (so 1000x) contributed to my confusion, and I wonder if it's intentional or if you are mistaken, and it should be changed microseconds to change the real default.

The only way current code would make sense to me is if the value you feed to k8s (100 milliseconds) is divided by 1000 at some point in time and then matches the real Linux default of 100 microseconds, but I haven't found such a place yet, and I would doubt that's the case.

@paskal paskal Jul 27, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@szuecs, the simplest form of the question I can form is the following.

Is the 100ms value of CPUCFSQuota result in the same k8s behaviour as the default Linux value of 100 microseconds, or is it 1000 times higher as its milliseconds versus microseconds in Linux?

@paskal paskal Aug 31, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wrong, and the original code was correct. 100ms is indeed a default value in Linux, and I got confused by conversions in two places in code that had no comments about them. PR #112123 adds comments to the code added here.

CPUCFSQuotaPeriod metav1.Duration
// maxOpenFiles is Number of files that can be opened by Kubelet process.
MaxOpenFiles int64
// contentType is contentType of requests sent to apiserver.
Expand Down
1 change: 1 addition & 0 deletions pkg/kubelet/apis/config/v1beta1/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ go_library(
],
importpath = "k8s.io/kubernetes/pkg/kubelet/apis/config/v1beta1",
deps = [
"//pkg/features:go_default_library",
"//pkg/kubelet/apis/config:go_default_library",
"//pkg/kubelet/qos:go_default_library",
"//pkg/kubelet/types:go_default_library",
Expand Down
4 changes: 4 additions & 0 deletions pkg/kubelet/apis/config/v1beta1/defaults.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import (

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
kruntime "k8s.io/apimachinery/pkg/runtime"
"k8s.io/kubernetes/pkg/features"
"k8s.io/kubernetes/pkg/kubelet/qos"
kubetypes "k8s.io/kubernetes/pkg/kubelet/types"
"k8s.io/kubernetes/pkg/master/ports"
Expand Down Expand Up @@ -154,6 +155,9 @@ func SetDefaults_KubeletConfiguration(obj *KubeletConfiguration) {
if obj.CPUCFSQuota == nil {
obj.CPUCFSQuota = utilpointer.BoolPtr(true)
}
if obj.CPUCFSQuotaPeriod == nil && obj.FeatureGates[string(features.CPUCFSQuotaPeriod)] {

@dashpole dashpole Sep 6, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if this actually works... We generally do not feature gate the defaulting of flags. We generally only place the feature gate around code we don't want to run, rather than around configuration.

@derekwaynecarr derekwaynecarr Sep 6, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does look like a problem. The flag should always default, it just shouldn’t have been used. Apologies for missing in review

obj.CPUCFSQuotaPeriod = &metav1.Duration{Duration: 100 * time.Millisecond}
}
if obj.MaxOpenFiles == 0 {
obj.MaxOpenFiles = 1000000
}
Expand Down
7 changes: 7 additions & 0 deletions pkg/kubelet/apis/config/v1beta1/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -466,6 +466,13 @@ type KubeletConfiguration struct {
// Default: true
// +optional
CPUCFSQuota *bool `json:"cpuCFSQuota,omitempty"`
// CPUCFSQuotaPeriod is the CPU CFS quota period value, cpu.cfs_period_us.
// Dynamic Kubelet Config (beta): If dynamically updating this field, consider that
// limits set for containers will result in different cpu.cfs_quota settings. This
// will trigger container restarts on the node being reconfigured.
// Default: "100ms"
// +optional
CPUCFSQuotaPeriod *metav1.Duration `json:"cpuCFSQuotaPeriod,omitempty"`
// maxOpenFiles is Number of files that can be opened by Kubelet process.
// Dynamic Kubelet Config (beta): If dynamically updating this field, consider that
// it may impact the ability of the Kubelet to interact with the node's filesystem.
Expand Down
6 changes: 6 additions & 0 deletions pkg/kubelet/apis/config/v1beta1/zz_generated.conversion.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions pkg/kubelet/apis/config/v1beta1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions pkg/kubelet/apis/config/validation/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ go_test(
embed = [":go_default_library"],
deps = [
"//pkg/kubelet/apis/config:go_default_library",
"//staging/src/k8s.io/apimachinery/pkg/apis/meta/v1:go_default_library",
"//staging/src/k8s.io/apimachinery/pkg/util/errors:go_default_library",
],
)
4 changes: 4 additions & 0 deletions pkg/kubelet/apis/config/validation/validation.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ package validation

import (
"fmt"
"time"

utilerrors "k8s.io/apimachinery/pkg/util/errors"
utilvalidation "k8s.io/apimachinery/pkg/util/validation"
Expand Down Expand Up @@ -54,6 +55,9 @@ func ValidateKubeletConfiguration(kc *kubeletconfig.KubeletConfiguration) error
if kc.HealthzPort != 0 && utilvalidation.IsValidPortNum(int(kc.HealthzPort)) != nil {
allErrors = append(allErrors, fmt.Errorf("invalid configuration: HealthzPort (--healthz-port) %v must be between 1 and 65535, inclusive", kc.HealthzPort))
}
if localFeatureGate.Enabled(features.CPUCFSQuotaPeriod) && utilvalidation.IsInRange(int(kc.CPUCFSQuotaPeriod.Duration), int(1*time.Microsecond), int(time.Second)) != nil {
allErrors = append(allErrors, fmt.Errorf("invalid configuration: CPUCFSQuotaPeriod (--cpu-cfs-quota-period) %v must be between 1usec and 1sec, inclusive", kc.CPUCFSQuotaPeriod))
}
if utilvalidation.IsInRange(int(kc.ImageGCHighThresholdPercent), 0, 100) != nil {
allErrors = append(allErrors, fmt.Errorf("invalid configuration: ImageGCHighThresholdPercent (--image-gc-high-threshold) %v must be between 0 and 100, inclusive", kc.ImageGCHighThresholdPercent))
}
Expand Down
4 changes: 4 additions & 0 deletions pkg/kubelet/apis/config/validation/validation_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,9 @@ package validation

import (
"testing"
"time"

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
utilerrors "k8s.io/apimachinery/pkg/util/errors"
kubeletconfig "k8s.io/kubernetes/pkg/kubelet/apis/config"
)
Expand Down Expand Up @@ -48,6 +50,7 @@ func TestValidateKubeletConfiguration(t *testing.T) {
RegistryPullQPS: 5,
HairpinMode: kubeletconfig.PromiscuousBridge,
NodeLeaseDurationSeconds: 1,
CPUCFSQuotaPeriod: metav1.Duration{Duration: 100 * time.Millisecond},
}
if allErrors := ValidateKubeletConfiguration(successCase); allErrors != nil {
t.Errorf("expect no errors, got %v", allErrors)
Expand Down Expand Up @@ -77,6 +80,7 @@ func TestValidateKubeletConfiguration(t *testing.T) {
RegistryPullQPS: -10,
HairpinMode: "foo",
NodeLeaseDurationSeconds: -1,
CPUCFSQuotaPeriod: metav1.Duration{Duration: 0},
}
const numErrs = 23
if allErrors := ValidateKubeletConfiguration(errorCase); len(allErrors.(utilerrors.Aggregate).Errors()) != numErrs {
Expand Down
1 change: 1 addition & 0 deletions pkg/kubelet/apis/config/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions pkg/kubelet/cm/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,11 +134,14 @@ go_test(
embed = [":go_default_library"],
deps = select({
"@io_bazel_rules_go//go/platform:linux": [
"//pkg/features:go_default_library",
"//pkg/kubelet/eviction/api:go_default_library",
"//pkg/util/mount:go_default_library",
"//staging/src/k8s.io/api/core/v1:go_default_library",
"//staging/src/k8s.io/apimachinery/pkg/api/resource:go_default_library",
"//staging/src/k8s.io/apimachinery/pkg/types:go_default_library",
"//staging/src/k8s.io/apiserver/pkg/util/feature:go_default_library",
"//staging/src/k8s.io/apiserver/pkg/util/feature/testing:go_default_library",
"//vendor/github.com/stretchr/testify/assert:go_default_library",
"//vendor/github.com/stretchr/testify/require:go_default_library",
],
Expand Down
1 change: 1 addition & 0 deletions pkg/kubelet/cm/container_manager.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ type NodeConfig struct {
ExperimentalCPUManagerReconcilePeriod time.Duration
ExperimentalPodPidsLimit int64
EnforceCPULimits bool
CPUCFSQuotaPeriod time.Duration
}

type NodeAllocatableConfig struct {
Expand Down
1 change: 1 addition & 0 deletions pkg/kubelet/cm/container_manager_linux.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,7 @@ func (cm *containerManagerImpl) NewPodContainerManager() PodContainerManager {
cgroupManager: cm.cgroupManager,
podPidsLimit: cm.ExperimentalPodPidsLimit,
enforceCPULimits: cm.EnforceCPULimits,
cpuCFSQuotaPeriod: uint64(cm.CPUCFSQuotaPeriod / time.Microsecond),
}
}
return &podContainerManagerNoop{
Expand Down
19 changes: 11 additions & 8 deletions pkg/kubelet/cm/helpers_linux.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,11 @@ import (

"k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/types"
utilfeature "k8s.io/apiserver/pkg/util/feature"
"k8s.io/kubernetes/pkg/api/v1/resource"
v1helper "k8s.io/kubernetes/pkg/apis/core/v1/helper"
v1qos "k8s.io/kubernetes/pkg/apis/core/v1/helper/qos"
kubefeatures "k8s.io/kubernetes/pkg/features"
)

const (
Expand All @@ -44,28 +46,29 @@ const (
)

// MilliCPUToQuota converts milliCPU to CFS quota and period values.
func MilliCPUToQuota(milliCPU int64) (quota int64, period uint64) {
func MilliCPUToQuota(milliCPU int64, period int64) (quota int64) {
// CFS quota is measured in two values:
// - cfs_period_us=100ms (the amount of time to measure usage across)
// - cfs_period_us=100ms (the amount of time to measure usage across given by period)
// - cfs_quota=20ms (the amount of cpu time allowed to be used across a period)
// so in the above example, you are limited to 20% of a single CPU
// for multi-cpu environments, you just scale equivalent amounts
// see https://www.kernel.org/doc/Documentation/scheduler/sched-bwc.txt for details

if milliCPU == 0 {
return
}

// we set the period to 100ms by default
period = QuotaPeriod
if !utilfeature.DefaultFeatureGate.Enabled(kubefeatures.CPUCFSQuotaPeriod) {
period = QuotaPeriod
}

// we then convert your milliCPU to a value normalized over a period
quota = (milliCPU * QuotaPeriod) / MilliCPUToCPU
quota = (milliCPU * period) / MilliCPUToCPU

// quota needs to be a minimum of 1ms.
if quota < MinQuotaPeriod {
quota = MinQuotaPeriod
}

return
}

Expand Down Expand Up @@ -103,7 +106,7 @@ func HugePageLimits(resourceList v1.ResourceList) map[int64]int64 {
}

// ResourceConfigForPod takes the input pod and outputs the cgroup resource config.
func ResourceConfigForPod(pod *v1.Pod, enforceCPULimits bool) *ResourceConfig {
func ResourceConfigForPod(pod *v1.Pod, enforceCPULimits bool, cpuPeriod uint64) *ResourceConfig {
// sum requests and limits.
reqs, limits := resource.PodRequestsAndLimits(pod)

Expand All @@ -122,7 +125,7 @@ func ResourceConfigForPod(pod *v1.Pod, enforceCPULimits bool) *ResourceConfig {

// convert to CFS values
cpuShares := MilliCPUToShares(cpuRequests)
cpuQuota, cpuPeriod := MilliCPUToQuota(cpuLimits)
cpuQuota := MilliCPUToQuota(cpuLimits, int64(cpuPeriod))

// track if limits were applied for each resource.
memoryLimitsDeclared := true
Expand Down
Loading