Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cleanup: Refactor BaseEndpointInfo to cache IP and Port values #120104

Merged
merged 1 commit into from Oct 28, 2023

Conversation

togettoyou
Copy link
Contributor

@togettoyou togettoyou commented Aug 22, 2023

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

This commit refactors the BaseEndpointInfo struct to add two new fields: ip and port. These fields are used to cache the IP and Port values to improve performance.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Aug 22, 2023
@k8s-ci-robot
Copy link
Contributor

Hi @togettoyou. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. area/ipvs area/kube-proxy sig/network Categorizes an issue or PR as relevant to SIG Network. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Aug 22, 2023
@@ -132,7 +132,7 @@ type endpointsInfo struct {
func newEndpointInfo(baseInfo *proxy.BaseEndpointInfo, svcPortName *proxy.ServicePortName) proxy.Endpoint {
return &endpointsInfo{
BaseEndpointInfo: baseInfo,
ChainName: servicePortEndpointChainName(svcPortName.String(), strings.ToLower(string(svcPortName.Protocol)), baseInfo.Endpoint),
ChainName: servicePortEndpointChainName(svcPortName.String(), strings.ToLower(string(svcPortName.Protocol)), baseInfo.Endpoint.String()),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have doubts this improves performance, since we need to concatenate IP port in each of these calls

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, currently the endpointString struct only stores the IP and Port, so the proxier doesn't need to perform additional parsing to extract them. However, for the String method, concatenation is needed.

I can add an endpoint field to the endpointString struct:

type endpointString struct {
	IP       string
	Port     int
	endpoint string
}

func (e *endpointString) String() string {
	if e.endpoint == "" {
		e.endpoint = net.JoinHostPort(e.IP, strconv.Itoa(e.Port))
	}
	return e.endpoint
}

What do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just saw you claimed the performance improvement, and I was wondering how we can know if we regress or improve, without that information is hard to evaluate if this change is necessary

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/kubernetes/kubernetes/blob/master/pkg/proxy/iptables/proxier.go#L1371

https://github.com/kubernetes/kubernetes/blob/master/pkg/proxy/ipvs/proxier.go#L1051

At least here, there is no need to repeatedly extract the Endpoint string when calling the IP() and Port() methods.

Do you have any suggestions for performance improvement testing?

@togettoyou togettoyou requested a review from aojea August 22, 2023 12:10
@togettoyou
Copy link
Contributor Author

When the Endpoint is of type string :

func BenchmarkEndpointIP(b *testing.B) {
	info := &BaseEndpointInfo{
		Endpoint: "1.1.1.1:11",
	}

	for i := 0; i < b.N; i++ {
		_ = info.IP()
	}
}

func BenchmarkEndpointPort(b *testing.B) {
	info := &BaseEndpointInfo{
		Endpoint: "1.1.1.1:11",
	}

	for i := 0; i < b.N; i++ {
		_, _ = info.Port()
	}
}

func BenchmarkEndpointString(b *testing.B) {
	info := &BaseEndpointInfo{
		Endpoint: "1.1.1.1:11",
	}

	for i := 0; i < b.N; i++ {
		_ = info.String()
	}
}
goos: windows
goarch: amd64
pkg: k8s.io/kubernetes/pkg/proxy
cpu: 11th Gen Intel(R) Core(TM) i5-11300H @ 3.10GHz

BenchmarkEndpointIP
BenchmarkEndpointIP-8            6057644               189.8 ns/op

BenchmarkEndpointPort
BenchmarkEndpointPort-8         39769996                28.53 ns/op

BenchmarkEndpointString
BenchmarkEndpointString-8       1000000000               0.4417 ns/op

When the Endpoint is of type endpointString :

func BenchmarkEndpointIP(b *testing.B) {
	info := &BaseEndpointInfo{
		Endpoint: endpointString{
			IP:   "1.1.1.1",
			Port: 11,
		},
	}

	for i := 0; i < b.N; i++ {
		_ = info.IP()
	}
}

func BenchmarkEndpointPort(b *testing.B) {
	info := &BaseEndpointInfo{
		Endpoint: endpointString{
			IP:   "1.1.1.1",
			Port: 11,
		},
	}

	for i := 0; i < b.N; i++ {
		_, _ = info.Port()
	}
}

func BenchmarkEndpointString(b *testing.B) {
	info := &BaseEndpointInfo{
		Endpoint: endpointString{
			IP:   "1.1.1.1",
			Port: 11,
		},
	}

	for i := 0; i < b.N; i++ {
		_ = info.String()
	}
}
goos: windows
goarch: amd64
pkg: k8s.io/kubernetes/pkg/proxy
cpu: 11th Gen Intel(R) Core(TM) i5-11300H @ 3.10GHz

BenchmarkEndpointIP
BenchmarkEndpointIP-8           1000000000               0.2925 ns/op

BenchmarkEndpointPort
BenchmarkEndpointPort-8         1000000000               0.2472 ns/op

BenchmarkEndpointString
BenchmarkEndpointString-8       673348149                1.834 ns/op

@togettoyou
Copy link
Contributor Author

Hey @aojea , Can this be used as an evaluation method?

endpoint string
}

func NewEndpointString(s string) endpointString {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, the NewEndpointString method is mainly used in *_test.go files, and I am not sure if it needs to return an err

@aojea
Copy link
Member

aojea commented Aug 23, 2023

ok, so both epInfo.IP( )and epInfo.String() are called in the same loop, so we can assume both are going to be called the same number of times

for _, ep := range allLocallyReachableEndpoints {
epInfo, ok := ep.(*endpointsInfo)
if !ok {
klog.ErrorS(nil, "Failed to cast endpointsInfo", "endpointsInfo", ep)
continue
}
endpointChain := epInfo.ChainName
// Create the endpoint chain
natChains.Write(utiliptables.MakeChainLine(endpointChain))
activeNATChains[endpointChain] = true
args = append(args[:0], "-A", string(endpointChain))
args = proxier.appendServiceCommentLocked(args, svcPortNameString)
// Handle traffic that loops back to the originator with SNAT.
natRules.Write(
args,
"-s", epInfo.IP(),
"-j", string(kubeMarkMasqChain))
// Update client-affinity lists.
if svcInfo.SessionAffinityType() == v1.ServiceAffinityClientIP {
args = append(args, "-m", "recent", "--name", string(endpointChain), "--set")
}
// DNAT to final destination.
args = append(args, "-m", protocol, "-p", protocol, "-j", "DNAT", "--to-destination", epInfo.Endpoint)

The benchmarks shows something that is expected, concatenation is faster than splitting

BenchmarkEndpointIP-8 6057644 189.8 ns/op

BenchmarkEndpointString-8 673348149 1.834 ns/op

There is one solution for these problems, that is sacrificing memory to cache the value instead of generating it each time

I do not know if we need to complicate this further creating a new type, and just adding the values to the BaseEndpointInfoshould be enough

diff --git a/pkg/proxy/endpoints.go b/pkg/proxy/endpoints.go
index fd1e9485cdb..5ab519ca988 100644
--- a/pkg/proxy/endpoints.go
+++ b/pkg/proxy/endpoints.go
@@ -30,7 +30,6 @@ import (
        "k8s.io/apimachinery/pkg/types"
        "k8s.io/apimachinery/pkg/util/sets"
        "k8s.io/kubernetes/pkg/proxy/metrics"
-       proxyutil "k8s.io/kubernetes/pkg/proxy/util"
 )
 
 var supportedEndpointSliceAddressTypes = sets.New[string](
@@ -43,7 +42,11 @@ var supportedEndpointSliceAddressTypes = sets.New[string](
 // or can be used for constructing a more specific EndpointInfo struct
 // defined by the proxier if needed.
 type BaseEndpointInfo struct {
-       Endpoint string // TODO: should be an endpointString type
+       // Cache this values to improve performance
+       ip   string
+       port int
+       // Endpoint is the same as net.JoinHostPort(ip,port)
+       Endpoint string
        // IsLocal indicates whether the endpoint is running in same host as kube-proxy.
        IsLocal bool
 
@@ -109,12 +112,12 @@ func (info *BaseEndpointInfo) GetZoneHints() sets.Set[string] {
 
 // IP returns just the IP part of the endpoint, it's a part of proxy.Endpoint interface.
 func (info *BaseEndpointInfo) IP() string {
-       return proxyutil.IPPart(info.Endpoint)
+       return info.ip
 }
 
 // Port returns just the Port part of the endpoint.
 func (info *BaseEndpointInfo) Port() (int, error) {
-       return proxyutil.PortPart(info.Endpoint)
+       return info.port, nil
 }
 
 // Equal is part of proxy.Endpoint interface.
@@ -137,6 +140,8 @@ func (info *BaseEndpointInfo) GetZone() string {
 func newBaseEndpointInfo(IP, nodeName, zone string, port int, isLocal bool,
        ready, serving, terminating bool, zoneHints sets.Set[string]) *BaseEndpointInfo {
        return &BaseEndpointInfo{
+               ip:          IP,
+               port:        port,
                Endpoint:    net.JoinHostPort(IP, strconv.Itoa(port)),
                IsLocal:     isLocal,
                Ready:       ready,

@togettoyou
Copy link
Contributor Author

Thank you @aojea , It seems that your changes have the smallest impact, but I'm not sure which one is better. Does anyone else have any suggestions?

Also, I found that before I added the endpoint string field to the endpointString struct , endpoints_test.go was working fine.

But after adding it in this commit (0891fe2), it stopped working. I tried your version, but the same problem occurred.

"removing port in update": {
endpointChangeTracker: NewEndpointChangeTracker("", nil, v1.IPv4Protocol, nil, nil),
expectedChanges: []*endpointsChange{{
previous: EndpointsMap{
svcPortName0: []Endpoint{newTestEp("10.0.1.1:80", "host1", true, true, false), newTestEp("10.0.1.2:80", "host1", true, true, false), newTestEp("10.0.1.3:80", "host1", false, false, false)},
svcPortName1: []Endpoint{newTestEp("10.0.1.1:443", "host1", true, true, false), newTestEp("10.0.1.2:443", "host1", true, true, false), newTestEp("10.0.1.3:443", "host1", false, false, false)},
},
current: EndpointsMap{
svcPortName0: []Endpoint{newTestEp("10.0.1.1:80", "host1", true, true, false), newTestEp("10.0.1.2:80", "host1", true, true, false), newTestEp("10.0.1.3:80", "host1", false, false, false)},
},
}},
appliedSlices: []*discovery.EndpointSlice{
generateEndpointSlice("svc1", "ns1", 1, 3, 3, 999, []string{"host1"}, []*int32{pointer.Int32(80), pointer.Int32(443)}),
},
pendingSlices: []*discovery.EndpointSlice{
generateEndpointSlice("svc1", "ns1", 1, 3, 3, 999, []string{"host1"}, []*int32{pointer.Int32(80)}),
},
},

=== RUN   TestCheckoutChanges
--- FAIL: TestCheckoutChanges (0.00s)
=== RUN   TestCheckoutChanges/removing_port_in_update
    endpoints_test.go:1631: [0] Expected change.previous: map[ns1/svc1:port-0:[10.0.1.1:80 10.0.1.2:80 10.0.1.3:80] ns1/svc1:port-1:[10.0.1.1:443 10.0.1.2:443 10.0.1.3:443]], got: map[ns1/svc1:port-0:[10.0.1.1:80 10.0.1.2:80 10.0.1.3:80] ns1/svc1:port-1:[10.0.1.1:443 10.0.1.2:443 10.0.1.3:443]]
    endpoints_test.go:1635: [0] Expected change.current: map[ns1/svc1:port-0:[10.0.1.1:80 10.0.1.2:80 10.0.1.3:80]], got: map[ns1/svc1:port-0:[10.0.1.1:80 10.0.1.2:80 10.0.1.3:80]]
    --- FAIL: TestCheckoutChanges/removing_port_in_update (0.00s)

What could be the reason for this? You can try your modified version as well, it should have the same problem.

@togettoyou
Copy link
Contributor Author

Thank you @aojea , It seems that your changes have the smallest impact, but I'm not sure which one is better. Does anyone else have any suggestions?

Also, I found that before I added the endpoint string field to the endpointString struct , endpoints_test.go was working fine.

But after adding it in this commit (0891fe2), it stopped working. I tried your version, but the same problem occurred.

"removing port in update": {
endpointChangeTracker: NewEndpointChangeTracker("", nil, v1.IPv4Protocol, nil, nil),
expectedChanges: []*endpointsChange{{
previous: EndpointsMap{
svcPortName0: []Endpoint{newTestEp("10.0.1.1:80", "host1", true, true, false), newTestEp("10.0.1.2:80", "host1", true, true, false), newTestEp("10.0.1.3:80", "host1", false, false, false)},
svcPortName1: []Endpoint{newTestEp("10.0.1.1:443", "host1", true, true, false), newTestEp("10.0.1.2:443", "host1", true, true, false), newTestEp("10.0.1.3:443", "host1", false, false, false)},
},
current: EndpointsMap{
svcPortName0: []Endpoint{newTestEp("10.0.1.1:80", "host1", true, true, false), newTestEp("10.0.1.2:80", "host1", true, true, false), newTestEp("10.0.1.3:80", "host1", false, false, false)},
},
}},
appliedSlices: []*discovery.EndpointSlice{
generateEndpointSlice("svc1", "ns1", 1, 3, 3, 999, []string{"host1"}, []*int32{pointer.Int32(80), pointer.Int32(443)}),
},
pendingSlices: []*discovery.EndpointSlice{
generateEndpointSlice("svc1", "ns1", 1, 3, 3, 999, []string{"host1"}, []*int32{pointer.Int32(80)}),
},
},

=== RUN   TestCheckoutChanges
--- FAIL: TestCheckoutChanges (0.00s)
=== RUN   TestCheckoutChanges/removing_port_in_update
    endpoints_test.go:1631: [0] Expected change.previous: map[ns1/svc1:port-0:[10.0.1.1:80 10.0.1.2:80 10.0.1.3:80] ns1/svc1:port-1:[10.0.1.1:443 10.0.1.2:443 10.0.1.3:443]], got: map[ns1/svc1:port-0:[10.0.1.1:80 10.0.1.2:80 10.0.1.3:80] ns1/svc1:port-1:[10.0.1.1:443 10.0.1.2:443 10.0.1.3:443]]
    endpoints_test.go:1635: [0] Expected change.current: map[ns1/svc1:port-0:[10.0.1.1:80 10.0.1.2:80 10.0.1.3:80]], got: map[ns1/svc1:port-0:[10.0.1.1:80 10.0.1.2:80 10.0.1.3:80]]
    --- FAIL: TestCheckoutChanges/removing_port_in_update (0.00s)

What could be the reason for this? You can try your modified version as well, it should have the same problem.

I have found the issue that caused the unit test to fail.

if !reflect.DeepEqual(change.current, expectedChange.current) {

It was because change.current called a endpointInfo.String() method, while expectedChange.current did not. This resulted in the endpoint value of endpointString being empty, leading to a discrepancy.

if _, exists := endpointSet[endpointInfo.String()]; !exists || isLocal {
endpointSet[endpointInfo.String()] = cache.makeEndpointInfo(endpointInfo, svcPortName)

Similarly, in your version, the issue was caused by expectedChange.current calling the newTestEp method without passing in the ip and port parameters. This resulted in a discrepancy when using reflect.DeepEqual for comparison.

func newTestEp(ep, host string, ready, serving, terminating bool) *BaseEndpointInfo {
endpointInfo := &BaseEndpointInfo{Endpoint: ep, Ready: ready, Serving: serving, Terminating: terminating}

I will fix my PR to ensure that the unit test can pass successfully.

@k8s-ci-robot k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Aug 24, 2023
@togettoyou
Copy link
Contributor Author

Hi @aojea , I have made a modification to the endpointString type. I believe that introducing the endpointString type will make it clearer than directly adding the ip and port fields.

The NewEndpointString method is mainly used for compatibility with unit tests.

Otherwise, many places would require importing the ip and port fields separately.

@togettoyou togettoyou force-pushed the cleanup_endpoints branch 2 times, most recently from f38f391 to 38931e8 Compare August 29, 2023 02:23
@danwinship
Copy link
Contributor

I do not know if we need to complicate this further creating a new type, and just adding the values to the BaseEndpointInfoshould be enough

Agreed. The type doesn't add much.

}

// Port returns just the Port part of the endpoint.
func (info *BaseEndpointInfo) Port() (int, error) {
return proxyutil.PortPart(info.Endpoint)
return info.Endpoint.port, nil
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's no need for this to still return an error, since it can't fail

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will require modifying the Endpoint interface

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now it should still return an err

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will require modifying the Endpoint interface

That's fine; these are all internal APIs. You can fix all of the consumers as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will require modifying the Endpoint interface

That's fine; these are all internal APIs. You can fix all of the consumers as well.

I have resubmitted it and now I need to keep returning err.

&BaseEndpointInfo{Endpoint: "10.1.2.4:80", ZoneHints: sets.New[string]("zone-b"), Ready: true},
&BaseEndpointInfo{Endpoint: "10.1.2.5:80", ZoneHints: sets.New[string]("zone-c"), Ready: true},
&BaseEndpointInfo{Endpoint: "10.1.2.6:80", ZoneHints: sets.New[string]("zone-a"), Ready: true},
&BaseEndpointInfo{Endpoint: NewEndpointString("10.1.2.3:80"), ZoneHints: sets.New[string]("zone-a"), Ready: true},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these can use newBaseEndpointInfo (or else some new helper function just in this file)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have made the modifications, and the changes are minimal, ensuring full compatibility with the previous unit tests.

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 13, 2023
@dims dims added the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Oct 24, 2023
@danwinship
Copy link
Contributor

ok, #121097 is merged. You should be able to rebase on top of that now...

@k8s-ci-robot k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 26, 2023
@togettoyou
Copy link
Contributor Author

ok, #121097 is merged. You should be able to rebase on top of that now...

ready ok-to-test

if err != nil {
portNumber = 0
}
portNumber := baseInfo.Port()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you could just do this inline below, now that you don't need to deal with an error (like it does with ip: baseInfo.IP())

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I overlooked it.

@danwinship
Copy link
Contributor

/ok-to-test
/remove-lifecycle rotten
/triage accepted
/lgtm
/approve

Looks good. Sorry for getting your nice simple improvement stuck behind my big refactoring, but I think the end result is better.

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Oct 27, 2023
@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 27, 2023
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: cd709d67fb76c8a13f10f4253bf91050fada0ec4

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: danwinship, togettoyou

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 27, 2023
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 27, 2023
@togettoyou
Copy link
Contributor Author

/retest

@danwinship
Copy link
Contributor

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 27, 2023
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: db403040068ab6167b01cbbd62dd4154622f1ad2

@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 28, 2023
@k8s-ci-robot k8s-ci-robot merged commit b8693aa into kubernetes:master Oct 28, 2023
15 checks passed
@k8s-ci-robot k8s-ci-robot added this to the v1.29 milestone Oct 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/ipvs area/kube-proxy cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note-none Denotes a PR that doesn't merit a release note. sig/network Categorizes an issue or PR as relevant to SIG Network. sig/windows Categorizes an issue or PR as relevant to SIG Windows. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

5 participants