Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimize proxier duplicate localaddrset #98083

Conversation

JornShen
Copy link
Member

@JornShen JornShen commented Jan 15, 2021

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

syncProxyRules in proxier.go of iptables and ipvs, there is a same code.

	localAddrs, err := utilproxy.GetLocalAddrs()
	if err != nil {
		klog.ErrorS(err, "Failed to get local addresses during proxy sync, assuming external IPs are not local")
	} else if len(localAddrs) == 0 {
		klog.InfoS("No local addresses found, assuming all external IPs are not local")
	}

	localAddrSet := utilnet.IPSet{}
	localAddrSet.Insert(localAddrs...)

At the same time, this code is in the front of syncProxyRules. Every time we should run it. But in fact, the count result localAddrSet is only use in Capture externalIPs

if (svcInfo.Protocol() != v1.ProtocolSCTP) && localAddrSet.Has(net.ParseIP(externalIP)) {

So I migrate it to the utils module.

Does this PR introduce a user-facing change?:

NONE

@k8s-ci-robot k8s-ci-robot added kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jan 15, 2021
@k8s-ci-robot
Copy link
Contributor

@JornShen: This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jan 15, 2021
@JornShen
Copy link
Member Author

/cc @thockin @aojea

@k8s-ci-robot k8s-ci-robot added area/ipvs sig/network Categorizes an issue or PR as relevant to SIG Network. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jan 15, 2021
@JornShen
Copy link
Member Author

pull-kubernetes-e2e-kind-ipv6 failed job is Conntrack should be able to preserve UDP traffic when server pod cycles for a NodePort service

ref issue #91236

@JornShen
Copy link
Member Author

/test pull-kubernetes-e2e-kind-ipv6

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Jan 18, 2021
@aojea
Copy link
Member

aojea commented Jan 18, 2021

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 18, 2021
@JornShen
Copy link
Member Author

/cc @thockin PTAL :)

@JornShen JornShen force-pushed the optimize_proxier_duplicate_localaddrset branch from e23af44 to 3f506ca Compare January 29, 2021 02:53
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 29, 2021
@JornShen
Copy link
Member Author

updated
PTAL :)

/cc @aojea @thockin

Copy link
Member

@thockin thockin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

/lgtm
/approve

@@ -1007,6 +992,12 @@ func (proxier *Proxier) syncProxyRules() {
proxier.endpointChainsNumber += len(proxier.endpointsMap[svcName])
}

localAddrSet := utilproxy.GetLocalAddrSet()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you put these right next to each other, they seem to overlap a bit. We should clean this up. Later.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which place you think it is overlap? :)

// If failed to get local addr, will assume no local ips.
func GetLocalAddrSet() utilnet.IPSet {
localAddrs, err := GetLocalAddrs()
if err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm torn. Part of me feels like this should be logged from the caller, which reduces this function to just a few LOC. I guess it doesn't matter much, its an internal API..

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 29, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JornShen, thockin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 29, 2021
@thockin
Copy link
Member

thockin commented Jan 29, 2021 via email

@JornShen
Copy link
Member Author

failed job: [sig-network] Proxy version v1 should proxy logs on node with explicit kubelet port using proxy subresource
the log as fellow:

error trying to reach service: dial tcp 10.40.0.5:10250: i... (500; 30.042417499s)
Jan 29 03:16:45.212: FAIL: Unexpected error:
    <*errors.StatusError | 0xc002d54d20>: {
        ErrStatus: {
            TypeMeta: {Kind: "", APIVersion: ""},
            ListMeta: {
                SelfLink: "",
                ResourceVersion: "",
                Continue: "",
                RemainingItemCount: nil,
            },
            Status: "Failure",
            Message: "an error on the server (\"unknown\") has prevented the request from succeeding",
            Reason: "InternalError",
            Details: {
                Name: "",
                Group: "",
                Kind: "",
                UID: "",
                Causes: [
                    {
                        Type: "UnexpectedServerResponse",
                        Message: "unknown",
                        Field: "",
                    },
                ],
                RetryAfterSeconds: 0,
            },
            Code: 500,
        },
    }
    an error on the server ("unknown") has prevented the request from succeeding
occurred

in kube-apiserver can see this log:

E0129 03:16:45.196675       9 status.go:71] apiserver received an error that is not an metav1.Status: &fmt.wrapError{msg:"error trying to reach service: dial tcp 10.40.0.5:10250: i/o timeout", err:(*net.OpError)(0xc004f9c460)}: error trying to reach service: dial tcp 10.40.0.5:10250: i/o timeout
I0129 03:16:45.197557       9 httplog.go:89] "HTTP" verb="GET" URI="/api/v1/nodes/e2e-600aa4dc11-a7d53-minion-group-rrxj:10250/proxy/logs/" latency="30.00430939s" userAgent="e2e.test/v1.21.0 (linux/amd64) kubernetes/4990939 -- [sig-network] Proxy version v1 should proxy logs on node with explicit kubelet port using proxy subresource " srcIP="104.198.188.116:44562" resp=500 statusStack="\ngoroutine 552213 [running]:\nk8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).recordStatus(0xc019ef6bd0, 0x1f4)\n\tstaging/src/k8s.io/apiserver/pkg/server/httplog/httplog.go:237 +0xcf\nk8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).WriteHeader(0xc019ef6bd0, 0x1f4)\n\tstaging/src/k8s.io/apiserver/pkg/server/httplog/httplog.go:216 +0x35\nk8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters.(*auditResponseWriter).WriteHeader(0xc0175ca000, 0x1f4)\n\tstaging/src/k8s.io/apiserver/pkg/endpoints/filters/audit.go:223 +0x65\nk8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/metrics.(*ResponseWriterDelegator).WriteHeader(0xc010b89590, 0x1f4)\n\tstaging/src/k8s.io/apiserver/pkg/endpoints/metrics/metrics.go:571 +0x45\nk8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/handlers/responsewriters.(*deferredResponseWriter).Write(0xc015531da0,

but node 's kubelet seems still ok at that time

I0129 03:16:45.914358    9672 reconciler.go:254] Starting operationExecutor.MountVolume for volume "pvc-f3e8f7b7-7aa7-4c34-b0eb-6cd0e6cde127" (UniqueName: "kubernetes.io/csi/csi-hostpath-provisioning-35^4cca78d7-61e0-11eb-93aa-4e0c565bd51b") pod "pod-subpath-test-dynamicpv-x84s" (UID: "6ea899a0-e968-471f-8d7a-edd6a718947b")
I0129 03:16:45.914410    9672 shared_informer.go:270] caches populated
I0129 03:16:46.014879    9672 reconciler.go:254] Starting operationExecutor.MountVolume for volume "pvc-f3e8f7b7-7aa7-4c34-b0eb-6cd0e6cde127" (UniqueName: "kubernetes.io/csi/csi-hostpath-provisioning-35^4cca78d7-61e0-11eb-93aa-4e0c565bd51b") pod "pod-subpath-test-dynamicpv-x84s" (UID: "6ea899a0-e968-471f-8d7a-edd6a718947b")
I0129 03:16:46.015176    9672 shared_informer.go:270] caches populated
I0129 03:16:46.115450    9672 reconciler.go:254] Starting operationExecutor.MountVolume for volume "pvc-f3e8f7b7-7aa7-4c34-b0eb-6cd0e6cde127" (UniqueName: "kubernetes.io/csi/csi-hostpath-provisioning-35^4cca78d7-61e0-11eb-93aa-4e0c565bd51b") pod "pod-subpath-test-dynamicpv-x84s" (UID: "6ea899a0-e968-471f-8d7a-edd6a718947b")
I0129 03:16:46.115761    9672 shared_informer.go:270] caches populated
I0129 03:16:46.216256    9672 shared_informer.go:270] caches populated
I0129 03:16:46.216279    9672 reconciler.go:254] Starting operationExecutor.MountVolume for volume "pvc-f3e8f7b7-7aa7-4c34-b0eb-6cd0e6cde127" (UniqueName: "kubernetes.io/csi/csi-hostpath-provisioning-35^4cca78d7-61e0-11eb-93aa-4e0c565bd51b") pod "pod-subpath-test-dynamicpv-x84s" (UID: "6ea899a0-e968-471f-8d7a-edd6a718947b")
I0129 03:16:46.317043    9672 shared_informer.go:270] caches populated
I0129 03:16:46.317089    9672 reconciler.go:254] Starting operationExecutor.MountVolume for volume "pvc-f3e8f7b7-7aa7-4c34-b0eb-6cd0e6cde127" (UniqueName: "kubernetes.io/csi/csi-hostpath-provisioning-35^4cca78d7-61e0-11eb-93aa-4e0c565bd51b") pod "pod-subpath-test-dynamicpv-x84s" (UID: "6ea899a0-e968-471f-8d7a-edd6a718947b")
I0129 03:16:46.369458    9672 prober.go:159] Exec-Probe Pod: ss-0, Container: webserver, Command: [test -f /data/statefulset-continue]
I0129 03:16:46.418560    9672 reconciler.go:254] Starting operationExecutor.MountVolume for volume "pvc-f3e8f7b7-7aa7-4c34-b0eb-6cd0e6cde127" (UniqueName: "kubernetes.io/csi/csi-hostpath-provisioning-35^4cca78d7-61e0-11eb-93aa-4e0c565bd51b") pod "pod-subpath-test-dynamicpv-x84s" (UID: "6ea899a0-e968-471f-8d7a-edd6a718947b")
I0129 03:16:46.418836    9672 shared_informer.go:270] caches populated
I0129 03:16:46.432343    9672 exec.go:62] Exec probe response: ""
I0129 03:16:46.432381    9672 prober.go:126] Readiness probe for "ss-0_statefulset-4204(49e955f3-e181-4011-809f-8cee90775b72):webserver" succeeded
I0129 03:16:46.519103    9672 reconciler.go:254] Starting operationExecutor.MountVolume for volume "pvc-f3e8f7b7-7aa7-4c34-b0eb-6cd0e6cde127" (UniqueName: "kubernetes.io/csi/csi-hostpath-provisioning-35^4cca78d7-61e0-11eb-93aa-4e0c565bd51b") pod "pod-subpath-test-dynamicpv-x84s" (UID: "6ea899a0-e968-471f-8d7a-edd6a718947b")
I0129 03:16:46.519283    9672 shared_informer.go:270] caches populated
I0129 03:16:46.619619    9672 reconciler.go:254] Starting operationExecutor.MountVolume for volume "pvc-f3e8f7b7-7aa7-4c34-b0eb-6cd0e6cde127" (UniqueName: "kubernetes.io/csi/csi-hostpath-provisioning-35^4cca78d7-61e0-11eb-93aa-4e0c565bd51b") pod "pod-subpath-test-dynamicpv-x84s" (UID: "6ea899a0-e968-471f-8d7a-edd6a718947b")
I0129 03:16:46.619738    9672 shared_informer.go:270] caches populated
I0129 03:16:46.720100    9672 reconciler.go:254] Starting operationExecutor.MountVolume for volume "pvc-f3e8f7b7-7aa7-4c34-b0eb-6cd0e6cde127" (UniqueName: "kubernetes.io/csi/csi-hostpath-provisioning-35^4cca78d7-61e0-11eb-93aa-4e0c565bd51b") pod "pod-subpath-test-dynamicpv-x84s" (UID: "6ea899a0-e968-471f-8d7a-edd6a718947b")
I0129 03:16:46.720692    9672 shared_informer.go:270] caches populated
I0129 03:16:46.820976    9672 reconciler.go:254] Starting operationExecutor.MountVolume for volume "pvc-f3e8f7b7-7aa7-4c34-b0eb-6cd0e6cde127" (UniqueName: "kubernetes.io/csi/csi-hostpath-provisioning-35^4cca78d7-61e0-11eb-93aa-4e0c565bd51b") pod "pod-subpath-test-dynamicpv-x84s" (UID: "6ea899a0-e968-471f-8d7a-edd6a718947b")
I0129 03:16:46.821105    9672 shared_informer.go:270] caches populated
I0129 03:16:46.878588    9672 prober.go:174] HTTP-Probe Host: http://10.64.3.134, Port: 9898, Path: /healthz
I0129 03:16:46.878649    9672 prober.go:177] HTTP-Probe Headers: map[]

seems network connection problems

/test pull-kubernetes-e2e-gce-ubuntu-containerd

@JornShen
Copy link
Member Author

Both seem to read the list of interface addresses. One filters that through a list of CIDRs, the other doesn't.

On Thu, Jan 28, 2021 at 11:42 PM Canhui Shen @.***> wrote: @JornShen commented on this pull request. ------------------------------ In pkg/proxy/iptables/proxier.go <#98083 (comment)> : > @@ -1007,6 +992,12 @@ func (proxier *Proxier) syncProxyRules() { proxier.endpointChainsNumber += len(proxier.endpointsMap[svcName]) } + localAddrSet := utilproxy.GetLocalAddrSet() which place you think it is overlap? :) — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#98083 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKWAVH2HWCP5LYODMQBTNTS4JRFTANCNFSM4WDLFYZQ .

filter through cidr, I think it is a validation of output data.

I know, now I just pull out the same operation to method GetLocalAddrSet. And it will call the same func GetLocalAddres. One the other hand, GetLocalAddrSet just give the ipset of result of GetLocalAddres. I think we can consider to merge them to one method.

But I worry about the method will do much more thing if we merge them.

@thockin

@k8s-ci-robot k8s-ci-robot merged commit e89e7b4 into kubernetes:master Jan 29, 2021
@k8s-ci-robot k8s-ci-robot added this to the v1.21 milestone Jan 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/ipvs cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. release-note-none Denotes a PR that doesn't merit a release note. sig/network Categorizes an issue or PR as relevant to SIG Network. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants