Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

iptables proxy reorg in preparation for minimizing iptables-restore #110266

Merged
merged 6 commits into from
Jul 27, 2022
Merged

iptables proxy reorg in preparation for minimizing iptables-restore #110266

merged 6 commits into from
Jul 27, 2022

Conversation

danwinship
Copy link
Contributor

@danwinship danwinship commented May 28, 2022 •
edited

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

This just moves code around in pkg/proxy/iptables/proxier.go with no real change to the logic of what rules we output. But the end result is that the loop in syncProxyRules is broken up into 3 phases:

  1. figure out what chains will be needed for this servicePort, and mark them in activeNATChains
  2. write rules to KUBE-SERVICES / KUBE-EXTERNAL-SERVICES / KUBE-NODEPORTS jumping to the servicePort-specific chains
  3. write the servicePort-specific chains (KUBE-SVC-*, KUBE-SVL-*, KUBE-EXT-*, KUBE-FW-*, KUBE-SEP-*)

A followup PR (#110268) will then change it so that we skip the third step if the servicePort in question hasn't changed since the last sync, so as to minimize the input to iptables-restore.

Which issue(s) this PR fixes:

none

Does this PR introduce a user-facing change?

NONE

/sig network
/priority important-longterm

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. sig/network Categorizes an issue or PR as relevant to SIG Network. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels May 28, 2022
@k8s-ci-robot
Copy link
Contributor

@danwinship: This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. label May 28, 2022
@k8s-ci-robot k8s-ci-robot requested review from dcbw and freehan May 28, 2022 17:34
@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 28, 2022
@danwinship danwinship mentioned this pull request May 28, 2022
Merged
@danwinship
Copy link
Contributor Author

/retest-required

@danwinship danwinship force-pushed the minimize-prep-reorg branch 2 times, most recently from 895624c to 81c55fb Compare June 29, 2022 20:46
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 7, 2022
@danwinship
proxy/iptables: belatedly simplify local traffic policy metrics
8a58019 
We figure out early on whether we're going to end up outputting no
endpoints, so update the metrics then.

(Also remove a redundant feature gate check; svcInfo already checks
the ServiceInternalTrafficPolicy feature gate itself and so
svcInfo.InternalPolicyLocal() will always return false if the gate is
not enabled.)
@danwinship
proxy/iptables: move endpoint chain rule creation to the end
da14a12 
Part of reorganizing the syncProxyRules loop to do:
  1. figure out what chains are needed, mark them in activeNATChains
  2. write servicePort jump rules to KUBE-SERVICES/KUBE-NODEPORTS
  3. write servicePort-specific chains (SVC, SVL, EXT, FW, SEP)

This fixes the handling of the endpoint chains. Previously they were
handled entirely at the top of the loop. Now we record which ones are
in use at the top but don't create them and fill them in until the
bottom.
@danwinship
proxy/iptables: reorganize cluster/local chain creation
8906ab3 
Part of reorganizing the syncProxyRules loop to do:
  1. figure out what chains are needed, mark them in activeNATChains
  2. write servicePort jump rules to KUBE-SERVICES/KUBE-NODEPORTS
  3. write servicePort-specific chains (SVC, SVL, EXT, FW, SEP)

This fixes the handling of the SVC and SVL chains. We were already
filling them in at the end of the loop; this fixes it to create them
at the bottom of the loop as well.
@danwinship
proxy/iptables: move EXT chain rule creation to the end
00f789c 
Part of reorganizing the syncProxyRules loop to do:
  1. figure out what chains are needed, mark them in activeNATChains
  2. write servicePort jump rules to KUBE-SERVICES/KUBE-NODEPORTS
  3. write servicePort-specific chains (SVC, SVL, EXT, FW, SEP)

This fixes the handling of the EXT chain.
@danwinship
proxy/iptables: move internal traffic setup code
2030591 
Part of reorganizing the syncProxyRules loop to do:
  1. figure out what chains are needed, mark them in activeNATChains
  2. write servicePort jump rules to KUBE-SERVICES/KUBE-NODEPORTS
  3. write servicePort-specific chains (SVC, SVL, EXT, FW, SEP)

This fixes the jump rules for internal traffic. Previously we were
handling "jumping from kubeServices to internalTrafficChain" and
"adding masquerade rules to internalTrafficChain" in the same place.
@danwinship
proxy/iptables: move firewall chain setup
367f18c 
Part of reorganizing the syncProxyRules loop to do:
  1. figure out what chains are needed, mark them in activeNATChains
  2. write servicePort jump rules to KUBE-SERVICES/KUBE-NODEPORTS
  3. write servicePort-specific chains (SVC, SVL, EXT, FW, SEP)

This moves the FW chain creation to the end (rather than having it in
the middle of adding the jump rules for the LB IPs).
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 9, 2022
@danwinship
Copy link
Contributor Author

/retest-required

thockin
thockin reviewed Jul 27, 2022
View reviewed changes
Copy link
Member

@thockin thockin left a comment •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

args used to be an optimization - I wonder if we have accidentally pessimized by breaking apart blocks that used to share one args slice?

Overall I see what this does and why it makes the next step possible. I find it a little harder to follow, personally, but I might just be too close to the current code. There are so many variables that are set in one place and then used hundreds of lines away.

We should look for ways to make this easier to read, still.

Thanks for good, self-contained commits. Exemplary.

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 27, 2022
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: danwinship, thockin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-triage-robot
Copy link

The Kubernetes project has merge-blocking tests that are currently too flaky to consistently pass.

This bot retests PRs for certain kubernetes repos according to the following rules:

  • The PR does have any do-not-merge/* labels
  • The PR does not have the needs-ok-to-test label
  • The PR is mergeable (does not have a needs-rebase label)
  • The PR is approved (has cncf-cla: yes, lgtm, approved labels)
  • The PR is failing tests required for merge

You can:

/retest

3 similar comments
@k8s-triage-robot
Copy link

The Kubernetes project has merge-blocking tests that are currently too flaky to consistently pass.

This bot retests PRs for certain kubernetes repos according to the following rules:

  • The PR does have any do-not-merge/* labels
  • The PR does not have the needs-ok-to-test label
  • The PR is mergeable (does not have a needs-rebase label)
  • The PR is approved (has cncf-cla: yes, lgtm, approved labels)
  • The PR is failing tests required for merge

You can:

/retest

@k8s-triage-robot
Copy link

The Kubernetes project has merge-blocking tests that are currently too flaky to consistently pass.

This bot retests PRs for certain kubernetes repos according to the following rules:

  • The PR does have any do-not-merge/* labels
  • The PR does not have the needs-ok-to-test label
  • The PR is mergeable (does not have a needs-rebase label)
  • The PR is approved (has cncf-cla: yes, lgtm, approved labels)
  • The PR is failing tests required for merge

You can:

/retest

@k8s-triage-robot
Copy link

The Kubernetes project has merge-blocking tests that are currently too flaky to consistently pass.

This bot retests PRs for certain kubernetes repos according to the following rules:

  • The PR does have any do-not-merge/* labels
  • The PR does not have the needs-ok-to-test label
  • The PR is mergeable (does not have a needs-rebase label)
  • The PR is approved (has cncf-cla: yes, lgtm, approved labels)
  • The PR is failing tests required for merge

You can:

/retest

@k8s-ci-robot k8s-ci-robot merged commit ce433f8 into kubernetes:master Jul 27, 2022
@k8s-ci-robot k8s-ci-robot added this to the v1.25 milestone Jul 27, 2022
@danwinship danwinship deleted the minimize-prep-reorg branch July 27, 2022 17:28
@danwinship
Copy link
Contributor Author

args used to be an optimization - I wonder if we have accidentally pessimized by breaking apart blocks that used to share one args slice?

yeah, this is why i filed #109481

@danwinship danwinship mentioned this pull request Oct 4, 2022
Merged
@danwinship danwinship mentioned this pull request Jul 7, 2023
Merged
@saiharshitachava saiharshitachava mentioned this pull request Oct 9, 2023
Closed
@danwinship danwinship mentioned this pull request Nov 29, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@thockin thockin thockin left review comments

@dcbw dcbw Awaiting requested review from dcbw

@freehan freehan Awaiting requested review from freehan

Assignees

@thockin thockin

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. release-note-none Denotes a PR that doesn't merit a release note. sig/network Categorizes an issue or PR as relevant to SIG Network. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
v1.25
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@danwinship @k8s-ci-robot @k8s-triage-robot @thockin