Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix in kube-proxy for sctp ipset entries #81477

Merged

Conversation

@paulsubrata55
Copy link
Contributor

commented Aug 15, 2019

What type of PR is this?

/kind bug

What this PR does / why we need it:
Kube-proxy will add ipset entries for all node ips for an SCTP nodeport service. This will solve the problem SCTP nodeport service is not working for all IPs present in the node when ipvs is enabled. It is working only for node's InternalIP.

Which issue(s) this PR fixes:
Fixes #81474

Does this PR introduce a user-facing change?:
NONE

Fix in kube-proxy for SCTP nodeport service which only works for node's InternalIP, but doesn't work for other IPs present in the node when ipvs is enabled.
@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented Aug 15, 2019

Hi @paulsubrata55. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@paulsubrata55

This comment has been minimized.

Copy link
Contributor Author

commented Aug 15, 2019

/cc m1093782566

@k8s-ci-robot k8s-ci-robot requested a review from m1093782566 Aug 15, 2019
@k8s-ci-robot k8s-ci-robot requested review from johnbelamaric and MrHohn Aug 15, 2019
@andrewsykim

This comment has been minimized.

Copy link
Member

commented Aug 15, 2019

/ok-to-test

/assign @andrewsykim @lbernail

@paulsubrata55

This comment has been minimized.

Copy link
Contributor Author

commented Aug 15, 2019

/test pull-kubernetes-bazel-test

@lbernail

This comment has been minimized.

Copy link
Contributor

commented Aug 15, 2019

The PR makes sense
Could you add a test case to TestNodePort in proxier_test with this scenario?

@k8s-ci-robot k8s-ci-robot added size/L and removed size/M labels Aug 16, 2019
@paulsubrata55

This comment has been minimized.

Copy link
Contributor Author

commented Aug 16, 2019

The PR makes sense
Could you add a test case to TestNodePort in proxier_test with this scenario?

Added a test case for this scenario.

if len(entries) == 1 {
if ents[0] != entries[0].String() {
t.Errorf("Check ipset entries failed for ipset: %q", set)
for _, entry := range entries {

This comment has been minimized.

Copy link
@andrewsykim

andrewsykim Aug 16, 2019

Member

Wondering if we can just reflect.DeepEqual the two ipset entries here and then t.Errorf if not equal, or does that fail for other reasons?

This comment has been minimized.

Copy link
@paulsubrata55

paulsubrata55 Aug 16, 2019

Author Contributor

Wondering if we can just reflect.DeepEqual the two ipset entries here and then t.Errorf if not equal, or does that fail for other reasons?

we can not use deepEqual, because the return of ipset.ListEntries does not guarantee any order of elements in the slice, which may not match with the order of expected entries. And Array values are deeply equal when their corresponding elements are deeply equal.

This comment has been minimized.

Copy link
@andrewsykim

andrewsykim Aug 16, 2019

Member

Sort then DeepEqual? Or update the fake ListEntries to use List() instead of UnsortedList.

This comment has been minimized.

Copy link
@paulsubrata55

paulsubrata55 Aug 17, 2019

Author Contributor

Sort then DeepEqual? Or update the fake ListEntries to use List() instead of UnsortedList.

Thanks for review, Added Sort then DeepEqual

@paulsubrata55

This comment has been minimized.

Copy link
Contributor Author

commented Aug 17, 2019

/test pull-kubernetes-e2e-gce-device-plugin-gpu

@lbernail

This comment has been minimized.

Copy link
Contributor

commented Aug 17, 2019

Thank you for the test case!
/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm label Aug 17, 2019
@lbernail

This comment has been minimized.

Copy link
Contributor

commented Aug 17, 2019

/kind bug

@k8s-ci-robot k8s-ci-robot added kind/bug and removed needs-kind labels Aug 17, 2019
@andrewsykim

This comment has been minimized.

Copy link
Member

commented Aug 17, 2019

Can you squash commits into meaningful commits please?

/priority important-soon

@paulsubrata55 paulsubrata55 force-pushed the paulsubrata55:kube-proxy-sctp-ipset-fix branch from fbae504 to 138b8b8 Aug 17, 2019
@k8s-ci-robot k8s-ci-robot removed the lgtm label Aug 17, 2019
@paulsubrata55

This comment has been minimized.

Copy link
Contributor Author

commented Aug 17, 2019

Can you squash commits into meaningful commits please?

/priority important-soon

Done

Kube-proxy will add ipset entries for all node ips for an SCTP nodeport service. This will solve the problem 'SCTP nodeport service is not working for all IPs present in the node when ipvs is enabled. It is working only for node's InternalIP.'
@lbernail

This comment has been minimized.

Copy link
Contributor

commented Aug 23, 2019

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm label Aug 23, 2019
@andrewsykim

This comment has been minimized.

Copy link
Member

commented Aug 26, 2019

Overall LGTM

@paulsubrata55 wondering if you had a chance to test these changes, some extra validation would be great since we don't have e2es for this specific case yet.

Also the release notes need to be updated.

@paulsubrata55

This comment has been minimized.

Copy link
Contributor Author

commented Aug 26, 2019

Overall LGTM

@paulsubrata55 wondering if you had a chance to test these changes, some extra validation would be great since we don't have e2es for this specific case yet.

Yes I have done the end-to-end test with this specific scenario and it works perfectly and after that only I created the PR.

@andrewsykim

This comment has been minimized.

Copy link
Member

commented Aug 26, 2019

Yes I have done the end-to-end test with this specific scenario and it works perfectly and after that only I created the PR.

Can you share output/results of how you tested this please? Thanks!

@paulsubrata55

This comment has been minimized.

Copy link
Contributor Author

commented Aug 28, 2019

Can you share output/results of how you tested this please? Thanks!

The end-to-end test I meant was more of a manual testing. It was done with 2 nodes with a sample sctp server running as deployment with 3 replicas and trying to connect with an sctp client via a nodeport service. I provided all the nodeIPs to the sctp client as server ips, then connected to the server one by one and sent some data. Then I monitored the traffic using tcpdump tool and analyzed that the DNAT and SNAT are working and also verified using ipvsadm command hitting the real-servers. Thats how I tested.
If you need results/output I can only provide the details using kubectl / tcpdump / ipvsadm / ipset etc. tools for various stages of the test.

@andrewsykim

This comment has been minimized.

Copy link
Member

commented Aug 28, 2019

If you need results/output I can only provide the details using kubectl / tcpdump / ipvsadm / ipset etc. tools for various stages of the test.

Given the lack of E2E tests for SCTP, I think this would be enough validation for now. We should have e2es for this eventually though.

@paulsubrata55

This comment has been minimized.

Copy link
Contributor Author

commented Aug 28, 2019

The test setup is like below: It's a 2 nodes cluster:
1. master node: nodename kmaster. InternalIP: 192.168.56.105. This node also has an extra ip from other subnet (100.100.100.4). Both ips are accessible from outside.
2. worker node: nodename knode1. InternalIP: 192.168.56.102.
3. An extra demo system with ips: [192.168.56.101, 100.100.100.3] and is not part of cluster. Basically from this system I will try to access the nodeport service.

Cluster is up and running.

user@kmaster:~$ kubectl get nodes
NAME      STATUS   ROLES    AGE   VERSION
kmaster   Ready    master   60m   v1.15.3
knode1    Ready    <none>   16s   v1.15.3

Below is SCTP server deployment:

apiVersion: apps/v1                                                             
kind: Deployment                                                                
metadata:                                                                       
  name: sample-sctp-server                                                      
  labels:                                                                       
    app: sample-sctp-server                                                     
spec:                                                                           
  replicas: 3                                                                   
  selector:                                                                     
    matchLabels:                                                                
      app: sample-sctp-server                                                   
  template:                                                                     
    metadata:                                                                   
      labels:                                                                   
        app: sample-sctp-server                                                 
    spec:                                                                       
      containers:                                                               
      - name: sample-sctp-server                                                
        image: master:5000/sctp-example:latest                                  
        ports:                                                                  
        - containerPort: 1000                                                   
          protocol: SCTP

user@kmaster:~$ kubectl get pods -o wide
NAME                                 READY   STATUS    RESTARTS   AGE   IP              NODE     NOMINATED NODE   READINESS GATES
sample-sctp-server-599fd64f9-jls7r   1/1     Running   0          12m   20.20.195.137   knode1   <none>           <none>
sample-sctp-server-599fd64f9-r59l2   1/1     Running   0          12m   20.20.195.136   knode1   <none>           <none>
sample-sctp-server-599fd64f9-zxb8l   1/1     Running   0          12m   20.20.195.135   knode1   <none>           <none>

And a nodeport service for the above:

kind: Service                                                                   
apiVersion: v1                                                                  
metadata:                                                                       
  name: sample-sctp-service                                                     
spec:                                                                           
  type: NodePort                                                                
  selector:                                                                     
    app: sample-sctp-server                                                     
  ports:                                                                        
  - protocol: SCTP                                                              
    port: 1000                                                                  
    targetPort: 1000                                                            
    nodePort: 31514                                                             

IPVS output on kmaster

user@kmaster:~$ sudo ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.96.0.1:443 rr
  -> 192.168.56.105:6443          Masq    1      3          0         
TCP  10.96.0.10:53 rr
  -> 20.20.210.3:53               Masq    1      0          0         
  -> 20.20.210.4:53               Masq    1      0          0         
TCP  10.96.0.10:9153 rr
  -> 20.20.210.3:9153             Masq    1      0          0         
  -> 20.20.210.4:9153             Masq    1      0          0         
TCP  10.96.232.136:6666 rr
  -> 192.168.56.105:6666          Masq    1      5          0         
UDP  10.96.0.10:53 rr
  -> 20.20.210.3:53               Masq    1      0          0         
  -> 20.20.210.4:53               Masq    1      0          0         
SCTP 10.0.2.15:31514 rr
  -> 20.20.195.135:1000           Masq    1      0          0         
  -> 20.20.195.136:1000           Masq    1      0          0         
  -> 20.20.195.137:1000           Masq    1      0          0         
SCTP 10.103.164.9:1000 rr
  -> 20.20.195.135:1000           Masq    1      0          0         
  -> 20.20.195.136:1000           Masq    1      0          0         
  -> 20.20.195.137:1000           Masq    1      0          0         
SCTP 20.20.210.0:31514 rr
  -> 20.20.195.135:1000           Masq    1      0          0         
  -> 20.20.195.136:1000           Masq    1      0          0         
  -> 20.20.195.137:1000           Masq    1      0          0         
SCTP 100.100.100.4:31514 rr
  -> 20.20.195.135:1000           Masq    1      0          0         
  -> 20.20.195.136:1000           Masq    1      0          0         
  -> 20.20.195.137:1000           Masq    1      0          0         
SCTP 127.0.0.1:31514 rr
  -> 20.20.195.135:1000           Masq    1      0          0         
  -> 20.20.195.136:1000           Masq    1      0          0         
  -> 20.20.195.137:1000           Masq    1      0          0         
SCTP 172.17.0.1:31514 rr
  -> 20.20.195.135:1000           Masq    1      0          0         
  -> 20.20.195.136:1000           Masq    1      0          0         
  -> 20.20.195.137:1000           Masq    1      0          0         
SCTP 192.168.56.105:31514 rr
  -> 20.20.195.135:1000           Masq    1      0          0         
  -> 20.20.195.136:1000           Masq    1      0          0         
  -> 20.20.195.137:1000           Masq    1      0          0         

The same on knode1:

root@knode1:~# sudo ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.96.0.1:443 rr
  -> 192.168.56.105:6443          Masq    1      0          0
TCP  10.96.0.10:53 rr
  -> 20.20.210.1:53               Masq    1      0          0
  -> 20.20.210.2:53               Masq    1      0          0
TCP  10.96.0.10:9153 rr
  -> 20.20.210.1:9153             Masq    1      0          0
  -> 20.20.210.2:9153             Masq    1      0          0
TCP  10.96.232.136:6666 rr
  -> 192.168.56.105:6666          Masq    1      3          0
UDP  10.96.0.10:53 rr
  -> 20.20.210.1:53               Masq    1      0          0
  -> 20.20.210.2:53               Masq    1      0          0
SCTP 192.168.56.102:31514 rr
  -> 20.20.195.135:1000           Masq    1      0          0
  -> 20.20.195.136:1000           Masq    1      0          0
  -> 20.20.195.137:1000           Masq    1      0          0
SCTP 10.0.3.15:31514 rr
  -> 20.20.195.135:1000           Masq    1      0          0
  -> 20.20.195.136:1000           Masq    1      0          0
  -> 20.20.195.137:1000           Masq    1      0          0
SCTP 10.0.4.1:31514 rr
  -> 20.20.195.135:1000           Masq    1      0          0
  -> 20.20.195.136:1000           Masq    1      0          0
  -> 20.20.195.137:1000           Masq    1      0          0
SCTP 10.103.164.9:1000 rr
  -> 20.20.195.135:1000           Masq    1      0          0
  -> 20.20.195.136:1000           Masq    1      0          0
  -> 20.20.195.137:1000           Masq    1      0          0
SCTP 20.20.195.134:31514 rr
  -> 20.20.195.135:1000           Masq    1      0          0
  -> 20.20.195.136:1000           Masq    1      0          0
  -> 20.20.195.137:1000           Masq    1      0          0
SCTP 127.0.0.1:31514 rr
  -> 20.20.195.135:1000           Masq    1      0          0
  -> 20.20.195.136:1000           Masq    1      0          0
  -> 20.20.195.137:1000           Masq    1      0          0
SCTP 172.17.0.1:31514 rr
  -> 20.20.195.135:1000           Masq    1      0          0
  -> 20.20.195.136:1000           Masq    1      0          0
  -> 20.20.195.137:1000           Masq    1      0          0

ipset output for nodeport on kmaster:

user@kmaster:~$ sudo ipset list KUBE-NODE-PORT-SCTP-HASH
Name: KUBE-NODE-PORT-SCTP-HASH
Type: hash:ip,port
Revision: 5
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 472
References: 1
Number of entries: 6
Members:
100.100.100.4,sctp:31514
10.0.2.15,sctp:31514
192.168.56.105,sctp:31514
127.0.0.1,sctp:31514
20.20.210.0,sctp:31514
172.17.0.1,sctp:31514

And the same on knode1:

root@knode1:~# ipset list KUBE-NODE-PORT-SCTP-HASH
Name: KUBE-NODE-PORT-SCTP-HASH
Type: hash:ip,port
Revision: 5
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 512
References: 1
Members:
127.0.0.1,sctp:31514
172.17.0.1,sctp:31514
10.0.4.1,sctp:31514
20.20.195.134,sctp:31514
192.168.56.102,sctp:31514
10.0.3.15,sctp:31514

Now tried establishing connection toward the sctp nodeport service from the extra demo system. Here I tried using all the ips from the kubernetes cluster which are accessible from outside cluster.

root@demo:~# sudo ./example -port 31514 -ip 192.168.56.105
2019/08/28 23:17:16 Resolved address '192.168.56.105' to 192.168.56.105
2019/08/28 23:17:16 SndBufSize: 212992, RcvBufSize: 212992
2019/08/28 23:17:16 write: len 256
2019/08/28 23:17:16 read: len 256, info: &{Stream:0 SSN:0 Flags:0 _:0 PPID:0 Context:0 TTL:0 TSN:1069362122 CumTSN:0 AssocID:7}
root@demo:~# sudo ./example -port 31514 -ip 100.100.100.4
2019/08/28 23:17:36 Resolved address '100.100.100.4' to 100.100.100.4
2019/08/28 23:17:36 SndBufSize: 212992, RcvBufSize: 212992
2019/08/28 23:17:36 write: len 256
2019/08/28 23:17:36 read: len 256, info: &{Stream:0 SSN:0 Flags:0 _:0 PPID:0 Context:0 TTL:0 TSN:2433597897 CumTSN:0 AssocID:8}
root@demo:~# sudo ./example -port 31514 -ip 192.168.56.102
2019/08/28 23:17:59 Resolved address '192.168.56.102' to 192.168.56.102
2019/08/28 23:17:59 SndBufSize: 212992, RcvBufSize: 212992
2019/08/28 23:17:59 write: len 256
2019/08/28 23:17:59 read: len 256, info: &{Stream:0 SSN:0 Flags:0 _:0 PPID:0 Context:0 TTL:0 TSN:1287490956 CumTSN:0 AssocID:9}

Tcpdump on kmaster:

user@kmaster:example$ sudo tcpdump -n -i any sctp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
23:17:16.442963 IP 192.168.56.101.48378 > 192.168.56.105.31514: sctp (1) [INIT] [init tag: 4284834759] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 896736855] 
23:17:16.443041 IP 20.20.210.0.48378 > 20.20.195.135.1000: sctp (1) [INIT] [init tag: 4284834759] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 896736855] 
23:17:16.443591 IP 20.20.195.135.1000 > 20.20.210.0.48378: sctp (1) [INIT ACK] [init tag: 3189175015] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 1069362122] 
23:17:16.443614 IP 192.168.56.105.31514 > 192.168.56.101.48378: sctp (1) [INIT ACK] [init tag: 3189175015] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 1069362122] 
23:17:16.443979 IP 192.168.56.101.48378 > 192.168.56.105.31514: sctp (1) [COOKIE ECHO] 
23:17:16.444003 IP 20.20.210.0.48378 > 20.20.195.135.1000: sctp (1) [COOKIE ECHO] 
23:17:16.444378 IP 20.20.195.135.1000 > 20.20.210.0.48378: sctp (1) [COOKIE ACK] 
23:17:16.444402 IP 192.168.56.105.31514 > 192.168.56.101.48378: sctp (1) [COOKIE ACK] 
23:17:16.444778 IP 192.168.56.101.48378 > 192.168.56.105.31514: sctp (1) [DATA] (B)(E) [TSN: 896736855] [SID: 0] [SSEQ 0] [PPID 0x0] 
23:17:16.444834 IP 20.20.210.0.48378 > 20.20.195.135.1000: sctp (1) [DATA] (B)(E) [TSN: 896736855] [SID: 0] [SSEQ 0] [PPID 0x0] 
23:17:16.445411 IP 20.20.195.135.1000 > 20.20.210.0.48378: sctp (1) [SACK] [cum ack 896736855] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0] 
23:17:16.445445 IP 192.168.56.105.31514 > 192.168.56.101.48378: sctp (1) [SACK] [cum ack 896736855] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0] 
23:17:16.445490 IP 20.20.195.135.1000 > 20.20.210.0.48378: sctp (1) [DATA] (B)(E) [TSN: 1069362122] [SID: 0] [SSEQ 0] [PPID 0x0] 
23:17:16.445504 IP 192.168.56.105.31514 > 192.168.56.101.48378: sctp (1) [DATA] (B)(E) [TSN: 1069362122] [SID: 0] [SSEQ 0] [PPID 0x0] 
23:17:16.445642 IP 192.168.56.101.48378 > 192.168.56.105.31514: sctp (1) [SACK] [cum ack 1069362122] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0] 
23:17:16.445670 IP 20.20.210.0.48378 > 20.20.195.135.1000: sctp (1) [SACK] [cum ack 1069362122] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0] 
23:17:17.447902 IP 192.168.56.101.48378 > 192.168.56.105.31514: sctp (1) [SHUTDOWN] 
23:17:17.448016 IP 20.20.210.0.48378 > 20.20.195.135.1000: sctp (1) [SHUTDOWN] 
23:17:17.449241 IP 20.20.195.135.1000 > 20.20.210.0.48378: sctp (1) [SHUTDOWN ACK] 
23:17:17.449321 IP 192.168.56.105.31514 > 192.168.56.101.48378: sctp (1) [SHUTDOWN ACK] 
23:17:17.450310 IP 192.168.56.101.48378 > 192.168.56.105.31514: sctp (1) [SHUTDOWN COMPLETE] 
23:17:17.450397 IP 20.20.210.0.48378 > 20.20.195.135.1000: sctp (1) [SHUTDOWN COMPLETE] 


23:17:36.901789 IP 100.100.100.3.52543 > 100.100.100.4.31514: sctp (1) [INIT] [init tag: 4219501946] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 3523091732] 
23:17:36.901858 IP 20.20.210.0.52543 > 20.20.195.136.1000: sctp (1) [INIT] [init tag: 4219501946] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 3523091732] 
23:17:36.902343 IP 20.20.195.136.1000 > 20.20.210.0.52543: sctp (1) [INIT ACK] [init tag: 1231668309] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 2433597897] 
23:17:36.902366 IP 100.100.100.4.31514 > 100.100.100.3.52543: sctp (1) [INIT ACK] [init tag: 1231668309] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 2433597897] 
23:17:36.902642 IP 100.100.100.3.52543 > 100.100.100.4.31514: sctp (1) [COOKIE ECHO] 
23:17:36.902679 IP 20.20.210.0.52543 > 20.20.195.136.1000: sctp (1) [COOKIE ECHO] 
23:17:36.903065 IP 20.20.195.136.1000 > 20.20.210.0.52543: sctp (1) [COOKIE ACK] 
23:17:36.903091 IP 100.100.100.4.31514 > 100.100.100.3.52543: sctp (1) [COOKIE ACK] 
23:17:36.903925 IP 100.100.100.3.52543 > 100.100.100.4.31514: sctp (1) [DATA] (B)(E) [TSN: 3523091732] [SID: 0] [SSEQ 0] [PPID 0x0] 
23:17:36.903964 IP 20.20.210.0.52543 > 20.20.195.136.1000: sctp (1) [DATA] (B)(E) [TSN: 3523091732] [SID: 0] [SSEQ 0] [PPID 0x0] 
23:17:36.904389 IP 20.20.195.136.1000 > 20.20.210.0.52543: sctp (1) [SACK] [cum ack 3523091732] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0] 
23:17:36.904432 IP 100.100.100.4.31514 > 100.100.100.3.52543: sctp (1) [SACK] [cum ack 3523091732] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0] 
23:17:36.904616 IP 20.20.195.136.1000 > 20.20.210.0.52543: sctp (1) [DATA] (B)(E) [TSN: 2433597897] [SID: 0] [SSEQ 0] [PPID 0x0] 
23:17:36.904638 IP 100.100.100.4.31514 > 100.100.100.3.52543: sctp (1) [DATA] (B)(E) [TSN: 2433597897] [SID: 0] [SSEQ 0] [PPID 0x0] 
23:17:36.904840 IP 100.100.100.3.52543 > 100.100.100.4.31514: sctp (1) [SACK] [cum ack 2433597897] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0] 
23:17:36.904873 IP 20.20.210.0.52543 > 20.20.195.136.1000: sctp (1) [SACK] [cum ack 2433597897] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0] 
23:17:37.908449 IP 100.100.100.3.52543 > 100.100.100.4.31514: sctp (1) [SHUTDOWN] 
23:17:37.908495 IP 20.20.210.0.52543 > 20.20.195.136.1000: sctp (1) [SHUTDOWN] 
23:17:37.909048 IP 20.20.195.136.1000 > 20.20.210.0.52543: sctp (1) [SHUTDOWN ACK] 
23:17:37.909072 IP 100.100.100.4.31514 > 100.100.100.3.52543: sctp (1) [SHUTDOWN ACK] 
23:17:37.909279 IP 100.100.100.3.52543 > 100.100.100.4.31514: sctp (1) [SHUTDOWN COMPLETE] 
23:17:37.909316 IP 20.20.210.0.52543 > 20.20.195.136.1000: sctp (1) [SHUTDOWN COMPLETE] 

Tcpdump on knode1:

root@knode1:~# sudo tcpdump -i any sctp -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
23:17:15.814058 IP 20.20.210.0.48378 > 20.20.195.135.1000: sctp (1) [INIT] [init tag: 4284834759] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 896736855]
23:17:15.814106 IP 20.20.210.0.48378 > 20.20.195.135.1000: sctp (1) [INIT] [init tag: 4284834759] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 896736855]
23:17:15.814179 IP 20.20.195.135.1000 > 20.20.210.0.48378: sctp (1) [INIT ACK] [init tag: 3189175015] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 1069362122]
23:17:15.814190 IP 20.20.195.135.1000 > 20.20.210.0.48378: sctp (1) [INIT ACK] [init tag: 3189175015] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 1069362122]
23:17:15.814967 IP 20.20.210.0.48378 > 20.20.195.135.1000: sctp (1) [COOKIE ECHO]
23:17:15.814982 IP 20.20.210.0.48378 > 20.20.195.135.1000: sctp (1) [COOKIE ECHO]
23:17:15.815037 IP 20.20.195.135.1000 > 20.20.210.0.48378: sctp (1) [COOKIE ACK]
23:17:15.815042 IP 20.20.195.135.1000 > 20.20.210.0.48378: sctp (1) [COOKIE ACK]
23:17:15.815968 IP 20.20.210.0.48378 > 20.20.195.135.1000: sctp (1) [DATA] (B)(E) [TSN: 896736855] [SID: 0] [SSEQ 0] [PPID 0x0]
23:17:15.815991 IP 20.20.210.0.48378 > 20.20.195.135.1000: sctp (1) [DATA] (B)(E) [TSN: 896736855] [SID: 0] [SSEQ 0] [PPID 0x0]
23:17:15.816018 IP 20.20.195.135.1000 > 20.20.210.0.48378: sctp (1) [SACK] [cum ack 896736855] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0]
23:17:15.816026 IP 20.20.195.135.1000 > 20.20.210.0.48378: sctp (1) [SACK] [cum ack 896736855] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0]
23:17:15.816096 IP 20.20.195.135.1000 > 20.20.210.0.48378: sctp (1) [DATA] (B)(E) [TSN: 1069362122] [SID: 0] [SSEQ 0] [PPID 0x0]
23:17:15.816105 IP 20.20.195.135.1000 > 20.20.210.0.48378: sctp (1) [DATA] (B)(E) [TSN: 1069362122] [SID: 0] [SSEQ 0] [PPID 0x0]
23:17:15.816661 IP 20.20.210.0.48378 > 20.20.195.135.1000: sctp (1) [SACK] [cum ack 1069362122] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0]
23:17:15.816678 IP 20.20.210.0.48378 > 20.20.195.135.1000: sctp (1) [SACK] [cum ack 1069362122] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0]
23:17:16.818916 IP 20.20.210.0.48378 > 20.20.195.135.1000: sctp (1) [SHUTDOWN]
23:17:16.818977 IP 20.20.210.0.48378 > 20.20.195.135.1000: sctp (1) [SHUTDOWN]
23:17:16.819114 IP 20.20.195.135.1000 > 20.20.210.0.48378: sctp (1) [SHUTDOWN ACK]
23:17:16.819138 IP 20.20.195.135.1000 > 20.20.210.0.48378: sctp (1) [SHUTDOWN ACK]
23:17:16.821228 IP 20.20.210.0.48378 > 20.20.195.135.1000: sctp (1) [SHUTDOWN COMPLETE]
23:17:16.821298 IP 20.20.210.0.48378 > 20.20.195.135.1000: sctp (1) [SHUTDOWN COMPLETE]


23:17:36.262648 IP 20.20.210.0.52543 > 20.20.195.136.1000: sctp (1) [INIT] [init tag: 4219501946] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 3523091732]
23:17:36.262688 IP 20.20.210.0.52543 > 20.20.195.136.1000: sctp (1) [INIT] [init tag: 4219501946] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 3523091732]
23:17:36.262745 IP 20.20.195.136.1000 > 20.20.210.0.52543: sctp (1) [INIT ACK] [init tag: 1231668309] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 2433597897]
23:17:36.262754 IP 20.20.195.136.1000 > 20.20.210.0.52543: sctp (1) [INIT ACK] [init tag: 1231668309] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 2433597897]
23:17:36.263416 IP 20.20.210.0.52543 > 20.20.195.136.1000: sctp (1) [COOKIE ECHO]
23:17:36.263432 IP 20.20.210.0.52543 > 20.20.195.136.1000: sctp (1) [COOKIE ECHO]
23:17:36.263490 IP 20.20.195.136.1000 > 20.20.210.0.52543: sctp (1) [COOKIE ACK]
23:17:36.263499 IP 20.20.195.136.1000 > 20.20.210.0.52543: sctp (1) [COOKIE ACK]
23:17:36.264732 IP 20.20.210.0.52543 > 20.20.195.136.1000: sctp (1) [DATA] (B)(E) [TSN: 3523091732] [SID: 0] [SSEQ 0] [PPID 0x0]
23:17:36.264756 IP 20.20.210.0.52543 > 20.20.195.136.1000: sctp (1) [DATA] (B)(E) [TSN: 3523091732] [SID: 0] [SSEQ 0] [PPID 0x0]
23:17:36.264806 IP 20.20.195.136.1000 > 20.20.210.0.52543: sctp (1) [SACK] [cum ack 3523091732] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0]
23:17:36.264817 IP 20.20.195.136.1000 > 20.20.210.0.52543: sctp (1) [SACK] [cum ack 3523091732] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0]
23:17:36.264962 IP 20.20.195.136.1000 > 20.20.210.0.52543: sctp (1) [DATA] (B)(E) [TSN: 2433597897] [SID: 0] [SSEQ 0] [PPID 0x0]
23:17:36.264973 IP 20.20.195.136.1000 > 20.20.210.0.52543: sctp (1) [DATA] (B)(E) [TSN: 2433597897] [SID: 0] [SSEQ 0] [PPID 0x0]
23:17:36.265672 IP 20.20.210.0.52543 > 20.20.195.136.1000: sctp (1) [SACK] [cum ack 2433597897] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0]
23:17:36.265689 IP 20.20.210.0.52543 > 20.20.195.136.1000: sctp (1) [SACK] [cum ack 2433597897] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0]
23:17:37.268801 IP 20.20.210.0.52543 > 20.20.195.136.1000: sctp (1) [SHUTDOWN]
23:17:37.268825 IP 20.20.210.0.52543 > 20.20.195.136.1000: sctp (1) [SHUTDOWN]
23:17:37.268885 IP 20.20.195.136.1000 > 20.20.210.0.52543: sctp (1) [SHUTDOWN ACK]
23:17:37.268896 IP 20.20.195.136.1000 > 20.20.210.0.52543: sctp (1) [SHUTDOWN ACK]
23:17:37.269578 IP 20.20.210.0.52543 > 20.20.195.136.1000: sctp (1) [SHUTDOWN COMPLETE]
23:17:37.269589 IP 20.20.210.0.52543 > 20.20.195.136.1000: sctp (1) [SHUTDOWN COMPLETE]


23:17:58.829946 IP 192.168.56.101.32947 > 192.168.56.102.31514: sctp (1) [INIT] [init tag: 1176805311] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 3079125443]
23:17:58.830067 IP 192.168.56.102.32947 > 20.20.195.137.1000: sctp (1) [INIT] [init tag: 1176805311] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 3079125443]
23:17:58.830168 IP 20.20.195.137.1000 > 192.168.56.102.32947: sctp (1) [INIT ACK] [init tag: 2450598349] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 1287490956]
23:17:58.830177 IP 192.168.56.102.31514 > 192.168.56.101.32947: sctp (1) [INIT ACK] [init tag: 2450598349] [rwnd: 106496] [OS: 65535] [MIS: 65535] [init TSN: 1287490956]
23:17:58.830450 IP 192.168.56.101.32947 > 192.168.56.102.31514: sctp (1) [COOKIE ECHO]
23:17:58.830463 IP 192.168.56.102.32947 > 20.20.195.137.1000: sctp (1) [COOKIE ECHO]
23:17:58.830531 IP 20.20.195.137.1000 > 192.168.56.102.32947: sctp (1) [COOKIE ACK]
23:17:58.830538 IP 192.168.56.102.31514 > 192.168.56.101.32947: sctp (1) [COOKIE ACK]
23:17:58.831283 IP 192.168.56.101.32947 > 192.168.56.102.31514: sctp (1) [DATA] (B)(E) [TSN: 3079125443] [SID: 0] [SSEQ 0] [PPID 0x0]
23:17:58.831298 IP 192.168.56.102.32947 > 20.20.195.137.1000: sctp (1) [DATA] (B)(E) [TSN: 3079125443] [SID: 0] [SSEQ 0] [PPID 0x0]
23:17:58.831392 IP 20.20.195.137.1000 > 192.168.56.102.32947: sctp (1) [SACK] [cum ack 3079125443] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0]
23:17:58.831402 IP 192.168.56.102.31514 > 192.168.56.101.32947: sctp (1) [SACK] [cum ack 3079125443] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0]
23:17:58.831546 IP 20.20.195.137.1000 > 192.168.56.102.32947: sctp (1) [DATA] (B)(E) [TSN: 1287490956] [SID: 0] [SSEQ 0] [PPID 0x0]
23:17:58.831577 IP 192.168.56.102.31514 > 192.168.56.101.32947: sctp (1) [DATA] (B)(E) [TSN: 1287490956] [SID: 0] [SSEQ 0] [PPID 0x0]
23:17:58.831834 IP 192.168.56.101.32947 > 192.168.56.102.31514: sctp (1) [SACK] [cum ack 1287490956] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0]
23:17:58.831848 IP 192.168.56.102.32947 > 20.20.195.137.1000: sctp (1) [SACK] [cum ack 1287490956] [a_rwnd 106240] [#gap acks 0] [#dup tsns 0]
23:17:59.834020 IP 192.168.56.101.32947 > 192.168.56.102.31514: sctp (1) [SHUTDOWN]
23:17:59.834097 IP 192.168.56.102.32947 > 20.20.195.137.1000: sctp (1) [SHUTDOWN]
23:17:59.834291 IP 20.20.195.137.1000 > 192.168.56.102.32947: sctp (1) [SHUTDOWN ACK]
23:17:59.834318 IP 192.168.56.102.31514 > 192.168.56.101.32947: sctp (1) [SHUTDOWN ACK]
23:17:59.834889 IP 192.168.56.101.32947 > 192.168.56.102.31514: sctp (1) [SHUTDOWN COMPLETE]
23:17:59.834936 IP 192.168.56.102.32947 > 20.20.195.137.1000: sctp (1) [SHUTDOWN COMPLETE]
@andrewsykim

This comment has been minimized.

Copy link
Member

commented Aug 28, 2019

/approve
/lgtm

Thanks for validating @paulsubrata55!

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented Aug 28, 2019

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andrewsykim, paulsubrata55

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit bd8a8db into kubernetes:master Aug 29, 2019
23 checks passed
23 checks passed
cla/linuxfoundation paulsubrata55 authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-conformance-image-test Skipped.
pull-kubernetes-cross Skipped.
pull-kubernetes-dependencies Job succeeded.
Details
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-csi-serial Skipped.
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-gce-iscsi Skipped.
pull-kubernetes-e2e-gce-iscsi-serial Skipped.
pull-kubernetes-e2e-gce-storage-slow Skipped.
pull-kubernetes-godeps Skipped.
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped.
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-node-e2e-containerd Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
pull-publishing-bot-validate Skipped.
tide In merge pool.
Details
@k8s-ci-robot k8s-ci-robot added this to the v1.16 milestone Aug 29, 2019
@paulsubrata55 paulsubrata55 deleted the paulsubrata55:kube-proxy-sctp-ipset-fix branch Aug 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.