Is there a way to communicate across kubernetes nodes?

[YYGCui]

I use flannel to communicate across kubernetes nodes. pods can communicate via the default interface eth0.

If I add more one, the cni configure file looks as follows:
---master---

      "type": "ptp",
      "ipam": {
        "type": "host-local",
        "subnet": "172.16.1.0/24",
        "routes": [
          { "dst": "0.0.0.0/0" }
        ],
        "gateway": "172.16.1.1"
     }

---minion---

      "type": "ptp",
      "ipam": {
        "type": "host-local",
        "subnet": "172.16.2.0/24",
        "routes": [
          { "dst": "0.0.0.0/0" }
        ],
        "gateway": "172.16.2.1"
     }

the pods in the same node can reachable, but across nodes unreachable because they are in different subnets.(use different subnets to avoid ip conflict)

I cannot find a way to make them reachable.

should I use the default route? (created by flannel-docker-bridge)

Is there are some example for that?

[rkamudhan]

Why can't you use flannel Multiple Network use case ? It is more efficient than using ptp plugin.

[YYGCui]

Actually, I'd like to use flannel Multiple network.
But I can't sure that if two flannel network can be used in a kubernetes cluster

[YYGCui]

If I understand well, the conf should be like

{
   "name":"mynet",
   "type":"multus",
   "delegates":[
      {
#the second network
         "type":"flannel",
         "delegate":{
            "bridge": "mybr0",
            "mtu":1472,
            "ipMasq":false,
            "isGateway":true,
            "ipam":{
               "type":"host-local",
               "subnet":"10.1.17.0/24"
            }
         }
      },
      {
#the default network
         "type":"flannel",
         "masterplugin":true,
         "delegate":{
            "isDefaultGateway":true
         }
      }
   ]
}

Am I right?

[rkamudhan]

Please follow the steps for the Mutliple flannel Network. To my knowledge ipam is not allowed in flannel delegate, please correct me if I am wrong. Flannel get the IPAM information from the /run/flannel/subnet.env

[rkamudhan]

@YYGCui - Is the Multiple Flannel Network, solved your issue ?

[YYGCui]

Multiple Flannel Network is worked for me. Thanks!

the conf file

{
    "name": "multus-demo-network",
    "type": "multus",
    "delegates": [
        {
                "type": "flannel",
                "name": "flannel.2",
                "subnetFile": "/run/flannel/networks/2.env",
                "dataDir": "/var/lib/cni/flannel/2",
                "delegate": {
                        "bridge": "kbr1"
                }
        },
        {
                "type": "flannel",
                "name": "flannel.1",
                "subnetFile": "/run/flannel/subnet.env",
                "dataDir": "/var/lib/cni/flannel",
                "masterplugin": true,
                "delegate": {
                        "bridge": "kbr0",
                        "isDefaultGateway": true
                }
        }
    ]
}

[rkamudhan]

@YYGCui added your CNI config information in the Multus README.md file