Multiple interfaces in a POD

[codemachino]

Hello guys, I was wondering if there are any thoughts/implementations on how to add multiple interfaces inside a POD and connect them to different, for example bridges.

Example:

{
    "interface": [{
        "name": "mynet",
        "type": "bridge",
        "bridge": "cni0",
        "isGateway": true,
        "ipMasq": true,
        "ipam": {
            "type": "host-local",
            "subnet": "10.22.0.0/16",
            "routes": [{
                "dst": "0.0.0.0/0"
            }]
        }
    }, {
        "name": "mynet2",
        "type": "bridge",
        "bridge": "cni1",
        "isGateway": true,
        "ipMasq": true,
        "ipam": {
            "type": "host-local",
            "subnet": "10.23.0.0/16",
            "routes": [{
                "dst": "0.0.0.0/0"
            }]
        }
    }]
}

If not would it be easy for someone to point at the right files to start hacking ? I am pretty new to open source communities but I would love to contribute here. I also understand that a POD is designed to have one IP address, but the the use case I am experimenting with adds this sort of complexity.

Thanks again

[steveej]

Hello @codemachino,
are you referring to a specific runtime? You can create multiple JSON configuration files and execute the plugin(s) multiple times for the same pod. Let me know if that works for your you.

The alternative would be to expect a list of dictionaries per file instead just one dictionary. Personally I would be okay with supporting both.

[steveej]

@codemachino, have you worked this out for yourself?

I also understand that a POD is designed to have one IP address, but the the use case I am experimenting with adds this sort of complexity.

I actually think that pods may as well have multiple addresses, and CNI should definitely support that. I'd like to learn about your use case!

[rkamudhan]

@steveej with single configuration file k8s_flannel.conf (for example -net=rkt.kubernetes.io), it is not possible to add Multiple interfaces, like eth0, eth1 to the pod ? we can create multiple interfaces with multiple network file, but we can create multiple interfaces with single network configuration

[rkamudhan]

@codemachino @steveej Please check the issue #343 , we developed Multus CNI for Multiple interface support in K8s kubernetes/kubernetes#27398

[thekalinga]

@steveej Here is one usecase where we need multiple virtual IPs for a single container (or pod) so that the reverse proxy can use one of these addresses randomly while proxying requests

1. https://blog.box.com/blog/ephemeral-port-exhaustion-and-web-services-at-scale/
2. https://making.pusher.com/ephemeral-port-exhaustion-and-how-to-avoid-it/

[bboreham]

@thekalinga your links are only tenuously linked to this subject. If you have any specific experience with multiple IPs on a container, that would be an interesting blog post.

More generally, as answered two years ago, CNI supports multiple interfaces by calling multiple times. So I'm going to close this as a non-issue.