networking: gateway in network configuration does not set default route

[steveej]

I'm running a pod having the following network defined:

{
    "name": "containers_ipv4",
    "type": "macvlan",
    "master": "enp0s20u1u1",
    "mode": "bridge",
    "ipam": {
            "type": "host-local",
            "subnet": "192.168.2.0/24",
            "gateway": "192.168.2.254"
    }
}

I would expect the defined gateway to be the default gateway on that interface, but that is not the case.
What would be the pros and cons of doing so?

[jonboulle]

@eyakubovich should we move this to cni?

[eyakubovich]

The defined gateway should be the gateway on the 192.168.2.0/24 route, not default. However that doesn't work as well. Moving this to CNI.

[eyakubovich]

@steveej My previous comment doesn't make much sense. However the behavior is as designed: ipam.gateway is a somewhat of a wart -- it just says that there is this gateway but it does not say that's the gateway for the default route. The default route has to be specified explicitly:

"ipam": {
    "type": "host-local",
    "subnet": "192.168.2.0/24",
    "routes": [
        { "dst": "0.0.0.0/0", "gw": "192.168.2.254" }
    ]
}

On the other hand, if "gateway" was specified, "gw" can be dropped:

"ipam": {
    "type": "host-local",
    "subnet": "192.168.2.0/24",
    "gateway": "192.168.2.254"
    "routes": [
        { "dst": "0.0.0.0/0" }
    ]
}

Why the complexity? It helps to look at what IPAM plugin returns: "ip", "gateway", and "routes". The main plugin uses "gateway" for things like setting an IP on the bridge (turning it into a gateway), if requested. And it will also use that value if "gw" is missing on the route. In case of "host-local", ipam.gateway configuration value just gets passed back in the result.

I will admit that it is confusing but it does allow for many use cases.

[jonboulle]

@steveej @eyakubovich what's the solution here, better documentation?

[alban]

any news on this?

[steveej]

This topic is still undecided in CNI. The tendency goes towards a new IPAM boolean attribute named similar to "gatewayIsDefault". IPAM would then generate the default route and pass it along.

[steveej]

In general the behavior remains as is, but the bridge plugin received a new 'isDefaultGateway' attribute for convenience.