Could docker network configure the IPAM with already configured bridge addressing ?

[thomas-maurice]

Hello !

TL;DR:
When I create a docker network with a bridge that is already configured, Docker overrides the addressing I've set up. Could it be possible that when I create a docker network with an already configured bridge, if I do not supply any additionnal parameters, it configures the IPAM with the in place bridge configuration ?

More details

What I have :

root@machine # ip address show br0
27: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 0c:c4:7a:07:6a:7f brd ff:ff:ff:ff:ff:ff
    inet 172.18.4.1/24 brd 172.18.4.255 scope global br0
       valid_lft forever preferred_lft forever
    inet6 fe80::ec4:7aff:fe07:6a7f/64 scope link 
       valid_lft forever preferred_lft forever

Then I create my network

root@machine # docker network create --driver bridge -o com.docker.network.bridge.name=br0 mynetwork
c228be0db663a5add797899688d9feb8c2699fc4b0a309edd0476a88526e576b
root@machine # docker network inspect c228be0db663a5add797899688d9feb8c2699fc4b0a309edd0476a88526e576b
[
    {
        "Name": "mynetwork",
        "Id": "c228be0db663a5add797899688d9feb8c2699fc4b0a309edd0476a88526e576b",
        "Scope": "local",
        "Driver": "bridge",
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.19.0.0/16",
                    "Gateway": "172.19.0.1/16"
                }
            ]
        },
        "Containers": {},
        "Options": {
            "com.docker.network.bridge.name": "br0"
        }
    }
]
root@machine # ip address show br0
27: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 0c:c4:7a:07:6a:7f brd ff:ff:ff:ff:ff:ff
    inet 172.19.0.1/16 scope global br0
       valid_lft forever preferred_lft forever
    inet6 fe80::ec4:7aff:fe07:6a7f/64 scope link 
       valid_lft forever preferred_lft forever

The bridge addressing went from 172.18.4.0/24 to 172.19.0.0/16. Could it be relevant to use already in place configuration ?

Also, side remark, could the bridge not be deleted if it has not been created with Docker networks ?

Thanks a lot !

[CaptainYarb]

I believe this is relevant to #20123

[mavenugo]

AFAIK @Blazedd this is unrelated to #20123

@thomas-maurice regarding your question, custom network doesn't provide the -b like behavior to retain and reuse an already created bridge. Since you are using the option com.docker.network.bridge.name, bridge driver creates a bridge with the given name (and it seems it is greedy). You can use the --subnet & --gateway option to setup the preferred subnet and bridge-ip. But if you wish to do any manual plumbing of the created bridge, it must be done after it is created via docker network create command.

[thomas-maurice]

@mavenugo yes but that is the "issue" I am pointing out. If I need to specify somehow the bridge I want docker to use, it is because I need a custom one, and probably I need one because I have a custom network configuration attached to it, that I want to manage and have full control over it.

So would it be possible to implement a -b libe behavior for custom network ? So that all my already configured bridges can be reused, and also not be deleted if I decide to destroy the network ?

Also this issue could be retagged "feature request" couldn't it ?

Regards :)

[dreamcat4]

I think its useful because OS networking is configured before docker daemon starts. At that earlier time docker daemon isnt running yet and we cant run some cmds like docker network create ..<bridge params>... And it is best place to setup persistent bridges (that stay after reebots) from the file /etc/network/interfaces. Like have to bridge to eth0 then, setting up parameters. It needs to re-route everything through the bridge. So there is a brief lost network connectivity on the host. That is best kept at boot time, and not after docker-daemon has started up. Because then if restart=always some containers may have already been auto-started by then, etc.

[mavenugo]

@thomas-maurice @dreamcat4 There are a few complexities with such a requirement. Especially with IPAM. As per the design, IPAM is managed independently of the network driver. So, if the user wishes the network to use the subnet as specified in the user-created bridge, the user must use the --subnet and --gateway options to match that of the bridge. If the user doesn't specify these IPAM options while creating a network, then the IPAM driver will choose any available network subnet that may not match the user-managed bridge.

Other than that, I dont see any other issue with using the com.docker.network.bridge.name driver option to not create a bridge if there is an user-managed bridge already present.

@aboch wdyt ?

[dreamcat4]

If the user doesn't specify these IPAM options while creating a network, then the IPAM driver will choose any available network subnet that may not match the user-managed bridge.

Yes indeed. They dicover the behaviour those --subnet and --gateway options now. Although maybe it could be better in some proper documentation / which I cannot seem to find just yet. We may also be concerned for other driver options which we do not fully know. For example if the driver is also overriding other settings of the bridge.

[mavenugo]

We may also be concerned for other driver options which we do not fully know. For example if the driver is also overriding other settings of the bridge.

@dreamcat4 sure. and hence my response :

Other than that, I don't see any other issue with using the com.docker.network.bridge.name driver option to not create a bridge if there is an user-managed bridge already present.

[gedl]

@thomas-maurice I have the exact same requirement. Have you got anywhere with this?

@mavenugo the problem with using com.docker.network.bridge.name=none is that the container won't have any means to reach the network, as the network doesn't have a device. I would expect com.docker.network.bridge.name to have the same behaviour as the -b, which it doesn't, and leaves no other options to this use case. Theoretically speaking, that is, I haven't tried but I'm pretty sure the above would create a bridge called "none".

A decent alternative would be to allow to specify a static IP address for pre-defined networks, as I commented on #19001 , making use of the -b daemon switch.

[thomas-maurice]

@gedl nope. I ended up not using docker networks at all and doing my own networking cooking at the OS level

[gedl]

@thomas-maurice I am almost tempted to just remove the "if" that prevents users from specifying an IP in a pre-defined network. My opinion is this restriction is intrusive and useless. Docker can't prevent every user-induced mess up, specially not without reducing functionality and options, like this is the case.

I still have some hope @aboch will reconsider and lobby to remove the restriction (that's definitely easier than allowing a -b switch on network creation).

[gedl]

@thomas-maurice were you at least able to delete the intruding default bridge network? even specifying -b=none is still creating me a useless default bridge network, occupying a range of IPs. It does not create the bridge device, but prevents me to use that range of IPs in my own networks.

[thomas-maurice]

Well I used the default bridge with address I forcedly assign, and then built my own networking stack on the top of it.

[gedl]

I just assigned a /30 range (the smaller it accepts). So annoying having to
remember that those address can't be used anywhere else in the network.

I managed to delete the default network once, in another installation, but
I can't remember how I did it. This was great because with -b=none there
was no leftovers of the default networking (which are useless at best in my
use case).

Gonçalo Luiz

On 20 May 2016 at 06:53, Thomas Maurice notifications@github.com wrote:

Well I used the default bridge with address I forcedly assign, and then
built my own networking stack on the top of it.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#20349 (comment)

[aboch]

@thomas-maurice
Regarding

, and also not be deleted if I decide to destroy the network ?

@mountkin 's change at moby/libnetwork#1301 provides the above and will be available in docker 1.13