New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The docker daemon API lets you create two networks with the same name #18864

Open
vikstrous opened this Issue Dec 23, 2015 · 19 comments

Comments

Projects
None yet
9 participants
@vikstrous
Contributor

vikstrous commented Dec 23, 2015

The docker daemon API lets you create two networks with the same name. Having two networks with the same name breaks all kinds of things. It seems like the CLI client checks before creating a second network, but this is still a race condition. This should be prevented in the daemon.

docker version
Client:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.5.1
 Git commit:   a34a1d5-dirty
 Built:        Sun Nov 22 00:15:15 UTC 2015
 OS/Arch:      linux/amd64

Server:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.5.1
 Git commit:   a34a1d5-dirty
 Built:        Sun Nov 22 00:15:15 UTC 2015
 OS/Arch:      linux/amd64
@bboreham

This comment has been minimized.

Show comment
Hide comment
@bboreham

bboreham Dec 23, 2015

Contributor

@vikstrous did you notice there is a field CheckDuplicate in the API ?

https://docs.docker.com/engine/reference/api/docker_remote_api_v1.21/#create-a-network

I'd agree that having it default off is surprising.

Contributor

bboreham commented Dec 23, 2015

@vikstrous did you notice there is a field CheckDuplicate in the API ?

https://docs.docker.com/engine/reference/api/docker_remote_api_v1.21/#create-a-network

I'd agree that having it default off is surprising.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah
Member

thaJeztah commented Dec 23, 2015

ping @mavenugo @mrjana wdyt?

@vikstrous

This comment has been minimized.

Show comment
Hide comment
@vikstrous

vikstrous Dec 23, 2015

Contributor

Ah, okay, I did not see that. I can't think of a case where you would want to create two networks with the same name though. It clearly doesn't work.

Contributor

vikstrous commented Dec 23, 2015

Ah, okay, I did not see that. I can't think of a case where you would want to create two networks with the same name though. It clearly doesn't work.

@bboreham

This comment has been minimized.

Show comment
Hide comment
@bboreham

bboreham Dec 23, 2015

Contributor

What are the specific steps to demonstrate it not working? Do you have a test program?

You have to use the hex ID to refer unambiguously to a network, but as I recall it did work when I tried it.

Contributor

bboreham commented Dec 23, 2015

What are the specific steps to demonstrate it not working? Do you have a test program?

You have to use the hex ID to refer unambiguously to a network, but as I recall it did work when I tried it.

@mrjana

This comment has been minimized.

Show comment
Hide comment
@mrjana

mrjana Dec 23, 2015

Contributor

@vikstrous @bboreham Allowing duplicate names is by design. Because network is primarily keyed based on a random ID and not on the name. The name is strictly a user-friendly alias to the network which is uniquely identified using ID. The checkDuplicate option is just there to provide a best effort checking of any networks which has the same name but it is not guaranteed to catch all name collisions. That is why it is not a default.

Contributor

mrjana commented Dec 23, 2015

@vikstrous @bboreham Allowing duplicate names is by design. Because network is primarily keyed based on a random ID and not on the name. The name is strictly a user-friendly alias to the network which is uniquely identified using ID. The checkDuplicate option is just there to provide a best effort checking of any networks which has the same name but it is not guaranteed to catch all name collisions. That is why it is not a default.

@vikstrous

This comment has been minimized.

Show comment
Hide comment
@vikstrous

vikstrous Dec 24, 2015

Contributor

Hmm... I had no idea that you can use the network's id (or that networks have IDs) when creating a container. We were using the name, so if a new network was created with the same name it would make the original network inaccessible. It looks like this is all intended behaviour and we should be using network IDs.

Imagine a tool that needs to run a container on a specific network and if it doesn't exist, create it. It can create it only by name, but if it creates a container on that network while referring to it by name, there's no guarantee that it's the same network it created. So it needs to store state about the network ID it created and somehow communicate this to other tools that need to use the same network.

I think it's useful to be able to create a network using some unique ID, but all of this behavior seems intentional, so you can close this issue.

Contributor

vikstrous commented Dec 24, 2015

Hmm... I had no idea that you can use the network's id (or that networks have IDs) when creating a container. We were using the name, so if a new network was created with the same name it would make the original network inaccessible. It looks like this is all intended behaviour and we should be using network IDs.

Imagine a tool that needs to run a container on a specific network and if it doesn't exist, create it. It can create it only by name, but if it creates a container on that network while referring to it by name, there's no guarantee that it's the same network it created. So it needs to store state about the network ID it created and somehow communicate this to other tools that need to use the same network.

I think it's useful to be able to create a network using some unique ID, but all of this behavior seems intentional, so you can close this issue.

@vikstrous

This comment has been minimized.

Show comment
Hide comment
@vikstrous

vikstrous Dec 24, 2015

Contributor

If the default in the docker client is to not allow creating two networks in the same name, maybe it should also be the default in the API and/or the dockerclient library. I guess this is something all libraries have to fix?

Contributor

vikstrous commented Dec 24, 2015

If the default in the docker client is to not allow creating two networks in the same name, maybe it should also be the default in the API and/or the dockerclient library. I guess this is something all libraries have to fix?

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Dec 24, 2015

Member

I still think we should look into this, and discuss if this is the desired behavior. I haven't tested yet what happens if I create two networks with the same name through the API, but from the docker CLI, there's no way to identify which network a container is connected to.

For example;

This container joined network foobar:

"Networks": {
            "foobar": {
                "EndpointID": "6dca6167ae2896c2276171b39fec35ba527b6deffa5521cb34ab188125944146",
                "Gateway": "172.18.0.1",
                "IPAddress": "172.18.0.2",
                "IPPrefixLen": 16,
                "IPv6Gateway": "",
                "GlobalIPv6Address": "",
                "GlobalIPv6PrefixLen": 0,
                "MacAddress": "02:42:ac:12:00:02"
            }
        }

And this one joined the same network through its ID

  "Networks": {
            "foobar": {
                "EndpointID": "5e5cd696f66bfeddea4b26b8466b9bb32b973be10b439e9c6094249fc40d821f",
                "Gateway": "172.18.0.1",
                "IPAddress": "172.18.0.3",
                "IPPrefixLen": 16,
                "IPv6Gateway": "",
                "GlobalIPv6Address": "",
                "GlobalIPv6PrefixLen": 0,
                "MacAddress": "02:42:ac:12:00:03"
            }
        }

The unique identifier of the network is not visible in the output here

Member

thaJeztah commented Dec 24, 2015

I still think we should look into this, and discuss if this is the desired behavior. I haven't tested yet what happens if I create two networks with the same name through the API, but from the docker CLI, there's no way to identify which network a container is connected to.

For example;

This container joined network foobar:

"Networks": {
            "foobar": {
                "EndpointID": "6dca6167ae2896c2276171b39fec35ba527b6deffa5521cb34ab188125944146",
                "Gateway": "172.18.0.1",
                "IPAddress": "172.18.0.2",
                "IPPrefixLen": 16,
                "IPv6Gateway": "",
                "GlobalIPv6Address": "",
                "GlobalIPv6PrefixLen": 0,
                "MacAddress": "02:42:ac:12:00:02"
            }
        }

And this one joined the same network through its ID

  "Networks": {
            "foobar": {
                "EndpointID": "5e5cd696f66bfeddea4b26b8466b9bb32b973be10b439e9c6094249fc40d821f",
                "Gateway": "172.18.0.1",
                "IPAddress": "172.18.0.3",
                "IPPrefixLen": 16,
                "IPv6Gateway": "",
                "GlobalIPv6Address": "",
                "GlobalIPv6PrefixLen": 0,
                "MacAddress": "02:42:ac:12:00:03"
            }
        }

The unique identifier of the network is not visible in the output here

@bboreham

This comment has been minimized.

Show comment
Hide comment
@bboreham

bboreham Dec 24, 2015

Contributor

@mrjana is this "best effort" nature of checkDuplicate documented anywhere?

The API docs say "Requests daemon to check for networks with same name".

Contributor

bboreham commented Dec 24, 2015

@mrjana is this "best effort" nature of checkDuplicate documented anywhere?

The API docs say "Requests daemon to check for networks with same name".

@thockin

This comment has been minimized.

Show comment
Hide comment
@thockin

thockin Dec 25, 2015

Contributor

Is there actually value to this semantic? Enough value ton outweigh the
obvious confusion? Containers don't allow the same name to be used more
than once, why should networks?
On Dec 24, 2015 6:28 AM, "Bryan Boreham" notifications@github.com wrote:

@mrjana https://github.com/mrjana is this "best effort" nature of
checkDuplicate documented anywhere?

The API docs say "Requests daemon to check for networks with same name".


Reply to this email directly or view it on GitHub
#18864 (comment).

Contributor

thockin commented Dec 25, 2015

Is there actually value to this semantic? Enough value ton outweigh the
obvious confusion? Containers don't allow the same name to be used more
than once, why should networks?
On Dec 24, 2015 6:28 AM, "Bryan Boreham" notifications@github.com wrote:

@mrjana https://github.com/mrjana is this "best effort" nature of
checkDuplicate documented anywhere?

The API docs say "Requests daemon to check for networks with same name".


Reply to this email directly or view it on GitHub
#18864 (comment).

@WeiZhang555

This comment has been minimized.

Show comment
Hide comment
@WeiZhang555

WeiZhang555 Dec 25, 2015

Contributor

I feel like @vikstrous is right, we should have CheckDuplicate default on because it's more rational to NOT allow networks to have same name, that may bring more confusions.
This is a design problem, easy to modify in code.

Contributor

WeiZhang555 commented Dec 25, 2015

I feel like @vikstrous is right, we should have CheckDuplicate default on because it's more rational to NOT allow networks to have same name, that may bring more confusions.
This is a design problem, easy to modify in code.

@mrjana

This comment has been minimized.

Show comment
Hide comment
@mrjana

mrjana Dec 25, 2015

Contributor

@vikstrous If we have to make the API and cli consistent then they both should not check for duplicates because there is no guaranteed way to check for duplicates across a cluster of docker hosts. This is because the network object is stored by design with only the network ID as the key and name is just a user friendly alias. This schema allows in the future to assign multiple aliases to the same network.

Contributor

mrjana commented Dec 25, 2015

@vikstrous If we have to make the API and cli consistent then they both should not check for duplicates because there is no guaranteed way to check for duplicates across a cluster of docker hosts. This is because the network object is stored by design with only the network ID as the key and name is just a user friendly alias. This schema allows in the future to assign multiple aliases to the same network.

@mrjana

This comment has been minimized.

Show comment
Hide comment
@mrjana

mrjana Dec 25, 2015

Contributor

@thaJeztah If the network ID does not show up in the inspect then we should fix that and that is all is needed to make the two networks usable (or avoid confusion)

Contributor

mrjana commented Dec 25, 2015

@thaJeztah If the network ID does not show up in the inspect then we should fix that and that is all is needed to make the two networks usable (or avoid confusion)

@mrjana

This comment has been minimized.

Show comment
Hide comment
@mrjana

mrjana Dec 25, 2015

Contributor

@bboreham I will check the API docs later but if it doesn't mention that checkDuplicate is only best effort then that should be fixed in the docs.

Contributor

mrjana commented Dec 25, 2015

@bboreham I will check the API docs later but if it doesn't mention that checkDuplicate is only best effort then that should be fixed in the docs.

@mrjana

This comment has been minimized.

Show comment
Hide comment
@mrjana

mrjana Dec 25, 2015

Contributor

@thockin The reason we have this semantic is because we want to adopt a schema which can pave the way in the future for a completely decentralized cluster of docker hosts (if scalability is needed). Already network objects are immutable for this reason(you can only create and delete networks but not modify). And because of the schema where the network is only truly identifiable by it's ID we can in the future cut over to a completely partition tolerant and decentralized cluster of docker hosts. That's the value.

Of course a stateful higher level orchestrator can implement a centralized solution and check for name collisions if it wants to.

Contributor

mrjana commented Dec 25, 2015

@thockin The reason we have this semantic is because we want to adopt a schema which can pave the way in the future for a completely decentralized cluster of docker hosts (if scalability is needed). Already network objects are immutable for this reason(you can only create and delete networks but not modify). And because of the schema where the network is only truly identifiable by it's ID we can in the future cut over to a completely partition tolerant and decentralized cluster of docker hosts. That's the value.

Of course a stateful higher level orchestrator can implement a centralized solution and check for name collisions if it wants to.

@mrjana

This comment has been minimized.

Show comment
Hide comment
@mrjana

mrjana Dec 25, 2015

Contributor

@WeiZhang555 See the above comments for the reasons on why this design choice was made intentionally.

Contributor

mrjana commented Dec 25, 2015

@WeiZhang555 See the above comments for the reasons on why this design choice was made intentionally.

@vikstrous

This comment has been minimized.

Show comment
Hide comment
@vikstrous

vikstrous Dec 25, 2015

Contributor

@mrjana This sounds reasonable. Since you understand the problem well, you should make individual issues for each of the things discussed so we can make sure that the cli, api and docs are all consistent and accurate.

Contributor

vikstrous commented Dec 25, 2015

@mrjana This sounds reasonable. Since you understand the problem well, you should make individual issues for each of the things discussed so we can make sure that the cli, api and docs are all consistent and accurate.

@doronp

This comment has been minimized.

Show comment
Hide comment
@doronp

doronp Feb 25, 2016

Contributor

@thockin IMHO there is no reason for container names to be unique as well. Supporting Mutli tenancy for example will have to change that won't it?

Contributor

doronp commented Feb 25, 2016

@thockin IMHO there is no reason for container names to be unique as well. Supporting Mutli tenancy for example will have to change that won't it?

@thanatos

This comment has been minimized.

Show comment
Hide comment
@thanatos

thanatos Oct 12, 2017

I'm going to echo @thockin here: Is there actually value to this semantic?

I have the use case of having a couple containers that should all be run in the same network. I feel like this should be an immensely common use case, too; when those containers are started, presently, they need to possibly create, and somehow know which network to connect to. The name would be the perfect thing to use, and extremely human-readable, but Docker needs to guarantee uniqueness, or at least some means of atomically creating a network of a particular name. Presently, it guarantees neither.

The CLI's present behavior of disallowing some duplicates due to a race condition lulls users like me into believing that network names are unique, since it errors out if you attempt to create duplicate networks slowly.

Docker's current behavior means that I (and I presume many others) need to invent some external means of figuring out what network to attach containers to, and locking around that network's creation. This is just plain annoying.

thanatos commented Oct 12, 2017

I'm going to echo @thockin here: Is there actually value to this semantic?

I have the use case of having a couple containers that should all be run in the same network. I feel like this should be an immensely common use case, too; when those containers are started, presently, they need to possibly create, and somehow know which network to connect to. The name would be the perfect thing to use, and extremely human-readable, but Docker needs to guarantee uniqueness, or at least some means of atomically creating a network of a particular name. Presently, it guarantees neither.

The CLI's present behavior of disallowing some duplicates due to a race condition lulls users like me into believing that network names are unique, since it errors out if you attempt to create duplicate networks slowly.

Docker's current behavior means that I (and I presume many others) need to invent some external means of figuring out what network to attach containers to, and locking around that network's creation. This is just plain annoying.

suzuki-shunsuke added a commit to suzuki-shunsuke/pipeline that referenced this issue Jun 5, 2018

Support docker network's CheckDuplicate
https://godoc.org/github.com/docker/docker/api/types#NetworkCreate
moby/moby#18864

Allow to set Docker network's CheckDuplicate option
to prevent name's collisions.

suzuki-shunsuke added a commit to suzuki-shunsuke/drone-cli that referenced this issue Jun 5, 2018

Support docker network's CheckDuplicate at drone exec
https://godoc.org/github.com/docker/docker/api/types#NetworkCreate
moby/moby#18864

Allow to set Docker network's CheckDuplicate option
to prevent name's collisions.
drone exec executes a local build, so make CheckDuplicate true is natural.

Change of Dependencies will fix after the following Pull Request has been merged.

cncd/pipeline#44

suzuki-shunsuke added a commit to suzuki-shunsuke/drone-cli that referenced this issue Jun 5, 2018

Support docker network's CheckDuplicate at drone exec
https://godoc.org/github.com/docker/docker/api/types#NetworkCreate
moby/moby#18864

Allow to set Docker network's CheckDuplicate option
to prevent name's collisions.
drone exec executes a local build, so make CheckDuplicate true is natural.

I will fix change of Dependencies after the following Pull Request has been merged.

cncd/pipeline#44

suzuki-shunsuke added a commit to suzuki-shunsuke/drone-cli that referenced this issue Jun 5, 2018

Support docker network's CheckDuplicate at drone exec
https://godoc.org/github.com/docker/docker/api/types#NetworkCreate
moby/moby#18864

Allow to set Docker network's CheckDuplicate option
to prevent name's collisions.
drone exec executes a local build, so make CheckDuplicate true is natural.

Change of Dependencies should be fixed after the following Pull Request has been merged.

cncd/pipeline#44

suzuki-shunsuke added a commit to suzuki-shunsuke/drone-cli that referenced this issue Jun 5, 2018

Support docker network's CheckDuplicate at drone exec
https://godoc.org/github.com/docker/docker/api/types#NetworkCreate
moby/moby#18864

Set Docker network's CheckDuplicate option to prevent name's collisions
at drone exec command.
drone exec executes a local build, so make CheckDuplicate true is natural.

Change of Dependencies should be fixed after the following Pull Request has been merged.

cncd/pipeline#44

suzuki-shunsuke added a commit to suzuki-shunsuke/drone-cli that referenced this issue Jun 5, 2018

Support docker network's CheckDuplicate at drone exec
https://godoc.org/github.com/docker/docker/api/types#NetworkCreate
moby/moby#18864

Set Docker network's CheckDuplicate option to prevent name's collisions
at drone exec command.
drone exec executes a local build,
so it is natural to make CheckDuplicate true.

Change of Dependencies should be fixed after the following Pull Request has been merged.

cncd/pipeline#44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment