Error response from daemon: configuration network 8-2 is in use #35101
Comments
|
Further info when trying to remove network... |
|
I was also having this issue while looking at #35099, and it looks to be related to a service using the network and failing to be creating due to an issue with the network's configuration Steps to reproduce; 1. Create config-only network, and create a network using that:2. Create a service using that networkThis fails because the $ docker service create --detach=false --network name=private-10-0-48,alias=${HOSTNAME} --replicas 1 --name bla nginx:alpine
yc8civv5wnnlc0bflymw2povj
overall progress: 0 out of 1 tasks
1/1: invalid subinterface vlan name eno1, example formatting is eth0.10
^COperation continuing in background.
Use `docker service ps yc8civv5wnnlc0bflymw2povj` to check progress.3. Remove the service$ docker service rm bla4. Cleanup the networksAfter running the above, I tried to cleanup the unused networks. Here's the list of networks before: $ docker network ls
NETWORK ID NAME DRIVER SCOPE
dc9b63fba648 10-0-48 null local
400ef4d0237b bridge bridge local
8b8af4f1eb84 docker_gwbridge bridge local
d4d040849154 edunet bridge local
57d193a9db9e foo bridge local
7f90f1113d26 foo2 bridge local
13cf280a1bbf host host local
yl7ch581l0tn ingress overlay swarm
d9e4c03728a0 none null local
t1j31pmmjvwg private-10-0-48 macvlan swarm
3ba0af34b2e5 testing bridge localRun $ docker network prune
WARNING! This will remove all networks not used by at least one container.
Are you sure you want to continue? [y/N] y
Deleted Networks:
private-10-0-48
edunet
testing
foo2
fooPruning unused networks removed the $ docker network ls
NETWORK ID NAME DRIVER SCOPE
dc9b63fba648 10-0-48 null local
400ef4d0237b bridge bridge local
8b8af4f1eb84 docker_gwbridge bridge local
13cf280a1bbf host host local
yl7ch581l0tn ingress overlay swarm
d9e4c03728a0 none null localAttempting to manually remove the config-only network doesn't work: $ docker network rm dc9b63fba648
Error response from daemon: configuration network "10-0-48" is in useInspecting the network (including $ docker network inspect -v dc9b63fba648
[
{
"Name": "10-0-48",
"Id": "dc9b63fba648567d7145f84dde8169ac2a1c2952f56c10295912d66b3346f9d6",
"Created": "2017-10-06T10:37:06.818391382Z",
"Scope": "local",
"Driver": "null",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": {},
"Config": [
{
"Subnet": "10.0.48.0/22",
"Gateway": "10.0.48.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": true,
"Containers": {},
"Options": {
"parent": "eno1"
},
"Labels": {}
}
]The logs also don't show anything useful (no reason for the error is printed): 5. Additional informationThis situation looks to be related to the service failing to be created. Only creating the network, then removing it does not reproduce the issue: $ docker network create -d macvlan --subnet 10.0.49.0/22 --gateway 10.0.49.1 -o parent=eno2 --config-only vlannie-config
$ docker network create -d macvlan --scope swarm --config-from foo-config --attachable vlannieThen prune the network: $ docker network prune
WARNING! This will remove all networks not used by at least one container.
Are you sure you want to continue? [y/N] y
Deleted Networks:
vlannieAnd remove the config-only network: $ docker network rm vlannie-config
vlannie-config |
thaJeztah
added
kind/bug
|
It looks like the problem is when we delete the service, I get this error: I get this error although there is is it relevant? ... |
|
removed spam comment |
|
@daledude was there a reason you closed this one? If it's not resolved, I'd like to reopen this one for investigation |
|
Confirming this still exists in the latest docker versions. |
|
Additonal behaviour related to this issue. I am trying to remove the config network after having added the config network and a swarm macvlan network based on it. I discovered that swarm doesn't support static IP's for macvlan which renders swarm useless for my usecase, so I need to recreate this network locally to launch local containers with the required static IPs, and it's now saying I cannot create a network from this config network because the parent interface is in use by a non existent network which I presume is some sort of identifier for the now deleted swarm network: |
|
This error seems related: moby/libnetwork#1743 |
|
+1 |
|
I am also experiencing this issue. Stopping docker, deleting |
The endpoint count for --config-only networks was being incremented even when the respective --config-from inherited network was failing to create a network This was due to a variable shadowing problem with err causing the deferred function to not execute correctly. Using the same err variable across the entire function fixes the issue Fixes: moby/moby#35101 Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
|
Note that there's pull request open to address this; moby/libnetwork#2373 |
|
wow, i just noticed this is still open. |
|
unfortunatly, unless I'm experiencing something else, I'd say it is :( |
|
reopening because moby/libnetwork#2373 was not yet vendored |
|
In what version can change will appear? |
|
Looks like this was included in the libnetwork bump that's part of #39295, which has been cherry-picked into the 19.03 release branch in docker#260, so this is fixed on master, and will be in the 19.03 release |
The endpoint count for --config-only networks was being incremented even when the respective --config-from inherited network failed to create a network This was due to a variable shadowing problem with err causing the deferred function to not execute correctly. Using the same err variable across the entire function fixes the issue Fixes: moby/moby#35101 Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com> (cherry picked from commit eacb56d) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The endpoint count for --config-only networks was being incremented even when the respective --config-from inherited network failed to create a network This was due to a variable shadowing problem with err causing the deferred function to not execute correctly. Using the same err variable across the entire function fixes the issue Fixes: moby/moby#35101 Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com> (cherry picked from commit eacb56d) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diff: moby/libnetwork@e7933d4...55685ba changes included: - moby/libnetwork#2382 Backporting PR 2069 to bump_18.09 - backport of https://github.com/docker/libnetwork#2069 Rolling back the port configs if failed to programIngress() - moby/libnetwork#2363 [18.09] align dependencies with engine 18.09 - moby/libnetwork#2400 [18.09 backport] Fix TestValidRemoteDriver GetCapabilities errors - moby/libnetwork#2391 [18.09 backport] Correctly clean up --config-only networks - backport of moby/libnetwork#2373 - fixes moby#35101 - moby/libnetwork#2392 [18.09 backport] remove gosimple - package is gone and it's not important Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diff: moby/libnetwork@e7933d4...55685ba changes included: - moby/libnetwork#2382 Backporting PR 2069 to bump_18.09 - backport of https://github.com/docker/libnetwork#2069 Rolling back the port configs if failed to programIngress() - moby/libnetwork#2363 [18.09] align dependencies with engine 18.09 - moby/libnetwork#2400 [18.09 backport] Fix TestValidRemoteDriver GetCapabilities errors - moby/libnetwork#2391 [18.09 backport] Correctly clean up --config-only networks - backport of moby/libnetwork#2373 - fixes moby/moby#35101 - moby/libnetwork#2392 [18.09 backport] remove gosimple - package is gone and it's not important Signed-off-by: Sebastiaan van Stijn <github@gone.nl> Upstream-commit: 0a3767c7e9803f0a595a07b0548e99d60e861062 Component: engine
|
@thaJeztah Is there any workaround if I don't have the fix in the version I'm running? EDIT: I'm sadly stuck on |
|
Don't think there's a workaround, but with Docker 18.09 being EOL, I'd highly recommend getting back to a supported version, as there are unpatched vulnerabilities in older versions. Is there a specific reason you can't update? |
|
Sadly yes - Synology DSM is on that version. And now I'm stuck with the daemon in this state. |
|
Hmm... right, so they build their own binaries of docker. You could manually replace the binaries with the static binaries perhaps; https://docs.docker.com/engine/install/binaries/ |
|
That would be a pain because the docker daemon is integrated with a UI component. Anyway - if you can think of a workaround to get myself unblocked, give me a buzz. |
|
good to see I'm not the only one facing this issue (using Docker 18.09.8 on Synology, so cannot upgrade)... Fingers crossed for a solution |
|
On a synology you can stop docker and then delete the file network /volume1/@docker/files/local-kv.db (similar as @SGStino mentioned), after you start docker again the file will be recreated with the standard networks (bridge,host,none) Please NOTE: This will delete ALL your custom networks, but you get rid of that conf network ;-) |
|
There is either a typo, or that path is different on my instance: |
|
you are right: /volume1/@docker/network/files/local-kv.db is the path to the file on the synology docker |
The endpoint count for --config-only networks was being incremented even when the respective --config-from inherited network failed to create a network This was due to a variable shadowing problem with err causing the deferred function to not execute correctly. Using the same err variable across the entire function fixes the issue Fixes: moby#35101 Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
|
I've just experienced this issue on version 20.10.21 on 2 out of 3 nodes in a Docker Swarm. I'm assuming deleting the Deleting the network utilising a config seems to have not released the dependency; now stuck with the config only network. Here's some logs showing that deleting the swarm network has left remnants on one of the nodes Anything I can do? |
|
@thaJeztah according to your comment above this seems to have been fixed in Plenty of other people have also reported the same behaviour in multiple docker versions of Many thanks in advance |
Description
It seems when an IPAM driver fails and the container is removed the network that used the IPAM driver becomes stuck and can no longer be removed. rm -rf /var/lib/docker is necessary (unless some way to edit the .db file?).
Steps to reproduce the issue:
Describe the results you received:
Cannot remove network.
Describe the results you expected:
Network should be removed.
Additional information you deem important (e.g. issue happens only occasionally):
Output of
docker version:Output of
docker info:Additional environment details (AWS, VirtualBox, physical, etc.):
Logs:
The text was updated successfully, but these errors were encountered: