Address already assigned in block after restart docker #746
Assignees
Labels
Comments
liuxu623
changed the title
|
Sounds like a change in the behavior of docker. It would be helpful if you could please do the following. Attach logs from CentOS and Ubuntu. I'd like to see the network creation, creation of the container, the restart of the docker service, and the restart of the workload. And the specific commands you are using. Could you please provide the docker version that your Ubuntu setup is using. I'm guessing the docker on Ubuntu is not re-requesting the IP assignment where it is on CentOS, but that is just a guess. |
CentOS |
Ubuntu |
|
Ubuntu have the same problem now. |
|
I think dockerd couldn't connect calico when dockerd restart led to this problem. |
|
I tried not using the docker to start calico,it work.And docker start fast than before. |
|
Why you use docker to start calico? |
|
You can setup and run the processes that run in the calico/node container outside of docker, I don't believe there is anything restricting you from doing that. Docker is used to start calico because it provides an easy mechanism for distribution that bundles the needed dependencies. |
|
Sounds like you've figured out the issue here so I'm going to close this issue. If there is more we can help you with feel free to open another issue, check out https://www.projectcalico.org//community, or join us on https://calicousers.slack.com/ |
|
@ansiz Just copy calico binary and config file from container to host , and use systemd manage calico process , you can see https://github.com/laincloud/lain/tree/master/playbooks/roles/calico, but it works on calico 2.1. |
|
@liuxu623 Thanks a lot, I have successfully run all the services outside the container. But if I assigned a calico network for a container which specified In order to successfully launch the docker service, I must run dockerd manually without Have you encountered a similar situation? |
|
@ansiz Maybe docker.service start before calico-xxx.service，you can modify docker.service file，make docker.service start after calico-xxx.service. You should look dockerd's log. If dockerd start succeed , but container start failed, maybe you should use calicoctl release ip address. |
I have configured service dependencies in systemd: And even though I didn't stop other services, Docker still can't restart. Here is the systemctl log of Dockerno error or warning and won't timeout The manual execution log: |
please cat /usr/lib/systemd/system/docker.service |
|
|
@liuxu623 Not caused by the Docker service, but a problem with calico-libnetwork.service，the Docker service can start normally after restarting the calico-libnetwork service, container will be stoped and here is error log: I think it should be caused by a calico-libnetwork data error. I will continue to investigate the issue Thanks for your reply. |
|
@liuxu623 I finally found the problem after reading the source code.(I'm using v1.1.3) // Use the Docker API to fetch the network name (so we don't have to use an ID everywhere)
dockerCli, err := dockerClient.NewEnvClient()
if err != nil {
err = errors.Wrap(err, "Error while attempting to instantiate docker client from env")
log.Errorln(err)
return nil, err
}
defer dockerCli.Close()
networkData, err := dockerCli.NetworkInspect(context.Background(), request.NetworkID)
if err != nil {
err = errors.Wrapf(err, "Network %v inspection error", request.NetworkID)
log.Errorln(err)
return nil, err
}When the Docker service starts, the container will start at the same time, it requests new endpoint via libnetwork-plugin, the libnetwork-plugin use the Docker API to fetch the network name, but the Docker service has not yet started...This will lead to a deadlock. I temporarily use Here is the diff code: ansiz/libnetwork-plugin@3f052ae |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Only in CentOS have this problem,Ubuntu does not have this problem.CentOS Linux release 7.3.1611 (Core)
docker 17.04.0-ce
calicoctl 1.1.3
http://docs.projectcalico.org/v2.1/getting-started/docker/tutorials/ipam
The text was updated successfully, but these errors were encountered: