Cannot connect docker container to another network

[azyobuzin]

Bug report

General Information

• Cilium version (run cilium version)
  • both 1.8.3 and 1.9.0-rc0 (1.8.90)
• Kernel version (run uname -a)
  • Linux ubuntu-18 4.15.0-106-generic #107-Ubuntu SMP Thu Jun 4 11:27:52 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
• Orchestration system version in use (e.g. kubectl version, Mesos, ...)
  • Docker 19.03.12
• Link to relevant artifacts (policies, deployments scripts, ...)
• Generate and upload a system zip:
  • All files in the zip are empty

How to reproduce the issue

1. Launch virtual machine with getting-started
2. Create networks
   • docker network create --driver cilium --ipam-driver cilium cilium-net
   • docker network create another-bridge
3. Launch a container connected to cilium-net
   • docker run -d --name app1 --net cilium-net cilium/demo-httpd
4. Connect the container to another bridge network
   • docker network connect another-bridge app1

Then, I got the following message.

Error response from daemon: failed to set gateway while updating gateway: file exists

The reverse order also results in an error.

$ docker run -d --name app1 --net another-bridge cilium/demo-httpd
$ docker network connect cilium-net app1
Error response from daemon: failed to add static route 0.0.0.0/0: file exists

Expected behavior

The container should be connected to both cilium-net and another-bridge.

I think the comment in driver.go is relevant to this issue.

cilium/plugins/cilium-docker/driver/driver.go

Lines 510 to 519 in f55458f

	//GatewayIPv4: driver.gatewayIPv4,
	}
	// FIXME? Having the following code results on a runtime error: docker: Error
	// response from daemon: oci runtime error: process_linux.go:334: running prestart
	// hook 0 caused "exit status 1: time=\"2016-10-26T06:33:17-07:00\" level=fatal
	// msg=\"failed to set gateway while updating gateway: file exists\" \n"
	//
	// If empty, it works as expected without docker runtime errors
	// res.Gateway = connector.IPv4Gateway(addr)

[aanm]

@azyobuzin thanks for opening the issue. What's the use case to have 2 networks in a single container?

[azyobuzin]

What's the use case to have 2 networks in a single container?

Thanks. I have two use cases in mind.

• Communicating with containers outside of the Cilium network. It is needed during migrating containers to Cilium network.
• Publishing ports of containers with --publish option using another bridge network because Cilium Docker plugin does not support that option.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[stale]

This issue has not seen any activity since it was marked stale. Closing.