Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DDS within docker containers on different hosts #768

Closed
dhet opened this issue Nov 26, 2017 · 5 comments

Comments

Projects
None yet
3 participants
@dhet
Copy link

commented Nov 26, 2017

My plan is to deploy individual DDS participants within docker containers distributed on separate hosts. The hosts are connected via LAN. I'm currently using the standard messenger example for testing.
The example works perfectly fine when I run the participants on different hosts without containers. However, as soon as I deploy one of the participants within a container it doesn't work anymore: the participants can't find each other. The problem is that docker containers need to specify which ports to expose to the host. My idea was to set the discovery ports in the config file and expose a port range in the container.
My config file looks like this

[common]
DCPSGlobalTransportConfig=$file
DCPSDefaultDiscovery=discoveryConf1

[rtps_discovery/discoveryConf1]
PB=8000
DG=10

[transport/transportConf1]
transport_type=rtps_udp

In my example this causes the participants to use ports 8420, 8422 and 8430 for service discovery. So I thought it would be enough to expose the port range from 8400 to 8500 like so:
sudo docker run -p 8400-8500 -ti -v $(pwd):/opt/workspace kugele1337/qemu-raspbian-opendds

Unfortunately, this doesn't change anything. The participants still can't find each other. What am I missing?

Update: I just realized I probably need docker swarm or kubernetes for this

@mitza-oci mitza-oci added the question label Nov 27, 2017

@mitza-oci

This comment has been minimized.

Copy link
Member

commented Nov 27, 2017

When you run on different hosts without containers, get a packet capture using a tool like tcpdump and make sure all the required ports are covered by your configuration. Debug logging will also show you which ports are being used.

@jrw972

This comment has been minimized.

Copy link
Contributor

commented Nov 27, 2017

RTPS discovery uses multicast which is not "well" supported in Docker. See moby/moby#3043 for a discussion. The TLDR is

  1. Container to container multicast on the same host is supported out of the box.
  2. Otherwise, you need to use --net=host which has its own downsides.

As an alternative, you can use the InfoRepo for discovery. We have explored using InfoRepo for discovery in Kubernetes. If you are interested in this approach, let us know and we can make some documentation available to you.

@dhet

This comment has been minimized.

Copy link
Author

commented Nov 28, 2017

Thanks for your input. --network=host does indeed solve the problem. However, it is debatable whether it is sensible to make such security sacrifices. I will look further into container orchestration platforms and report back. I would love to hear about your attempts to use Kubernetes.

@jrw972

This comment has been minimized.

Copy link
Contributor

commented Nov 30, 2017

@dhet I've created a repo with an example of running the Messenger example in Kubernetes.
https://github.com/jrw972/KubernetesMessenger

@dhet

This comment has been minimized.

Copy link
Author

commented Dec 6, 2017

Thank you @jrw972. I like the script to modify the config files... quite creative 😄
I've spent some time trying to get cross host / cross container multicast to work with swarm. I've tested several network setups using different network drivers:

None worked. Multicast support seems to be a much requested feature in the docker community: docker/libnetwork/issues/552

@jrw972 jrw972 closed this Dec 14, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.