Linux cluster does not support UDP when using a container host #208

Closed
vjrantal opened this Issue Mar 24, 2017 · 4 comments

Comments

Projects
None yet
4 participants
@vjrantal

Steps to reproduce (assumes you have a connection to a Linux cluster opened with something like: azure servicefabric cluster connect ...):

$ git clone https://github.com/vjrantal/CaliforniumContainer
$ cd CaliforniumContainer
$ ./install.sh

In the Azure Load Balancer configuration, create a rule to allow also UDP traffic so that the result is something like this:

app-rules

First verify that the TCP connection works with something like this:

$ telnet <azure-load-balancer-public-ip> 5683
// This works and shows the current date, because the code in the Docker container accepts TCP sockets and returns the current date

Then try to use UDP with something like this:

$ npm install coap-cli -g
$ coap get coap://<azure-load-balancer-public-ip>:5683
// Here expected result would be a response, but actual result is a timeout after 30 seconds or so

The root cause seems to be that if you SSH into the node where the app ends up being deployed, you can see that docker-proxy is started with only TCP protocol support:

root     61069  0.0  0.1 109176  3632 pts/5    Sl+  05:14   0:00 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 5683 -container-ip 172.17.0.2 -container-port 5683

Also, you can see that the UDP port isn't mapped correctly when docker is run:

$ sudo docker ps
CONTAINER ID        IMAGE                           COMMAND                  CREATED             STATUS              PORTS                              NAMES
7410bf203822        vjrantal/cf-helloworld-server   "java -jar cf-hellowo"   2 hours ago         Up 2 hours          5683/udp, 0.0.0.0:5683->5683/tcp   sf-1-f683a246-b089-ec41-b663-e5264237e1cb

As a fix proposal, a good idea might be to allow explicitly requesting UDP routing in the ServiceManifest.xml by allowing something like this:

  <Resources>
    <Endpoints>
      <Endpoint Name="Endpoint1" Port="5683" Protocol="udp" />
      <Endpoint Name="Endpoint1" Port="5683" Protocol="tcp" />
    </Endpoints>
  </Resources>

Currently, Protocol="udp" is not allowed and throws an error.

@mani-ramaswamy

This comment has been minimized.

Show comment
Hide comment
@mani-ramaswamy

mani-ramaswamy Mar 24, 2017

Thanks for reporting this. Adding @RajeetN, and we'll loop back on what we can do here.

Thanks for reporting this. Adding @RajeetN, and we'll loop back on what we can do here.

@mani-ramaswamy

This comment has been minimized.

Show comment
Hide comment
@mani-ramaswamy

mani-ramaswamy Mar 24, 2017

We've opened a work item to track this - we'll let you know as soon as a fix is on the way.

We've opened a work item to track this - we'll let you know as soon as a fix is on the way.

@mani-ramaswamy mani-ramaswamy added bug and removed enhancement labels Mar 24, 2017

@vjrantal

This comment has been minimized.

Show comment
Hide comment
@vjrantal

vjrantal Mar 28, 2017

@mani-ramaswamy Thanks for confirming this issue is being looked at!

I wanted to mention that I realized this same limitation also affects Windows clusters. The current endpoint definition only allows the load balancer to be configured for TCP rules. I was able to make UDP traffic flow E2E when using guest executables on Windows cluster, but that requires manually configuring a load balancer rule for the UDP traffic.

So to summarize, the proposed syntax:

    <Endpoints>
      <Endpoint Name="Endpoint1" Port="5683" Protocol="udp" />
      <Endpoint Name="Endpoint1" Port="5683" Protocol="tcp" />
    </Endpoints>

should be support on both Windows and Linux clusters and the title of this issue could be updated to reflect that this issue is not only in case of Linux and containers.

@mani-ramaswamy Thanks for confirming this issue is being looked at!

I wanted to mention that I realized this same limitation also affects Windows clusters. The current endpoint definition only allows the load balancer to be configured for TCP rules. I was able to make UDP traffic flow E2E when using guest executables on Windows cluster, but that requires manually configuring a load balancer rule for the UDP traffic.

So to summarize, the proposed syntax:

    <Endpoints>
      <Endpoint Name="Endpoint1" Port="5683" Protocol="udp" />
      <Endpoint Name="Endpoint1" Port="5683" Protocol="tcp" />
    </Endpoints>

should be support on both Windows and Linux clusters and the title of this issue could be updated to reflect that this issue is not only in case of Linux and containers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment