enable broadcast traffic relay by sevenEng · Pull Request #243 · me-box/databox

sevenEng · 2018-02-23T14:56:06Z

together with changes from core-network#bcast to enable broadcast traffic relay to drivers' and apps' networks

another helper service databox-broadcast-relay will be started together with core-network, the host docker network is attached to this new service's container, this way all the interfaces from the host will be mapped into that container. The relay utility within the new service's container will choose the interface whose IP matches the value given to -h to listen, and only relay link local broadcast traffic to the fifo, given by -f.

tested by installing tcpdump into a driver, then capturing and comparing the broadcast traffic with those captured from host network

Toshbrown · 2018-03-20T17:37:25Z

I don't seem to be able to get this to work using docker for mac (not tested on Linux)

everything seems to build fine but the databox-broadcast-relay exits with an error:

[2018-03-20 17:33:50 databox-start]: Starting Databox
w9jp1ivvces9mnjcmrb2ti73p
WARNING: The Docker Engine you're using is running in swarm mode.

Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.

To deploy your application across the swarm, use `docker stack deploy`.

Creating databoxbcast_databox-network_1         ... done
Creating databoxbcast_databox-broadcast-relay_1 ... done
Attaching to databoxbcast_databox-broadcast-relay_1, databoxbcast_databox-network_1
databox-network_1          | starting core-network...
databox-network_1          | 2018-03-20 17:34:04 +00:00: INF [monitor] found 2 existed phy interfaces
databox-network_1          | 2018-03-20 17:34:04 +00:00: INF [monitor] set mtu of eth0 to 4000
databox-network_1          | Netif: plugging into eth0 with mac 02:42:0a:00:00:03
databox-network_1          | Netif: connect eth0
databox-network_1          | 2018-03-20 17:34:04 +00:00: INF [monitor] set mtu of eth1 to 4000
databox-network_1          | Netif: plugging into eth1 with mac 02:42:ac:12:00:03
databox-network_1          | Netif: connect eth1
databox-network_1          | 2018-03-20 17:34:04 +00:00: INF [core] starting interface monitor...
databox-network_1          | 2018-03-20 17:34:05 +00:00: INF [core] starting junction...
databox-network_1          | 2018-03-20 17:34:05 +00:00: INF [policy] allow privileged hostname: arbiter
databox-network_1          | 2018-03-20 17:34:05 +00:00: INF [policy] allow privileged network: 10.0.0.0/24
databox-broadcast-relay_1  | 2018-03-20 17:34:05 +00:00: INF [relay] Opened /tmp/relay for write bcast pkts.
databox-network_1          | 2018-03-20 17:34:05 +00:00: INF [junction] register intf eth0 10.0.0.3 10.0.0.0/24
databox-network_1          | 2018-03-20 17:34:05 +00:00: INF [junction] start local service on eth0...
databox-network_1          | 2018-03-20 17:34:05 +00:00: INF [junction] set gateway for eth1(172.18.0.0/16) to 172.18.0.1
databox-network_1          | 2018-03-20 17:34:05 +00:00: INF [junction] register intf eth1 172.18.0.3 172.18.0.0/16
databox-broadcast-relay_1  | 2018-03-20 17:34:05 +00:00: INF [relay] found 4 existed phy interfaces
databox-broadcast-relay_1  | 2018-03-20 17:34:05 +00:00: ERR [relay] no interface with address 10.154.189.243 found
databox-broadcast-relay_1  | Fatal error: exception (Invalid_argument 10.154.189.243)
databoxbcast_databox-broadcast-relay_1 exited with code 2

sevenEng · 2018-03-20T17:59:18Z

will have a check on this

sevenEng · 2018-03-20T18:39:03Z

inside databox-broadcast-relay, it tries to find the name of the interface whose IP address matches the address that databox is started on, 10.154.189.243 in this case, by pattern matching the output from ip address show: https://github.com/me-box/core-network/blob/bcast/bin/relay.ml#L7-L23

haven't confirmed if the traffic relay works, but the service is not crashing on a linux machine

maybe I missed some cases with my regexp for the output on mac? the error means it cannot find an interface whose IP address is 10.154.189.243 from wihtin the crashing relay container

sevenEng · 2018-03-21T09:45:32Z

@Toshbrown if I can get a dump of ip address show on mac from within that container, I can see if the regexp is the case of the problem, thanks!

Toshbrown · 2018-03-21T09:53:54Z

The ip command is not available on macOS by default. ifconfig is and is already required by the startup scripts. Can you move to using that? The output looks like this on a mac:

[Repeted for each interface]
en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	ether 78:4f:43:83:f0:6f
	inet6 fe80::8f:e346:139c:185a%en0 prefixlen 64 secured scopeid 0x9
	inet 10.154.189.243 netmask 0xfffffc00 broadcast 10.154.191.255
	nd6 options=201<PERFORMNUD,DAD>
	media: autoselect
	status: active

Or you could do it in the start script and pass it through as an env var like you do with the IP. This might be better as the platform specific stuff wont be berried in OCaml code ;-)

Toshbrown · 2018-03-21T15:53:49Z

@sevenEng I've just worked out what you were asking for your using the ip command inside the container, sorry. Here is the dump:

Toshs-MacBook-Pro:databox-bcast tosh$ d run --rm --network host --entrypoint=sh -it core-network-relay
/core-network # ip address show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 02:50:00:00:00:01 brd ff:ff:ff:ff:ff:ff
    inet 192.168.65.3/24 brd 192.168.65.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::3946:41fc:dc11:b5c1/64 scope link
       valid_lft forever preferred_lft forever
3: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1
    link/ipip 0.0.0.0 brd 0.0.0.0
4: ip6tnl0@NONE: <NOARP> mtu 1452 qdisc noop state DOWN group default qlen 1
    link/tunnel6 :: brd ::
5: docker_gwbridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:31:40:94:fb brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.1/16 brd 172.18.255.255 scope global docker_gwbridge
       valid_lft forever preferred_lft forever
    inet6 fe80::42:31ff:fe40:94fb/64 scope link
       valid_lft forever preferred_lft forever
6: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:b2:a8:f5:c8 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:b2ff:fea8:f5c8/64 scope link
       valid_lft forever preferred_lft forever
334: vethf4670df@if333: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker_gwbridge state UP group default
    link/ether 76:e2:76:60:f7:03 brd ff:ff:ff:ff:ff:ff link-netnsid 3
    inet6 fe80::74e2:76ff:fe60:f703/64 scope link
       valid_lft forever preferred_lft forever
78: vethe3f2f52@if77: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default
    link/ether 52:fd:c1:e5:08:b5 brd ff:ff:ff:ff:ff:ff link-netnsid 5
    inet6 fe80::50fd:c1ff:fee5:8b5/64 scope link
       valid_lft forever preferred_lft forever

Because docker for mac is running in a VM it cant see the host interfaces. This will stop broadcast traffic from being seen anyway. So its probably ok just to disable this on macOS maybe display a warning/add a line or to the docs so people know it won't work. (it will be the same no windows or if its runs in VM with NATed networking)

sevenEng · 2018-04-25T10:19:01Z

@Toshbrown hi tosh, I pushed a new commit to the bcast branch of core-network: https://github.com/me-box/core-network/tree/bcast, which should not crash for mac now, but I could only test it on my linux machine by giving it a fake nonexistent IP to search for, this is to simulate what happened on a mac machine, where containers only see virtualized interfaces but not the real host's interfaces, if you have time, could you maybe give it a run on your machine to see how things go, thanks!

sevenEng · 2018-04-25T10:38:02Z

docker-core-network.yaml

+    network_mode: "host"
+    volumes:
+      - '${BCAST_FIFO}:/tmp/relay'
+    command: ["-f", "/tmp/relay", "-h", "${BCAST_IP}"]


tested by setting BCAST_IP here to an nonexistent IP

on mac, BCAST_IP is the host's IP on the LAN, but from within the container interfaces are all virtualized, such an IP could not be found, which previously would crash the service

Toshbrown · 2018-04-26T09:24:14Z

@sevenEng giving it a test this morning

Toshbrown · 2018-04-26T10:21:36Z

@sevenEng All seems to work! LGTM

Set core-network back to master branch after merging me-box/core-network#5

enable broadcast traffic relay

fa8a76d

sevenEng commented Apr 25, 2018

View reviewed changes

Update databox-components

a3b01ec

Set core-network back to master branch after merging me-box/core-network#5

Toshbrown merged commit d017a06 into me-box:master Apr 26, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

enable broadcast traffic relay#243

enable broadcast traffic relay#243
Toshbrown merged 2 commits intome-box:masterfrom
sevenEng:bcast

sevenEng commented Feb 23, 2018

Uh oh!

Toshbrown commented Mar 20, 2018

Uh oh!

sevenEng commented Mar 20, 2018

Uh oh!

sevenEng commented Mar 20, 2018 •

edited

Loading

Uh oh!

sevenEng commented Mar 21, 2018 •

edited

Loading

Uh oh!

Toshbrown commented Mar 21, 2018

Uh oh!

Toshbrown commented Mar 21, 2018 •

edited

Loading

Uh oh!

sevenEng commented Apr 25, 2018

Uh oh!

sevenEng Apr 25, 2018 •

edited

Loading

Uh oh!

Toshbrown commented Apr 26, 2018

Uh oh!

Toshbrown commented Apr 26, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

sevenEng commented Feb 23, 2018

Uh oh!

Toshbrown commented Mar 20, 2018

Uh oh!

sevenEng commented Mar 20, 2018

Uh oh!

sevenEng commented Mar 20, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sevenEng commented Mar 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Toshbrown commented Mar 21, 2018

Uh oh!

Toshbrown commented Mar 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sevenEng commented Apr 25, 2018

Uh oh!

sevenEng Apr 25, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Toshbrown commented Apr 26, 2018

Uh oh!

Toshbrown commented Apr 26, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sevenEng commented Mar 20, 2018 •

edited

Loading

sevenEng commented Mar 21, 2018 •

edited

Loading

Toshbrown commented Mar 21, 2018 •

edited

Loading

sevenEng Apr 25, 2018 •

edited

Loading