advice on how to redirect udp multicast messages

[deanhuff]

I am attempting to redirect broadcast UDP messages to javascript webpage clients.

My current approach is as follows as follows:

1. create a named pipe: mkisofs /tmp/pipe
2. tail the pipe and forward to websocat: tail -f /tmp/pipe | websocat -t -s 10.0.0.1:1234
3. run a socat command in an infinite loop to receive messages from my base service that publishes UDP broadcast messages and append them to the pipe like this: while true; do socat -t 0 UDP4-RECVFROM:30000,ip-add-membership=224.223.222.221:127.0.0.1,reuseaddr STDOUT >> /tmp/pipe; done

The issue I'm seeing in my java script webpage is that some messages seem to batch up and arrive in groups while others do not. Messages are JSON arriving once every 4-8 seconds.

Is there a more straight forward approach I should be using to forward the UDP messages? Any ideas on messages seeming to batch?

thanks!

[vi]

Why not just listen UDP with websocat directly, without going though a pipe?

Currently there's no option for adding memberships (you probably meant "multicast", not "broadcast"), but just listening UDP on localhost should be OK.

First part: socat UDP4-RECVFROM:30000,ip-add-membership=224.223.222.221:127.0.0.1,reuseaddr udp:127.0.0.1:1234

Second part: websocat -tE ws-l:10.0.0.1:1234 reuse-broadcast:udp-l:127.0.0.1:1234.

I'll treat this issue as a request to implement ip-add-membership option in websocat. It's trying to be on par with socat features after all.

If you can somehow add membership though external means then maybe you can listen UDP port 30000 directly from websocat.

[vi]

websocat -s

websocat -s is a "simple server" mode. It it aimed to be used for simple one-off development tests. For bridging something to websockets better use two-arguments websocat command-line.

Maybe there will be a warning for websocat -s when stdin is not a terminal.

create a named pipe: mkisofs /tmp/pipe

mkfifo

[vi]

-t

If content being streamed over multicast UDP is mpegts-packed multimedia or RTP then it's better to use binary WebSocket messages (-b) instead of text (-t).

[deanhuff]

Yes you are correct. These are UDP Multicast messages (not Broadcast, i've updated the title).

Your two commands work flawlessly to get the job done.

I have two notes:

1. Rather than the infinite loop in bash that I was using, I've discovered that I can use fork option to spin off the message and keep running.
2. websocat is throwing a warning: [WARN websocat::net_peer] New client for the same listening UDP socket I suspect this is due to the fork option in my socat process?

thanks for your help with this...next up is to attempt to get this working using secure web sockets.

[vi]

I suspect this is due to the fork option in my socat process?

Yes. It sees new source address, meaning replies wouldn't be sent to old address anymore.

Rather than the infinite loop in bash that I was using, I've discovered that I can use fork option to spin off the message and keep running.

Why looping or forking at all? Why not just persistently receive messages from the using the same UDP socket in socat and also send to websocat using the same UDP socket?

[deanhuff]

without the fork or loop option, socat is closing upon receipt of a single message.

[vi]

socat is closing upon receipt of a single message.

Because of -t 0?

[vi]

Another way to interconnect socat and websocat for datagram flow:

$ rm -f /tmp/qwerty
$ seqpackettool --unidirectional --listen-once --allow-empty start -- /usr/bin/socat socat udp-recv:1234 - -- listen_unix /tmp/qwerty
$ websocat -tE ws-l:127.0.0.1:1234 reuse:seqpacket:/tmp/qwerty

This example is unidirectional. With udp: instead of udp-recv: and removed --unidirectional it becomes bi-directional, but supporting only incoming single UDP session.

[deanhuff]

i actually got rid of the -t 0 in my commands. I know the issue is me not knowing how to use socat and hast nothing to do with websocat.

socat UDP4-RECVFROM:30000,ip-add-membership=224.223.222.221:127.0.0.1,reuseaddr udp:127.0.0.1:1234 - results in 1 message received then hangs up

socat UDP4-RECVFROM:30000,ip-add-membership=224.223.222.221:127.0.0.1,reuseaddr,ignoreeof udp:127.0.0.1:1234 - (added ignoreeof) results in 2 messages received then hangs up

socat UDP4-RECVFROM:30000,ip-add-membership=224.223.222.221:127.0.0.1,reuseaddr,fork udp:127.0.0.1:1234 - (with fork) runs forever

[vi]

,fork is a workaround, not a proper solution.

Maybe the real, non-workaround way is to implement ip-add-membership to websocat. I can try making a special pre-release version of websocat (for some one platform) with that option in.

Another workaround I devised just now:

1. Use socat to connect to multicast group.
2. Use iptables to redirect traffic to local port 30000 to local port 30001
3. Listen UDP port 30001 with websocat

Socat just sits around, not receiving any messages, just keeping igmp membership alive.

[deanhuff]

agreed, fork=hack in my case.

I am not familiar with the iptables forwarding for local traffic but I will look into it. You are essentially saying take multicast message and forward to unicast correct?

here's the output of a socat that is hanging up after 2 messages. I've changed the output to STDOUT so i could see the message....I attempted to set a timeout of 60 seconds on this one (although the program only ran for 8 seconds). I see at the end it says socket1 is at EOF (but i sent ignoreeof. also I poll timed out (no data within 60.00000 seconds) but it was less than 1 second since last packet.

socat -d -d -d -t 60 UDP4-RECVFROM:30000,ip-add-membership=224.223.222.221:127.0.0.1,reuseaddr,ignoreeof,keepalive STDOUT
2019/08/16 15:15:39 socat[32060] I socat by Gerhard Rieger - see www.dest-unreach.org
2019/08/16 15:15:39 socat[32060] I This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit. (http://www.openssl.org/)
2019/08/16 15:15:39 socat[32060] I This product includes software written by Tim Hudson (tjh@cryptsoft.com)
2019/08/16 15:15:39 socat[32060] I setting option "ip-add-membership" to {"224.223.222.221","127.0.0.1"}
2019/08/16 15:15:39 socat[32060] I setting option "so-reuseaddr" to 1
2019/08/16 15:15:39 socat[32060] I setting option "ignoreeof" to 1
2019/08/16 15:15:39 socat[32060] I setting option "so-keepalive" to 1
2019/08/16 15:15:39 socat[32060] I socket(2, 2, 17) -> 5
2019/08/16 15:15:39 socat[32060] I socket(2, 2, 0) -> 6
2019/08/16 15:15:39 socat[32060] I ioctl(6, SIOCGIFINDEX, {"127.0.0.1"}): No such device
2019/08/16 15:15:39 socat[32060] I close(6)
2019/08/16 15:15:39 socat[32060] I starting recvfrom loop
2019/08/16 15:15:39 socat[32060] N receiving on AF=2 0.0.0.0:30000
2019/08/16 15:15:42 socat[32060] N receiving packet from AF=2 10.15.207.216:30000
2019/08/16 15:15:42 socat[32060] I permitting packet from AF=2 10.15.207.216:30000
2019/08/16 15:15:42 socat[32060] N using stdout for reading and writing
2019/08/16 15:15:42 socat[32060] I resolved and opened all sock addresses
2019/08/16 15:15:42 socat[32060] N starting data transfer loop with FDs [5,5] and [1,1]
2019/08/16 15:15:42 socat[32060] N received packet with 414 bytes from AF=2 10.15.207.216:30000
[JSON PRINTED TO STDOUT]
2019/08/16 15:15:42 socat[32060] I transferred 414 bytes from 5 to 1
2019/08/16 15:15:46 socat[32060] N received packet with 421 bytes from AF=2 10.15.207.216:30000
[JSON PRINTED TO STDOUT]
2019/08/16 15:15:46 socat[32060] I transferred 421 bytes from 5 to 1
2019/08/16 15:15:46 socat[32060] N socket 1 (fd 5) is at EOF
2019/08/16 15:15:47 socat[32060] I poll timed out (no data within 60.000000 seconds)
2019/08/16 15:15:47 socat[32060] N exiting with status 0

[deanhuff]

Maybe the real, non-workaround way is to implement ip-add-membership to websocat. I can try making a special pre-release version of websocat (for some one platform) with that option in.

If you're interested in putting in the time my particular platform is 16.04 Ubuntu Linux :)

[vi]

Please try https://github.com/vi/websocat/releases/download/v1.5.0/websocat_amd64-linux-static+udp

./websocat_amd64-linux-static+udp -Et ws-l:10.0.0.1:1234 reuse:udp-l:0.0.0.0:30000 --udp-multicast=224.223.222.221 --udp-multicast-iface-v4=127.0.0.1

Note that UDP socket is only opened and added to multicast group once first WebSocket client gets connected.

If multiple clients are sending to the multicast group, expect warnings from Websocat each time source address switches.

I'm not sure how WebSocket -> UDP multicast direction would work.

---

New options:

FLAGS:
        --udp-broadcast                         [A] Set SO_BROADCAST
        --udp-multicast-loop                    [A] Set IP[V6]_MULTICAST_LOOP

OPTIONS:
        --udp-multicast <udp_join_multicast_addr>...
            [A] Issue IP[V6]_ADD_MEMBERSHIP for specified multicast address. Can be specified multiple times.

        --udp-multicast-iface-v4 <udp_join_multicast_iface_v4>...
            [A] IPv4 address of multicast network interface Has to be either not specified or specified the same number
            of times as multicast addresses
        --udp-multicast-iface-v6 <udp_join_multicast_iface_v6>...
            [A] Index of network interface for IPv6 multicast Has to be either not specified or specified the same
            number of times as multicast addresses
        --udp-ttl <udp_ttl>                                          [A] Set IP_TTL, also IP_MULTICAST_TTL if applicable

[deanhuff]

vi, thank you. I will try this tomorrow morning!

[deanhuff]

Sorry it has taken me so long to be able to test this.

I am receiving an error "address in use". On my socat command prior, I had to use the reuseaddress option. Do you think this is related?

./websocat_amd64-linux-static+udp -Et ws-l:10.15.207.216:1234 reuse:udp-l:0.0.0.0:30000 --udp-multicast=224.223.222.221 --udp-multicast-iface-v4=127.0.0.1
<<wait for client connection (from browser)>>
websocat: Address in use (os error 98)

thanks
-Dean

[vi]

Updated the relese file, added new option: --udp-reuseaddr. Please check.

[deanhuff]

we have a winner! it is working like a champ.

./websocat_amd64-linux-static+udp -Et ws-l:10.15.207.216:1234 reuse:udp-l:0.0.0.0:30000 --udp-multicast=224.223.222.221 --udp-multicast-iface-v4=127.0.0.1 --udp-reuseaddr

it even works with my cert using ssl

./websocat_amd64-linux-static+udp -Et --pkcs12-der=/services/staging/keys/websocat_keycert.pkcs12 wss-l:10.15.207.216:1234 reuse:udp-l:0.0.0.0:30000 --udp-multicast=224.223.222.221 --udp-multicast-iface-v4=127.0.0.1 --udp-reuseaddr

[deanhuff]

i'm gonna close out my question here. thank you for all of your assistance with getting my particular usage handled so quickly. I'll keep an eye out for official releases that support joining multicast groups. In the meantime i'll use your custom build you've provided.