TURN TLS not working

[licaon-kter]

Environment

• ejabberd version: 20.04.9
• Erlang version: Erlang (SMP,ASYNC_THREADS) (BEAM) emulator version 10.2.4
• OS: Debian Buster arm64
• Installed from: source

Listeners (& certs as expected):

  -
    port: 3478
    transport: udp
    use_turn: true
    turn_min_port: 60000
    turn_max_port: 65000
    turn_ip: MY.EXTE.RNAL.IP
    module: ejabberd_stun
  -
    port: 3478
    use_turn: true
    turn_min_port: 60000
    turn_max_port: 65000
    turn_ip: MY.EXTE.RNAL.IP
    module: ejabberd_stun
  -
    port: 5349
    use_turn: true
    turn_min_port: 60000
    turn_max_port: 65000
    turn_ip: MY.EXTE.RNAL.IP
    tls: true
    module: ejabberd_stun

Modules:

mod_stun_disco:
    services:
      -
        host: MY.EXTE.RNAL.IP
        port: 3478
        type: stun
      -
        host: MY.EXTE.RNAL.IP
        port: 3478
        type: turn
      -
        host: MY.EXTE.RNAL.IP
        port: 3478
        type: stun
        transport: tcp
      -
        host: MY.EXTE.RNAL.IP
        port: 3478
        type: turn
        transport: tcp
      -
        host: mydomain.tld
        port: 443
        type: stuns
      -
        host: mydomain.tld
        port: 443
        type: turns

SSLH will route all 443 traffic to port 5349 eventually (yrutschle/sslh#246 (comment))

Between Conversations contacts, the call is not connected after pickup.

10:10:14.211 [info] <0.394.0>@mod_stun_disco:handle_call:332 Offering STUN/TURN services to user2@mydomain.tld/myConversations3.FfGe (14c8f5dbc592ffcd)
10:10:14.359 [info] <0.1634.1>@ejabberd_listener:accept:256 (<0.1636.1>) Accepted connection hidden_by_ejabberd -> LANIP:5349
10:10:14.361 [info] <0.1634.1>@ejabberd_listener:accept:256 (<0.1637.1>) Accepted connection hidden_by_ejabberd -> LANIP:5349
10:10:14.443 [info] <0.394.0>@mod_stun_disco:handle_call:332 Offering STUN/TURN services to user1@mydomain.tld/myConversations3.NjCv (f907ff75c592ffcd)
10:10:14.484 [info] <0.1637.1> accepted long-term STUN authentication for 1589008094:14c8f5dbc592ffcd@mydomain.tld from LANIP:47372
10:10:14.495 [info] <0.1636.1> accepted long-term STUN authentication for 1589008094:14c8f5dbc592ffcd@mydomain.tld from LANIP:47370
10:10:16.000 [info] <0.1634.1>@ejabberd_listener:accept:256 (<0.1640.1>) Accepted connection hidden_by_ejabberd -> LANIP:5349
10:10:16.066 [info] <0.1636.1> accepted long-term STUN authentication for 1589008094:14c8f5dbc592ffcd@mydomain.tld from LANIP:47370
10:10:16.103 [info] <0.1637.1> accepted long-term STUN authentication for 1589008094:14c8f5dbc592ffcd@mydomain.tld from LANIP:47372
10:10:16.230 [info] <0.1640.1> accepted long-term STUN authentication for 1589008094:f907ff75c592ffcd@mydomain.tld from LANIP:47374
10:10:16.275 [info] <0.1640.1> accepted long-term STUN authentication for 1589008094:f907ff75c592ffcd@mydomain.tld from LANIP:47374
10:10:16.275 [info] <0.1640.1> accepted long-term STUN authentication for 1589008094:f907ff75c592ffcd@mydomain.tld from LANIP:47374
10:10:16.301 [info] <0.1637.1> accepted long-term STUN authentication for 1589008094:14c8f5dbc592ffcd@mydomain.tld from LANIP:47372

Conversations gives up...
10:10:31.510 [info] <0.1640.1> accepted long-term STUN authentication for 1589008094:f907ff75c592ffcd@mydomain.tld from LANIP:47374
10:10:31.567 [info] <0.1637.1> accepted long-term STUN authentication for 1589008094:14c8f5dbc592ffcd@mydomain.tld from LANIP:47372

But, if I comment the listeners, add a secret, reload_config, start coturn on the same ports 3478/5349, it will work just fine.

10:33:47.727 [info] <0.394.0>@mod_stun_disco:handle_call:332 Offering STUN/TURN services to user2@mydomain.tld/myConversations3.FfGe (14c8f5dbc592ffcd)
10:33:48.045 [info] <0.394.0>@mod_stun_disco:handle_call:332 Offering STUN/TURN services to user1@mydomain.tld/myConversations3.NjCv (f907ff75c592ffcd)

What am I missing?

[weiss]

I'm a bit confused by you having both TLS and non-TLS listeners configured and then concluding that TLS specifically doesn't work.

Anyway, TLS worked in the past for me, and to double-check I now tested again using the following TLS-only configuration:

listen:
  -
    module: ejabberd_stun
    transport: tcp
    port: 3478
    ip: ADDRESS
    use_turn: false
  -
    module: ejabberd_stun
    transport: tcp
    port: 5349
    ip: ADDRESS
    turn_ip: ADDRESS
    use_turn: true
    tls: true
    certfile: TLS_CERT_FILE

modules:
  mod_stun_disco:
    offer_local_services: false
    services:
      - # We need a non-TLS STUN server as libwebrtc doesn't support STUNS.
        host: stun.example.com
        type: stun
        transport: tcp
      -
        host: turns.example.com
        type: turns
        transport: tcp

When using Conversations 2.8.2 (older versions don't work with TLS), the streams are TURN-relayed just fine with this setup. I don't have sslh in between, though. Did you double-check you can reproduce the issue without involving sslh?

[licaon-kter]

My bad, forgot to mention that I've disabled router mapping for 3478/5349 and left only 443 open to force Conversations to try to connect to TURN on that port.

Will try your config too.

[licaon-kter]

Ok, used your snippet @weiss but it does not work.

/LE: stopping the listeners and starting coturn instead (disabled udp and tcp, to force it only on tls), yet still does not work, so your initial hint (that coturn was not doing actually TLS either) might be correct.

[licaon-kter]

Retesting coturn with your disco snippet (disabled udp and tcp, to force it only on tls), added external-ip, and looks like TURN TLS worked, but then I removed external-ip and TLS was still working.

Logcat said:

D myconversations3: user2@mydomain.tld: discovered ICE Server: [stun:mydomain.tld:3478?transport=tcp] [:] [TLS_CERT_POLICY_INSECURE_NO_CHECK] [] [null] [null]
D myconversations3: user2@mydomain.tld: discovered ICE Server: [turns:mydomain.tld:5349?transport=tcp] [1589087740:fce330f2c592ffcd:jpFKn3NZ2vD3LqpI54X/mjnM6tM=] [TLS_CERT_POLICY_INSECURE_NO_CHECK] [] [null] [null]

...which means that it worked, right @iNPUTmice?

[iNPUTmice]

Discovered ICE server just means that Conversations parsed the 0215 correctly.

What's interesting are the 'sending candidate' lines and ultimately the 'selected candidate' lines

[weiss]

As I can't reproduce the problem I would assume it's somehow specific to your environment. The log lines you posted indicate that the TLS connection actually works for you as well, as the (TLS-encrypted) authentication handshake (which involves several packets being transmitted between the TURN client and the TURN server ) succeeds, e.g.:

10:10:14.484 [info] <0.1637.1> accepted long-term STUN authentication for 1589008094:14c8f5dbc592ffcd@mydomain.tld from LANIP:47372

So things probably fail later on. For the actual relay, the TURN server allocates a UDP socket the peer's client will talk to. Maybe this UDP communication fails for some reason. I'd try to track this down by letting tcpdump watch the UDP traffic on the server (and then, if necessary, also on the client-side).

[licaon-kter]

@weiss

Coturn supports TCP relaying rather than UDP. That would also match my theory that UDP is somehow borked.

Indeed, if I apply no-tcp-relay then coturn is not working anymore either.

My router mapping (in case this is the problem):

[weiss]

And you have turn_min_port: 60000 and turn_max_port: 65000 in your ejabberd_stun configuration for the TLS listener?

[licaon-kter]

@weiss Yes, I've copied that part from my normal config to your snippet to match my router config, open ports.

[mremond]

@licaon-kter Is that ticket solved now by doc clarification ?

[licaon-kter]

Since this hasn't been a big thing I didn't follow up.