[EPIC] Signal Server Facilitates Relay + Removal of TURN

[neonphog]

stateDiagram-v2
direction LR
state "Signal Connect" as sc
state "Relay Messages via Signal" as rms
state "Try Open WebRTC" as rtc
note left of rtc: STUN only
state rtc_ok <<choice>>
state "Relay Messages via WebRTC" as rmrtc
sc --> rms
sc --> rtc
rtc --> rtc_ok
rtc_ok --> rms: error
rtc_ok --> rmrtc: success

• [Epic] SBD message server and client
  • SBD: Write the base rust client and server that can exchange messages over plain-text websockets #6
  • SBD: Add TLS to the base rust client and server #7
  • SBD: Libsodium secret stream-based e2e encryption wrapper for sbd client #19
  • SBD: measure 1-instance vertical scaling with multiple clients #8
  • (can be deferred) SBD: Disk-based ip deny list for limit or protocol violations #9
  • (can be deferred) SBD: Trusted ip header to work with reverse proxy like cloudflare #10
  • (can be deferred) SBD: prometheus metrics #11
  • (can be deferred) SBD: ip-based rate limiting #12
  • (can be deferred) SBD: Sbd server can watch a directory and reload effective TLS certs without process restart #13
  • (can be deferred) SBD: Backchannel communications to facilitate scaling #14
  • (can be deferred) SBD: measure N-instance horizontal scaling with multiple clients #15
• [Epic] Cloudflare Durable-Object based version of SBD server #16
• Tx5: Tx5 uses sbd client with e2e encryption instead of previous signal client #17
• Tx5: Tx5 messages first directly through sbd & attempts webrtc connection #18

[neonphog]

Moving these design musings down to a comment so the description can be real epic details:

---

Signal Server Protocol

• register includes query string timestamped proof of private key
  • wait period then nonce response verify of private key
  • send limit values to client
• forward request bytes be sent to peer (dest peer replaced by source peer when sent out again)
• keepalive periodic client send

Limiting

• limit X concurrent pending connections
• limit X connections per IP
• limit X IPs in memory
• limit X messages per connection per time
• limit message size to X MiB

Back Channel Protocol

• online notify a peer server of client availability (after register success)
• offline notify a peer server of client disconnect
• forward request bytes be sent to peer
• keepalive both peer servers should periodically keepalive

---

Signal Client Protocol

Client protocol is sodium secret stream based with logic to respect rate-limits and buffer messages into batch sends without going over the max message size returned by the server. Within the encryption are framed message types:

• message an actual client message send--this is client communication relayed via the signal server
• request_offer a polite node wants an offer from the impolite node
• offer an impolite node wants to try negotiating webrtc
• answer a polite node responding to an offer
• ice webrtc connectivity message
• open sent upon webrtc data channel connect success. When a node has both sent and received the open message, it switches to sending all data over the webrtc channel. (Note, this may not be needed, data channel open is probably a good enough signal to start using it).