Initial implementation of imdtls and omdtls modules

[alorbach]

Initial implementation of imdtls and omdtls modules

• Extracted basic OpenSSL helper functions into own module net_ossl.h/net_ossl.c
  Both are compiled into lmnsd_ossl.
• Cleanup of OpenSSL code, fixed minor compiler and linking issues.
• Added DTLS Sender option DTLS into tcpflood for testbench.
• Add initial implementation of imdtls input module. Added to configure and makefile
• Add initial implementation of omdtls output module. Added to configure and makefile
• Add multiple basic tests for imdtls receiving data by using tcpflood.
• Add multiple send-receive test for imdtls and omdtls based on existing tls tests.
• Add timeout and sessionbreak tests for imdtls stress testing.

closes: #5211

[rgerhards]

I reviewed the most important files. While I found a lot, it's mostly nits (with comment issues being the top issues). However, there are some more substantial comments, but nothing really bad :-)

[davidelang]

On Tue, 28 Nov 2023, Rainer Gerhards wrote: I reviewed the most important files. While I found a lot, it's mostly nits (with comment issues being the top issues). However, there are some more substantial comments, but nothing really bad :-) > +#define MAX_DTLS_MSGSIZE 65536 +#define DTLS_LISTEN_PORT "4433" + + +/* Module static data */ +DEF_OMOD_STATIC_DATA +DEFobjCurrIf(glbl) +DEFobjCurrIf(datetime) +DEFobjCurrIf(prop) +DEFobjCurrIf(ruleset) +DEFobjCurrIf(statsobj) +DEFobjCurrIf(net_ossl) + +#define DTLS_MAX_RCVBUF 1380 /* Maximum DTLS packet 1380 bytes to avoid fragmentation (smaller than the common + * Ethernet MTU of 1,500 bytes to accommodate potential IP and UDP headers). */ +// #define DTLS_MAX_RCVBUF 16 * 1024 + 1/* TLS RFC 8449: max size of buffer for message reception */ remove comment -- or is there any reason we can go above 1380 bytes? Looking further down in buffer handling, I actually think larger messages are possible, especially if "jumbo frames" are enabled at the network layer (some background: https://www.networkworld.com/article/745164/mtu-size-issues.html)

fragmented UDP packets are a thing, they make the messages far more fragile in congested enviornments, but they do work to much larger messages, even without jumbo frames. not saying they are a good idea, but they can work. if the size is set too large, I would suggest a warning on the topic, rather than just disallowing. David Lang

[rgerhards]

Made announcement of this effort: https://www.rsyslog.com/elevating-syslog-security-rsyslog-introduces-dtls-plugins-for-udp/

[davidelang]

I haven't dug through things yet, but given that out-of-order recovery and fragmentation handling can be used for DOS purposes (valid handshake, then start sending everything but the first packet, per the dtls rfc the receiver it supposed to hang on to the later packets waiting for the first one)

there should be configuration for timeouts, but for retransmission and also for throwing away later packets in the sequence.

[alorbach]

I haven't dug through things yet, but given that out-of-order recovery and fragmentation handling can be used for DOS purposes (valid handshake, then start sending everything but the first packet, per the dtls rfc the receiver it supposed to hang on to the later packets waiting for the first one)

there should be configuration for timeouts, but for retransmission and also for throwing away later packets in the sequence.

Good point. I will add timeout handling to imdtls module. I would suggest a default timeout of 30 minutes for a DTLS session, do you agree?

[davidelang]

This is less about timing out an entire session than timing out individual chunks. the rfc suggests a 1s timeout for lost chunks (before you are supposed to retransmit them) but says nothing about timing out n+x chunks when you haven't received chunk N yet (it says if you receive future chunks you are supposed to hold on to them and process them after you receive chunk N). If you have a proper sender who will retry chunk N quickly, this isn't bad (and avoids having to retransmit all newer chunks because one arrives out of order), but if you have a malicous sender you need to have a short timeout (single digit seconds) that you hang on to these out-of-order chunks before you throw them away and force the sender to retransmit them David Lang

…

On Thu, 11 Jan 2024, Andre Lorbach wrote: > I haven't dug through things yet, but given that out-of-order recovery and fragmentation handling can be used for DOS purposes (valid handshake, then start sending everything but the first packet, per the dtls rfc the receiver it supposed to hang on to the later packets waiting for the first one) > > there should be configuration for timeouts, but for retransmission and also for throwing away later packets in the sequence. Good point. I will add timeout handling to imdtls module. I would suggest a default timeout of 30 minutes for a DTLS session, do you agree?

[alorbach]

This is less about timing out an entire session than timing out individual chunks. the rfc suggests a 1s timeout for lost chunks (before you are supposed to retransmit them) but says nothing about timing out n+x chunks when you haven't received chunk N yet (it says if you receive future chunks you are supposed to hold on to them and process them after you receive chunk N). If you have a proper sender who will retry chunk N quickly, this isn't bad (and avoids having to retransmit all newer chunks because one arrives out of order), but if you have a malicous sender you need to have a short timeout (single digit seconds) that you hang on to these out-of-order chunks before you throw them away and force the sender to retransmit them David Lang
…
On Thu, 11 Jan 2024, Andre Lorbach wrote: > I haven't dug through things yet, but given that out-of-order recovery and fragmentation handling can be used for DOS purposes (valid handshake, then start sending everything but the first packet, per the dtls rfc the receiver it supposed to hang on to the later packets waiting for the first one) > > there should be configuration for timeouts, but for retransmission and also for throwing away later packets in the sequence. Good point. I will add timeout handling to imdtls module. I would suggest a default timeout of 30 minutes for a DTLS session, do you agree?

I think this is protocol handling and it is done inside OpenSSL. And I assume it should be properly handled inside OpenSSL. We use the DTLS_method() context and the same OpenSSL API's to receive by DTLS as we use in the nsd_ossl.c code (SSL_read).

[rgerhards]

I think this is protocol handling and it is done inside OpenSSL. And I assume it should be properly handled inside OpenSSL.

I would check if openSSL accepts config parameters for the timeouts @davidelang mentioned. Actually I would assume that such params exist.