case clause error when TLS handshake is closed abruptly

[lukebakken]

I apologize that this report is not up to my usual standard where I provide a reliable means to reproduce it.

Describe the bug

Based on logs provided by a RabbitMQ customer, it appears there is something creating a TCP connection that initiates a TLS handshake but then closes the socket, producing this log message:

2023-07-17 07:58:37.456662-04:00 [notice] <0.2070.0> TLS server: In state connection received CLIENT ALERT: Warning - User Canceled

In addition, sometimes this leads to a case clause error as shown in the attached log file:
case-clause-log.txt

Code: https://github.com/erlang/otp/blob/OTP-25.0.4/lib/ssl/src/ssl_gen_statem.erl#L1964-L1969

I am assuming this is due to the state moving to "closed" but RabbitMQ trying to still get socket options. I'm wondering if the {ok, []} case should be handled in ssl_gen_statem:get_socket_opts/6.

To Reproduce

I have not yet been able to reproduce this.

Affected versions
25.0.4

Additional context

The main reason we're concerned about this crash is that this customer also experiences high memory use by the ssl_gen_statem process - frequently over 100MiB. I'm wondering if this case clause issue prevents data from being garbage collected.

[IngelaAndin]

This seems strange. It would mean that inet:getopts would return {ok, []} which it should not unless you give it an empty list as input. I suppose you do not have some wrapper transport module? I could be helpful if you could trace tls_socket:getopts/3 when it happens if you have that possibility.

[RaimoNiskanen]

I just did a thorough read of the code in inet_drv.c that handles inet:getopts/2, and sure enough; if the standard C library call getsockopt(2) returns an error for an option - then no return value is created for that option. There is not check what I can see about the socket's state in the call chain; if the inet_drv port is still open, then the getsockopt(2) call is executed.

So it should be possible that inet:getsockopt(Port, [sndbuf]) (or recbuf), for a Port that is not dead yet but the underlying socket is closed, may return {ok,[]}. And I think, as you say, that ssl_gen_statem:get_socket_opts/6 should handle that, at least not crash.

[lukebakken]

Thanks everyone for taking a look. I thought it seemed strange when we got that stack trace. If I ever have a clue how to reproduce it of course I will follow up.

[IngelaAndin]

See #7522