Allow disabling SNI on SSL Connections

[Evengard]

When connecting through SSL, Npgsql sends along the SNI hostname, yet this is not always desired. The "conformant" libpq library doesn't seem to send SNI on SSL connections at all, and some server software (poolers, etc) are not playing well when there is an IP address sent in SNI (when you connect not to a hostname, but to an IP address).

It seems like there is no way to disable the SNI, as it seems to be just set from the ConnectionString Host parameter.

It would be appreciated, if there would be some way to disable the passing of SNI to the remote server.

Ref to this:

https://github.com/npgsql/npgsql/blob/eb050c0a0fd4b70f9d11c6d41027076e9e4e01c1/src/Npgsql/Internal/NpgsqlConnector.cs#L879C17-L883C1

Here not passing through Host to AuthenticateAsClient and instead passing an empty string will effectively disable SNI.

[vonzshik]

Interesting. Could you please share what kind of problems did you encounter (you did mention poolers misbehaving)? I'm not really opposed to adding an option to disable SNI hostname, just would like to know more which actual problems it's going to help with (and as a bonus we'll be able to guide other users to use that option in the future in case they'll also encounter it).

[Evengard]

In my specific case it was to do with an inhouse PG pooler implementation (can't divulge too much about it for now), which used standard Rust libraries which do NOT allow SNIs to be IP addresses at all. And we connect to it through an IP address. The connection ended with an SSL error.

My case is kinda specific, but the devs of the said pooler said that they were relying on the behaviour of the considered to be conformant libpq, which doesn't send any SNI.

[Evengard]

There's another take to this problem: detecting that Host is an IP address and not a hostname, and avoid sending SNI, as actually it seems to break TLS RFCs which states only DNS hostnames are supported as SNIs.

[xeromorph]

@vonzshik According to RFC 4366 TLS Extensions
SNI server_name extension only supports DNS hostnames:

Currently, the only server names supported are DNS hostnames;

"HostName" contains the fully qualified DNS hostname of the server,
as understood by the client. The hostname is represented as a byte
string using UTF-8 encoding [UTF8], without a trailing dot.

and

Literal IPv4 and IPv6 addresses are not permitted in "HostName".

Indeed, SslClientAuthenticationOptions.TargetHost property states

Gets or sets the name of the server the client is trying to connect to. That name is used for server certificate validation. It can be a DNS name or an IP address.

but if that value actually ends up in SNI server_name extension, it's a contradiction to RFC.

Overall, I think it would be suitable to have connection string option to disable SNI similar to libpq's one

[vonzshik]

OK. So, on one hand, we have RFC which does explicitly say that IP address shouldn't be used as a hostname. On the other, .NET documentation states that it can be both hostname and IP address (though it also says that's it's used for certificate validation, so it might be doing something else beside sending the hostname to the server?).
Since libpq does have an option to stop sending hostname, I think it's OK to also add a similar thing to Npgsql. @roji what's your opinion?

[roji]

Hmm, I also think it's reasonable for us to have a connection string parameter to disable sending the host, like the libpq option. However, it seems that most AuthenticateAsClient overloads accept a non-nullable Host parameter, except one which doesn't accept a Host, but also doesn't accept any parameter aside from the SslClientAuthenticationOptions. So how would one e.g. pass a custom client certificate list (or enable certificate revocation) while at the same time stop sending the Host?

We may want to better understand the AuthenticateAsClient behavior and the meaning of the Host parameter.. I'd actually post this as a question on the runtime repo to see what they say...

[Evengard]

I actually did dive down the call stack of AuthenticateAsClient in the internals of Dotnet. All you need to do is pass an empty string (aka string.Empty) for it to not generate any SNI yet not being nullable.

Check here: https://github.com/dotnet/runtime/blob/cf16ed8fa95f136e007f55591f174689c020f386/src/libraries/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.OpenSsl.cs#L396

It is similar for Windows interop. Even stated in the comment.

[roji]

@Evengard thanks for looking into it - in that case it does sound reasonable to add the option.

I'd still start a conversation in the runtime repo about possibly making the Host parameter nullable - special handling for an empty string seems like somewhat squishy/imprecise behavior to me (and we may learn a bit more about why including/not including the target host is a good idea etc.).

[Evengard]

Technically, the actual code seems to accept null strings for at least OpenSSL interop, although haven't checked other kinds of interop. A clarification would be good, that's for sure.

[roji]

(please link the runtime thread here when you open it, it would be good to follow)

[Evengard]

Hrm, after further investigation, it seems like the IP address is not sent as SNI in .NET 8.0, that's good to know (ref dotnet/runtime#89289 and linked issues in that pull request). Seems like string.Empty may have side effects (even though their own initial fix was to actually make TargetHost a string.Empty the first time).

I've opened a discussion here: dotnet/runtime#97443

[roji]

FYI converted the discussion to an issue (dotnet/runtime#97444), which I think it more appropriate (incorrect nullability annotations?) and will get more attention (discussions are generally more for the community).

[NinoFloris]

Would #5483 work for you as well @Evengard?

[Evengard]

It probably would, although using a configuration string parameter seems to be better. See, we're using a wrapper around Npgsql. With a config variable, that wrapper would need no adjustments, with that callback it would need to be adjusted. It sure is better than no way of configuring it at all though!

[xeromorph]

@NinoFloris this should do as long as it's exposed via DataSourceBuilder

[roji]

FWIW I think this is a good case for a connection string parameter, rather than an NpgsqlDataSourceBuilder config option... I think in general, environmental config should be modifiable without rebuilding the application (and this seems very environmental).

[vonzshik]

Discussed this issue with @roji and @NinoFloris offline yesterday.
We would prefer to not add a connection string parameter. The main reasons are:

1. It doesn't look like currently popular poolers are affected (the same thing for pg itself)
2. The issue is fixed in .NET 8
3. We do plan to add a callback so users can modify SslStream settings

Even then, there is still a problem of a user experience. Even if that parameter was added, they would have to either:

• Use .NET 8 or
• Use Npgsql 9 and set a specific parameter

Both options seem quite restrictive, would have to be documented (and in case of Npgsql 9, will only be released in 10 months). So instead I propose that we do the same thing that was done in dotnet/runtime#81631: we check whether the host is a valid address, and if it's not, replace with string.Empty. We can hide it behind #if so it will naturally go away after we drop support for anything below .NET 8. Additionally, we can backport this fix to the currently supported version of Npgsql (6, 7 and 8).
@Evengard @xeromorph what do you say?

[Evengard]

A backport would be appreciated, I think fixing just the part when the IP is sent as SNI is enough for a backport and general resolution of this issue, with additional SslStream customizations for npgsql 9.x+ only.

[xeromorph]

@vonzshik a backport to Npgsql 6 and 7 is much desired, thank you.

[Evengard]

Sorry to ping about this, but is there a planned backported release planned with this fix?

[vonzshik]

@Evengard we're planning for a 8.0.3 release by the end of this month, most likely we'll release the other versions at the same time.