Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kestrel inconsistent certificate chain handling between endpoint and default configuration #60709

Closed
1 task done
jnjudge1 opened this issue Mar 3, 2025 · 2 comments · Fixed by #60710
Closed
1 task done
Labels
area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions
Milestone

Comments

@jnjudge1
Copy link
Contributor

jnjudge1 commented Mar 3, 2025

Is there an existing issue for this?

  • I have searched the existing issues

Describe the bug

On Kestrel server, there is inconsistent behavior configuring certificates per endpoint versus in the “Default” configuration. The discrepancy occurs when specifying a certificate file containing a chain of certificates where the chain is not included in the system's trust store.

I tested this on Windows, Mac, and Ubuntu using certificate files of the form:

server_certificate.crt:
-----BEGIN CERTIFICATE-----
<leaf>
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
<intermediate>
-----END CERTIFICATE-----
server_key.key
-----BEGIN EC PRIVATE KEY-----
<key>
-----END EC PRIVATE KEY-----

The files were generated using step CLI, and none of the root or intermediate certificates were in the system trust store initially. For each of the following, I removed the certificates from the trust store before each test.

On Windows:

If a certificate file is specified in an endpoint's configuration section, Kestrel will:

  • Output the full certificate chain in its TLS response
  • Add the intermediate certificate to the Windows certificate store if it is not already present.
appsettings.json:
{
  "Kestrel": {
    "Endpoints": {
      "UsingEndpointConfiguration": {
        "Url": "https://localhost:5000",
        "Certificate": {
          "Path": "server_certificate.crt",
          "KeyPath": "server_key.key"
        }
      },
      "UsingDefaultConfiguration": {
        "Url": "https://localhost:5001",
      }
    },
    "Certificates": {
      "Default":{
        "Path":"server_certificate.crt",
        "KeyPath": "server_key.key"
      }
    }
  }
}

openssl s_client -connect localhost:5000 -showcerts and openssl s_client -connect localhost:5001 -showcerts correctly responds with a full certificate chain as specified in server_certificate.crt.

The intermediate will also be added to the Windows Certificate Store if it is not present.

If the certificate file is specified only in the default configuration section:

  • The chain is not presented if it is not already in the Windows certificate store.
  • The intermediate certificate is not added to the store.
{
  "Kestrel": {
    "Endpoints": {
      "UsingDefaultConfiguration": {
        "Url": "https://localhost:5000"
      }
    },
    "Certificates": {
      "Default":{
        "Path":"server_certificate.crt",
        "KeyPath": "server_key.key"
      }
    }
  }
}

openssl s_client -connect localhost:5000 -showcerts only responds with the leaf certificate specified in server_certificate.crt, and the intermediate is not added to the certificate store.

On MacOS:

If a certificate file is specified in an endpoint's configuration section, Kestrel will:

  • Output the full certificate chain in its TLS response for that endpoint. We will receive the full chain from the following configuration.
appsettings.json:
{
  "Kestrel": {
    "Endpoints": {
      "UsingEndpointConfiguration": {
        "Url": "https://localhost:5000",
        "Certificate": {
          "Path": "server_certificate.crt",
          "KeyPath": "server_key.key"
        }
      }
    }
  }
}

If the certificate file is specified in the default configuration section:

  • The certificate chain is not presented by endpoints using the default configuration, regardless of if it is specified for other endpoints.
appsettings.json:
{
  "Kestrel": {
    "Endpoints": {
      "UsingEndpointConfiguration": {
        "Url": "https://localhost:5000",
        "Certificate": {
          "Path": "server_certificate.crt",
          "KeyPath": "server_key.key"
        }
      },
      "UsingDefaultConfiguration": {
        "Url": "https://localhost:5001",
      }
    },
    "Certificates": {
      "Default":{
        "Path":"server_certificate.crt",
        "KeyPath": "server_key.key"
      }
    }
  }
}

openssl s_client -connect localhost:5001 -showcerts will only present us the leaf certificate.

Behavior on Ubuntu:

  • The behavior observed on Ubuntu is the same as on MacOS.

Relevant Code Snippet

This behavior appears to be caused by this line in TlsConfigurationLoader.cs:

 var (defaultCert, _ /* cert chain */) = _certificateConfigLoader.LoadCertificate(defaultCertConfig, "Default");

which does not process the full certificate chain when it is used in the default configuration section. As a result, the subsequent call to SslStreamCertificateContext.Create does not add the full certificate chain:

_serverCertificateContext = SslStreamCertificateContext.Create(certificate, additionalCertificates: options.ServerCertificateChain);

On Mac and Ubuntu this means the chain is never presented on endpoints using the default configuration.

On Windows, SslStreamCertificateContext.Create adds intermediates specified in the call to the Windows Certificate Store if they are not already present. This means if the certificate chain has been specified on an endpoint level configuration, it will be added to the Windows trust store, and subsequently any endpoints specified using the default configuration will also have their full chain presented.

However, if the file is only specified on the default configuration, the chain will be thrown out and never added to the trust store from this call, so any endpoints using the default configuration will only present their leaf certificates.

Expected Behavior

The behavior when specifying the certificate file in an individual endpoint’s configuration should have identical behavior to endpoints using the default configuration. The full certificate chain should be included for endpoints using the default configuration when possible.

Steps To Reproduce

  1. Generate some certificates using step CLI or similar CA such as Hashicorp Vault, where intermediates are not in the system trust store
  2. Create a minimal dotnet app, for example: repo
  3. Run the app, then openssl s_client -connect localhost:5000 -showcerts, and observe the certificate chain

Exceptions (if any)

No response

.NET Version

9.0.103

Anything else?

IDE: VSCode 1.97.2

Tools

step cli: https://github.com/smallstep/cli

@dotnet-issue-labeler dotnet-issue-labeler bot added the area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions label Mar 3, 2025
@jnjudge1
Copy link
Contributor Author

jnjudge1 commented Mar 3, 2025

I have attempted a fix for this issue here: #60710

@adityamandaleeka
Copy link
Member

@jnjudge1, welcome to the repo, and thanks for filing a high quality issue. We'll take a look at this (and the associated PR you opened) soon.

@BrennanConroy BrennanConroy added this to the 10.0-preview3 milestone Mar 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions
Projects
None yet
3 participants