feature: add option for creating a global truststore (keycloak#24473)

closes keycloak#24148

Co-authored-by: Václav Muzikář <vaclav@muzikari.cz>
Co-authored-by: Martin Bartoš <mabartos@redhat.com>
Signed-off-by: ShefeeqPM <86718986+ShefeeqPM@users.noreply.github.com>

server-spi-private/src/main/java/org/keycloak/truststore/HostnameVerificationPolicy.java → common/src/main/java/org/keycloak/common/enums/HostnameVerificationPolicy.java

@@ -15,7 +15,7 @@
  * limitations under the License.
  */
 
-package org.keycloak.truststore;
+package org.keycloak.common.enums;
 
 public enum HostnameVerificationPolicy {
 

docs/documentation/release_notes/index.adoc

@@ -13,6 +13,9 @@ include::topics/templates/document-attributes.adoc[]
 :release_header_latest_link: {releasenotes_link_latest}
 include::topics/templates/release-header.adoc[]
 
+== {project_name_full} 24.0.0
+include::topics/24_0_0.adoc[leveloffset=2]
+
 == {project_name_full} 23.0.0
 include::topics/23_0_0.adoc[leveloffset=2]
 

docs/documentation/release_notes/topics/24_0_0.adoc

@@ -2,8 +2,12 @@
 
 The Keycloak JS adapter now uses the https://webpack.js.org/guides/package-exports/[`exports` field] in `package.json`. This improves support for more modern bundlers like Webpack 5 and Vite, but comes with some unavoidable breaking changes. Consult the link:{upgradingguide_link}[{upgradingguide_name}] for more details.
 
+= Truststore Improvements
+
+Keycloak introduces an improved truststores configuration options. The Keycloak truststore is now used across the server: for outgoing connections, mTLS, database drivers and more. It's no longer needed to configure separate truststores for individual areas. To configure the truststore, you can put your truststores files or certificates in the default `conf/truststores`, or use the new `truststore-paths` config option. For details refer to the relevant https://www.keycloak.org/server/keycloak-truststore[guide].
+
 == Automatic certificate management for SAML identity providers
 
 The SAML identity providers can now be configured to automatically download the signing certificates from the IDP entity metadata descriptor endpoint. In order to use the new feature the option `Metadata descriptor URL` should be configured in the provider (URL where the IDP metadata information with the certificates is published) and `Use metadata descriptor URL` needs to be `ON`. The certificates are automatically downloaded and cached in the `public-key-storage` SPI from that URL. The certificates can also be reloaded or imported from the admin console, using the action combo in the provider page.
 
-See the https://www.keycloak.org/docs/latest/server_admin/index.html#saml-v2-0-identity-providers[documentation] for more details about the new options.
+See the https://www.keycloak.org/docs/latest/server_admin/index.html#saml-v2-0-identity-providers[documentation] for more details about the new options.

docs/documentation/server_admin/topics/user-federation/ldap.adoc

@@ -74,11 +74,9 @@ Hover the mouse pointer over the tooltips in the Admin Console to see more detai
 
 ==== Connecting to LDAP over SSL
 
-When you configure a secure connection URL to your LDAP store (for example,`ldaps://myhost.com:636`), {project_name} uses SSL to communicate with the LDAP server. Configure a truststore on the {project_name} server side so that {project_name} can trust the SSL connection to LDAP.
+When you configure a secure connection URL to your LDAP store (for example,`ldaps://myhost.com:636`), {project_name} uses SSL to communicate with the LDAP server. Configure a truststore on the {project_name} server side so that {project_name} can trust the SSL connection to LDAP - see https://www.keycloak.org/server/keycloak-truststore[Configuring a Truststore] {section}.
 
-Configure the global truststore for {project_name} with the Truststore SPI. For more information about configuring the global truststore, see the https://www.keycloak.org/server/keycloak-truststore[Configuring a Truststore] {section}. If you do not configure the Truststore SPI, the truststore falls back to the default mechanism provided by Java, which can be the file supplied by the `javax.net.ssl.trustStore` system property or the cacerts file from the JDK if the system property is unset.
-
-The `Use Truststore SPI` configuration property, in the LDAP federation provider configuration, controls the truststore SPI. By default, {project_name} sets the property to `Always`, which is adequate for most deployments.  {project_name} uses the Truststore SPI if the connection URL to LDAP starts with `ldaps` only.
+The `Use Truststore SPI` configuration property is deprecated.  It should normally be left as `Always`.
 
 ==== Synchronizing LDAP users to {project_name}
 

docs/documentation/upgrading/topics/keycloak/changes-24_0_0.adoc

@@ -12,3 +12,9 @@ import AuthZ from 'keycloak-js/dist/keycloak-authz.js';
 import Keycloak from 'keycloak-js';
 import AuthZ from 'keycloak-js/authz';
 ----
+
+= Truststore Changes
+
+The `spi-truststore-file-*` options and the truststore related options `https-trust-store-*` are deprecated, please use the new default location for truststore material, `conf/truststores`, or specify your desired paths via the `truststore-paths` option. For details refer to the relevant https://www.keycloak.org/server/keycloak-truststore[guide].
+
+The `tls-hostname-verifier` property should be used instead of the `spi-truststore-file-hostname-verification-policy` property.

docs/documentation/upgrading/topics/keycloak/changes.adoc

@@ -1,5 +1,9 @@
 == Migration Changes
 
+=== Migrating to 24.0.0
+
+include::changes-24_0_0.adoc[leveloffset=3]
+
 === Migrating to 23.0.2
 
 include::changes-23_0_2.adoc[leveloffset=3]

docs/guides/server/keycloak-truststore.adoc

@@ -2,52 +2,36 @@
 <#import "/templates/kc.adoc" as kc>
 
 <@tmpl.guide
-title="Configuring trusted certificates for outgoing requests"
-summary="How to configure the {project_name} Truststore to communicate with external services through TLS."
-includedOptions="">
+title="Configuring trusted certificates"
+summary="How to configure the {project_name} Truststore to communicate through TLS."
+includedOptions="truststore-paths tls-hostname-verifier">
 
-When {project_name} communicates with external services through TLS, it has to validate the remote server’s certificate in order to ensure it is connecting to a trusted server. This is necessary in order to prevent man-in-the-middle attacks. The certificates of these remote server’s or the CA that signed these certificates must be put in a truststore. This truststore is managed by the Keycloak server.
+When {project_name} communicates with external services or has an incoming connection through TLS, it has to validate the remote certificate in order to ensure it is connecting to a trusted server. This is necessary in order to prevent man-in-the-middle attacks.
 
-The truststore is used when connecting securely to identity brokers, LDAP identity providers, when sending emails, and for backchannel communication with client applications. It is also useful
-when you want to change the policy on how host names are verified and trusted by the server.
+The certificates of these clients or servers, or the CA that signed these certificates, must be put in a truststore. This truststore is then configured for use by Keycloak.
 
-By default, a truststore provider is not configured, and any TLS/HTTPS connections fall back to standard Java Truststore configuration. If there is no trust established, then these outgoing requests will fail.
+== Configuring the System Truststore
 
-== Configuring the {project_name} Truststore
+The existing Java default truststore certs will always be trusted.  If you need additional certificates, which will be the case if you have self-signed or internal certificate authorities that are not recognized by the JRE, they can be included in the `conf/truststores` directory or subdirectories.  The certs may be in PEM files, or PKCS12 files with extension `.p12` or `.pfx`.  If in PKCS12, the certs must be unencrypted - meaning no password is expected.
 
-You can add your truststore configuration by entering this command:
+If you need an alternative path, use the `--truststore-paths` option to specify additional files or directories where PEM or PKCS12 files are located. Paths are relative to where you launched {project_name}, so absolute paths are recommended instead. If a directory is specified, it will be recursively scanned for truststore files.
 
-<@kc.start parameters="--spi-truststore-file-file=myTrustStore.jks --spi-truststore-file-password=password --spi-truststore-file-hostname-verification-policy=ANY"/>
+After all applicable certs are included, the truststore will be used as the system default truststore via the `javax.net.ssl` properties, and as the default for internal usage within {project_name}.  
 
-The following are possible configuration options for this setting:
+For example:
 
-file::
-The path to a Java keystore file.
-TLS requests need a way to verify the host of the server to which they are talking.
-This is what the truststore does.
-The keystore contains one or more trusted host certificates or certificate authorities.
-This truststore file should only contain public certificates of your secured hosts.
-This is _REQUIRED_ if any of these properties are defined.
+<@kc.start parameters="--truststore-paths=/opt/truststore/myTrustStore.pfx,/opt/other-truststore/myOtherTrustStore.pem" />
 
-password::
-Password of the keystore.
-This option is _REQUIRED_ if any of these properties are defined.
+It is still possible to directly set your own `javax.net.ssl` truststore System properties, but it's recommended to use the `--truststore-paths` instead.
 
-hostname-verification-policy::
-For HTTPS requests, this option verifies the hostname of the server's certificate. Default: `WILDCARD`
+== Hostname Verification Policy
+
+You may refine how hostnames are verified by TLS connections with the `tls-hostname-verifier` property.
+
+* `WILDCARD` (the default) allows wildcards in subdomain names, such as *.foo.com.
 * `ANY` means that the hostname is not verified.
-* `WILDCARD` allows wildcards in subdomain names, such as *.foo.com.
 * When using `STRICT`, the Common Name (CN) must match the hostname exactly.
 +
 Please note that this setting does not apply to LDAP secure connections, which require strict hostname checking.
 
-type::
-The type of truststore, such as `jks`, `pkcs12` or `bcfks`. If not provided, the type would be detected based on the truststore
-file extension or platform default type.
-
-=== Example of a truststore configuration
-The following is an example configuration for a truststore that allows you to create trustful connections to all `mycompany.org` domains and its subdomains:
-
-<@kc.start parameters="--spi-truststore-file-file=path/to/truststore.jks --spi-truststore-file-password=change_me --spi-truststore-file-hostname-verification-policy=WILDCARD"/>
-
 </@tmpl.guide>

quarkus/config-api/src/main/java/org/keycloak/config/HttpOptions.java

@@ -89,17 +89,17 @@ public enum ClientAuth {
 
     public static final Option HTTPS_TRUST_STORE_FILE = new OptionBuilder<>("https-trust-store-file", File.class)
             .category(OptionCategory.HTTP)
-            .description("The trust store which holds the certificate information of the certificates to trust.")
+            .description("DEPRECATED: The trust store which holds the certificate information of the certificates to trust.")
             .build();
 
     public static final Option HTTPS_TRUST_STORE_PASSWORD = new OptionBuilder<>("https-trust-store-password", String.class)
             .category(OptionCategory.HTTP)
-            .description("The password of the trust store file.")
+            .description("DEPRECATED: The password of the trust store file.")
             .build();
 
     public static final Option<String> HTTPS_TRUST_STORE_TYPE = new OptionBuilder<>("https-trust-store-type", String.class)
             .category(OptionCategory.HTTP)
-            .description("The type of the trust store file. " +
+            .description("DEPRECATED: The type of the trust store file. " +
                     "If not given, the type is automatically detected based on the file name. " +
                     "If '" + SecurityOptions.FIPS_MODE.getKey() + "' is set to '" + FipsMode.STRICT + "' and no value is set, it defaults to 'BCFKS'.")
             .build();

quarkus/config-api/src/main/java/org/keycloak/config/OptionCategory.java

@@ -7,13 +7,14 @@ public enum OptionCategory {
     TRANSACTION("Transaction",30, ConfigSupportLevel.SUPPORTED),
     FEATURE("Feature", 40, ConfigSupportLevel.SUPPORTED),
     HOSTNAME("Hostname", 50, ConfigSupportLevel.SUPPORTED),
-    HTTP("HTTP/TLS", 60, ConfigSupportLevel.SUPPORTED),
+    HTTP("HTTP(S)", 60, ConfigSupportLevel.SUPPORTED),
     HEALTH("Health", 70, ConfigSupportLevel.SUPPORTED),
     CONFIG("Config", 75, ConfigSupportLevel.SUPPORTED),
     METRICS("Metrics", 80, ConfigSupportLevel.SUPPORTED),
     PROXY("Proxy", 90, ConfigSupportLevel.SUPPORTED),
     VAULT("Vault", 100, ConfigSupportLevel.SUPPORTED),
     LOGGING("Logging", 110, ConfigSupportLevel.SUPPORTED),
+    TRUSTSTORE("Truststore", 115, ConfigSupportLevel.SUPPORTED),
     SECURITY("Security", 120, ConfigSupportLevel.SUPPORTED),
     EXPORT("Export", 130, ConfigSupportLevel.SUPPORTED),
     IMPORT("Import", 140, ConfigSupportLevel.SUPPORTED),

quarkus/config-api/src/main/java/org/keycloak/config/SecurityOptions.java

@@ -1,11 +1,11 @@
 package org.keycloak.config;
 
-import java.util.Arrays;
-import java.util.List;
-
 import org.keycloak.common.Profile;
 import org.keycloak.common.crypto.FipsMode;
 
+import java.util.Arrays;
+import java.util.List;
+
 public class SecurityOptions {
 
     public static final Option<FipsMode> FIPS_MODE = new OptionBuilder<>("fips-mode", FipsMode.class)

quarkus/config-api/src/main/java/org/keycloak/config/TruststoreOptions.java

@@ -0,0 +1,18 @@
+package org.keycloak.config;
+
+import org.keycloak.common.enums.HostnameVerificationPolicy;
+
+public class TruststoreOptions {
+
+    public static final Option<String> TRUSTSTORE_PATHS = new OptionBuilder<>("truststore-paths", String.class)
+            .category(OptionCategory.TRUSTSTORE)
+            .description("List of pkcs12 (p12 or pfx file extensions), PEM files, or directories containing those files that will be used as a system truststore.")
+            .build();
+
+    public static final Option<HostnameVerificationPolicy> HOSTNAME_VERIFICATION_POLICY = new OptionBuilder<>("tls-hostname-verifier", HostnameVerificationPolicy.class)
+            .category(OptionCategory.TRUSTSTORE)
+            .description("The TLS hostname verification policy for out-going HTTPS and SMTP requests.")
+            .defaultValue(HostnameVerificationPolicy.WILDCARD)
+            .build();
+
+}

quarkus/deployment/src/main/java/org/keycloak/quarkus/deployment/KeycloakProcessor.java

@@ -230,6 +230,13 @@ void configureProfile(KeycloakRecorder recorder) {
         recorder.configureProfile(profile.getName(), profile.getFeatures());
     }
 
+    @Record(ExecutionTime.STATIC_INIT)
+    @BuildStep
+    @Consume(ConfigBuildItem.class)
+    void configureTruststore(KeycloakRecorder recorder) {
+        recorder.configureTruststore();
+    }
+
     /**
      * Check whether JDBC driver is present for the specified DB
      *

quarkus/dist/assembly.xml

@@ -80,6 +80,14 @@
                 <include>*.*</include>
             </includes>
         </fileSet>
+        <!-- create an empty truststores directory -->
+        <fileSet>
+            <directory>src/main/content/conf</directory>
+            <outputDirectory>conf/truststores</outputDirectory>
+            <excludes>
+                <exclude>**/*</exclude>
+            </excludes>
+        </fileSet>
         <fileSet>
             <directory>src/main/content/data</directory>
             <outputDirectory>data</outputDirectory>

quarkus/dist/src/main/content/conf/README.md

@@ -1,4 +1,7 @@
 Configure the server
 ====================
 
-Files in this directory are used to configure the server. Please consult the [configuration guides](https://www.keycloak.org/guides#server) for more information.
+Files in this directory are used to configure the server. Please consult the [configuration guides](https://www.keycloak.org/guides#server) for more information.
+
+Use the truststores subdirectory to provide additional trusted certificates.  Please consult the [truststore guide](https://www.keycloak.org/server/keycloak-truststore) for more information.
+

quarkus/runtime/src/main/java/org/keycloak/quarkus/runtime/KeycloakRecorder.java

@@ -17,11 +17,13 @@
 
 package org.keycloak.quarkus.runtime;
 
+import java.io.File;
 import java.lang.annotation.Annotation;
 import java.util.List;
 import java.util.Map;
+import java.util.Optional;
 import java.util.function.BiConsumer;
-import java.util.function.Predicate;
+import java.util.stream.Stream;
 
 import io.agroal.api.AgroalDataSource;
 import io.quarkus.agroal.DataSource;
@@ -32,13 +34,13 @@
 
 import org.hibernate.cfg.AvailableSettings;
 import org.infinispan.manager.DefaultCacheManager;
-import io.vertx.ext.web.RoutingContext;
 
 import org.keycloak.Config;
 import org.keycloak.common.Profile;
 import org.keycloak.common.crypto.CryptoIntegration;
 import org.keycloak.common.crypto.CryptoProvider;
 import org.keycloak.common.crypto.FipsMode;
+import org.keycloak.config.TruststoreOptions;
 import org.keycloak.quarkus.runtime.configuration.Configuration;
 import org.keycloak.quarkus.runtime.configuration.MicroProfileConfigProvider;
 import org.keycloak.quarkus.runtime.integration.QuarkusKeycloakSessionFactory;
@@ -48,6 +50,7 @@
 import org.keycloak.provider.Spi;
 import org.keycloak.quarkus.runtime.storage.legacy.infinispan.CacheManagerFactory;
 import org.keycloak.theme.ClasspathThemeProviderFactory;
+import org.keycloak.truststore.TruststoreBuilder;
 
 import io.quarkus.runtime.RuntimeValue;
 import io.quarkus.runtime.ShutdownContext;
@@ -68,6 +71,23 @@ public void configureProfile(Profile.ProfileName profileName, Map<Profile.Featur
         Profile.init(profileName, features);
     }
 
+    public void configureTruststore() {
+        String[] truststores = Configuration.getOptionalKcValue(TruststoreOptions.TRUSTSTORE_PATHS.getKey())
+                .map(s -> s.split(",")).orElse(new String[0]);
+
+        String dataDir = Environment.getDataDir();
+
+        File truststoresDir = Optional.ofNullable(Environment.getHomePath()).map(path -> path.resolve("conf").resolve("truststores").toFile()).orElse(null);
+
+        if (truststoresDir != null && truststoresDir.exists() && Optional.ofNullable(truststoresDir.list()).map(a -> a.length).orElse(0) > 0) {
+            truststores = Stream.concat(Stream.of(truststoresDir.getAbsolutePath()), Stream.of(truststores)).toArray(String[]::new);
+        } else if (truststores.length == 0) {
+            return; // nothing to configure, we'll just use the system default
+        }
+
+        TruststoreBuilder.setSystemTruststore(truststores, true, dataDir);
+    }
+
     public void configureLiquibase(Map<String, List<String>> services) {
         ServiceLocator locator = Scope.getCurrentScope().getServiceLocator();
         if (locator instanceof FastServiceLocator)

quarkus/runtime/src/main/java/org/keycloak/quarkus/runtime/configuration/mappers/PropertyMappers.java

@@ -40,6 +40,7 @@ private PropertyMappers(){}
         MAPPERS.addAll(SecurityPropertyMappers.getMappers());
         MAPPERS.addAll(ExportPropertyMappers.getMappers());
         MAPPERS.addAll(ImportPropertyMappers.getMappers());
+        MAPPERS.addAll(TruststorePropertyMappers.getMappers());
     }
 
     public static ConfigValue getValue(ConfigSourceInterceptorContext context, String name) {

quarkus/runtime/src/main/java/org/keycloak/quarkus/runtime/configuration/mappers/SecurityPropertyMappers.java

@@ -21,7 +21,7 @@ public static PropertyMapper[] getMappers() {
         return new PropertyMapper[] {
                 fromOption(SecurityOptions.FIPS_MODE).transformer(SecurityPropertyMappers::resolveFipsMode)
                         .paramLabel("mode")
-                        .build()
+                        .build(),
         };
     }
 

quarkus/runtime/src/main/java/org/keycloak/quarkus/runtime/configuration/mappers/TruststorePropertyMappers.java

@@ -0,0 +1,21 @@
+package org.keycloak.quarkus.runtime.configuration.mappers;
+
+import org.keycloak.config.TruststoreOptions;
+
+import static org.keycloak.quarkus.runtime.configuration.mappers.PropertyMapper.fromOption;
+
+public class TruststorePropertyMappers {
+
+    public static PropertyMapper[] getMappers() {
+        return new PropertyMapper[] {
+                fromOption(TruststoreOptions.TRUSTSTORE_PATHS)
+                        .paramLabel(TruststoreOptions.TRUSTSTORE_PATHS.getKey())
+                        .build(),
+                fromOption(TruststoreOptions.HOSTNAME_VERIFICATION_POLICY)
+                        .paramLabel(TruststoreOptions.HOSTNAME_VERIFICATION_POLICY.getKey())
+                        .to("kc.spi-truststore-file-hostname-verification-policy")
+                        .build(),
+        };
+    }
+
+}