Ability to configure SSL truststore and keystore for SR Clients #957

mageshn · 2018-12-14T01:35:46Z

Historically, SR clients were required to specify the key store and trust store settings via the JVM args. This was not consistent with the rest of the platform and AK. This PR enabled the ability to specify these via configurations. If the approach looks good, I will revise this PR to include doc changes.

jonchiu

LGTM, @xvrl ?

xvrl · 2018-12-18T18:11:31Z

client/src/main/java/io/confluent/kafka/schemaregistry/client/SchemaRegistryClientConfig.java

@@ -23,4 +23,51 @@
      "schema.registry.basic.auth.user.info";
  public static final String USER_INFO_CONFIG =
      "basic.auth.user.info";
+
+  public static final String SSL_KEYSTORE_LOCATION_CONFIG = "ssl.keystore.location";


is there a way we can avoid duplicating those configs and somehow reuse existing configdefs?

The existing config defs were only on the server side. Moreover, on the client side, the configs need namespacing as well.

Can we extract the configs used across clients and server into a single utility with various methods for adding config sets to a ConfigDef?

@mageshn, I used AK security configs client side in Elastic connector: confluentinc/kafka-connect-elasticsearch#278

@rhauch I already extracted the configs to the client side and the server side also uses the same. @cyrusv unfortunately, we can't use the AK configdef as-is since SR uses the config-def from rest-utils

I don't understand why config-def from rest-utils precludes using AK configs?

@mageshn Any update? Can you fix the conflict as well?

yohei1126 · 2019-05-07T00:26:03Z

any update?

cyrusv · 2019-05-08T16:47:48Z

@mageshn, the ES Connector also is an SSL Client, and I think this strategy of code re-use works really well. WDYT?
https://github.com/confluentinc/kafka-connect-elasticsearch/pull/278/files

PraveenAdlakha · 2019-05-14T04:34:14Z

Can some one please take look at it and merge it ? If its just waiting on conflict resolution I can take it up?

yohei1126 · 2019-05-17T07:59:23Z

Would you check it? I want to use this feature.

PraveenAdlakha · 2019-05-17T10:14:07Z

@yohei1126 I can...But I need some support of commiters who can merge my code.

rhauch

A few comments so far.

rhauch · 2019-02-27T20:47:37Z

avro-serializer/src/main/java/io/confluent/kafka/serializers/AbstractKafkaAvroSerDeConfig.java

-    super(config, props);
+            Importance.MEDIUM, VALUE_SUBJECT_NAME_STRATEGY_DOC)
+        .define(
+            CLIENT_NAMESPACE + SchemaRegistryClientConfig.SSL_KEYSTORE_LOCATION_CONFIG,


IIUC this will create a config name of schema.registryssl.keystore.location without a . between registry and ssl. Should the CLIENT_NAMESPACE constant value end with . to make concatenation easier?

rhauch · 2019-02-27T20:49:11Z

client/src/main/java/io/confluent/kafka/schemaregistry/client/security/SslFactory.java

    try {
+      createKeystore(
+          (String) configs.get(prefix + SchemaRegistryClientConfig.SSL_KEYSTORE_TYPE_CONFIG),


Same question here about the prefix needed a trailing ..

rhauch · 2019-02-27T20:50:20Z

core/src/main/java/io/confluent/kafka/schemaregistry/storage/KafkaSchemaRegistry.java

@@ -118,7 +118,7 @@ public KafkaSchemaRegistry(SchemaRegistryConfig config,
    this.isEligibleForMasterElector = config.getBoolean(SchemaRegistryConfig.MASTER_ELIGIBILITY);
    this.myIdentity = new SchemaRegistryIdentity(host, schemeAndPort.port,
        isEligibleForMasterElector, schemeAndPort.scheme);
-    this.sslFactory = new SslFactory(config);
+    this.sslFactory = new SslFactory(config.originalsWithPrefix(""), false);


Why not originals()?

rhauch · 2019-07-22T16:44:49Z

@mageshn do you want to resolve the conflicts?

rayokota

LGTM

cyrusv · 2019-08-05T20:20:22Z

There doesn't seem to be any testing.

cyrusv · 2019-09-03T14:52:40Z

Thanks for the integration tests -- they will pay off for stability +1

rayokota

Thanks for the tests @mageshn ! Just left a minor comment, otherwise LGTM!

rayokota · 2019-09-03T17:13:15Z

core/src/test/java/io/confluent/kafka/schemaregistry/rest/RestApiSslTest.java

+      props.put(SchemaRegistryConfig.SSL_CLIENT_AUTHENTICATION_CONFIG, SchemaRegistryConfig.SSL_CLIENT_AUTHENTICATION_REQUIRED);
+
+    } catch (Exception e) {
+      e.printStackTrace();


Do you want to rethrow the exception?

fml2 · 2020-01-30T17:11:19Z

Is it possible to specify the keystore and the truststore (not their locations!) via API? The goal is to work without having to deal with the file system alltogether. This is of importance in e.g. cloud environment.

Thanks

maduris · 2020-02-26T06:03:56Z

hi @mageshn,
I am using the client 5.4.0
+- io.confluent:kafka-avro-serializer:jar:5.4.0:compile
[INFO] | +- io.confluent:kafka-schema-registry-client:jar:5.4.0:compile

I am still seeing
PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

I am just wondering if the new jar is only required in the clients or would the schema registry server needs to be update to 5.4.0 also

sergeiwaigant · 2020-05-12T12:03:57Z

I am still facing that issue with the required JVM level properties for SR clients using Avro SerDe.
At least with the kafka-avro-console-producer...

## rpm -qa confluent*
confluent-common-5.4.2-1.noarch
confluent-control-center-fe-5.4.2-1.noarch
confluent-control-center-5.4.2-1.noarch
confluent-hub-client-5.4.2-1.noarch
confluent-kafka-2.12-5.4.2-1.noarch
confluent-rest-utils-5.4.2-1.noarch
confluent-rebalancer-5.4.2-1.noarch
confluent-schema-registry-5.4.2-1.noarch

## cat /etc/ikep/secrets/local_client.ssl.properties
security.protocol=SSL
ssl.truststore.location=/etc/ikep/secrets/local_client.truststore.p12
ssl.truststore.type=PKCS12
ssl.truststore.password=localPwd
ssl.keystore.location=/etc/ikep/secrets/local_client.keystore.p12
ssl.keystore.type=PKCS12
ssl.keystore.password=localPwd
ssl.key.password=localPwd
ssl.endpoint.identification.algorithm=

## Start avro-console-producer
echo '{"f1":"value1"}' | kafka-avro-console-producer \
   --broker-list kafka:9${IKEP_LOCAL_ENV:-09}5 \
   --topic local_a_testAvro \
   --producer.config /etc/ikep/secrets/local_client.ssl.properties \
   --property value.schema='{"type":"record","name":"myrecord","fields":[{"name":"f1","type":"string"}]}' \
   --property schema.registry.url="https://schema-registry:8${IKEP_LOCAL_ENV:-08}1"

[2020-05-12 12:00:50,613] ERROR Failed to send HTTP request to endpoint: https://schema-registry:8081/subjects/local_a_testAvro-value/versions (io.confluent.kafka.schemaregistry.client.rest.RestService:254)
javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:320)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:263)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:258)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:645)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:464)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:360)
	at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421)
	at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:177)
	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164)
	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1151)
	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1062)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:402)
	at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:567)
	at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1362)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1337)
	at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:246)
	at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:250)
	at io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:334)
	at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:434)
	at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:426)
	at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:412)
	at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.registerAndGetId(CachedSchemaRegistryClient.java:140)
	at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.register(CachedSchemaRegistryClient.java:196)
	at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.register(CachedSchemaRegistryClient.java:172)
	at io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.serializeImpl(AbstractKafkaAvroSerializer.java:71)
	at io.confluent.kafka.formatter.AvroMessageReader.readMessage(AvroMessageReader.java:181)
	at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:55)
	at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439)
	at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306)
	at java.base/sun.security.validator.Validator.validate(Validator.java:264)
	at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:222)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:629)
	... 27 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
	at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
	at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434)
	... 33 more
org.apache.kafka.common.errors.SerializationException: Error serializing Avro message
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:320)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:263)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:258)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:645)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:464)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:360)
	at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421)
	at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:177)
	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164)
	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1151)
	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1062)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:402)
	at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:567)
	at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1362)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1337)
	at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:246)
	at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:250)
	at io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:334)
	at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:434)
	at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:426)
	at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:412)
	at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.registerAndGetId(CachedSchemaRegistryClient.java:140)
	at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.register(CachedSchemaRegistryClient.java:196)
	at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.register(CachedSchemaRegistryClient.java:172)
	at io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.serializeImpl(AbstractKafkaAvroSerializer.java:71)
	at io.confluent.kafka.formatter.AvroMessageReader.readMessage(AvroMessageReader.java:181)
	at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:55)
	at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439)
	at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306)
	at java.base/sun.security.validator.Validator.validate(Validator.java:264)
	at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:222)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:629)
	... 27 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
	at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
	at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434)
	... 33 more

ddhanak · 2021-05-04T01:01:58Z

What is the solution here? What configuration can we use to specify a SR that uses a truststore/keystore?
I tried pretty much everything and nothing seems to be working.

mageshn requested review from xvrl, jonchiu and a team December 14, 2018 01:36

jonchiu approved these changes Dec 17, 2018

View reviewed changes

xvrl reviewed Dec 18, 2018

View reviewed changes

mageshn mentioned this pull request Feb 5, 2019

Producer with Schema Registry connection not using ssl settings #943

Open

ppyzik approved these changes May 6, 2019

View reviewed changes

tchiotludo mentioned this pull request Jul 11, 2019

Support schema registry connection with TLS mutual authentication tchiotludo/akhq#88

Closed

rhauch reviewed Jul 22, 2019

View reviewed changes

mageshn added 2 commits July 29, 2019 10:24

Ability to configure SSL truststore and keystore for SR Clients

57354af

Rebase & use configs defs from AK

2b4bc11

mageshn force-pushed the CC-3074 branch from 1bc4206 to 2b4bc11 Compare July 29, 2019 21:48

Fix formatting

d2da35e

rayokota approved these changes Aug 5, 2019

View reviewed changes

Add integration test for SSL

fd84db3

cyrusv approved these changes Sep 3, 2019

View reviewed changes

rayokota approved these changes Sep 3, 2019

View reviewed changes

mageshn and others added 2 commits September 10, 2019 15:41

throw exception in test case

da94f60

Merge branch 'master' into CC-3074

0ec98b0

mageshn merged commit 18144c8 into master Sep 17, 2019

mageshn deleted the CC-3074 branch September 17, 2019 18:41

ybyzek mentioned this pull request Dec 5, 2019

DEVX-1214: Ability to configure SSL truststore and keystore for SR Clients confluentinc/cp-demo#140

Merged

dhuang mentioned this pull request Sep 16, 2020

Invalid SSL configs breaking connector snowflakedb/snowflake-kafka-connector#221

Closed

andsel mentioned this pull request Feb 14, 2023

Adds schema registry's truststore and keystore settings logstash-plugins/logstash-integration-kafka#137

Merged

5 tasks

tilgalas mentioned this pull request Mar 12, 2024

[Task]: Update confluent libraries versions from 5.3.2 to more recent in kafka io extension apache/beam#30610

Closed

16 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ability to configure SSL truststore and keystore for SR Clients #957

Ability to configure SSL truststore and keystore for SR Clients #957

mageshn commented Dec 14, 2018

jonchiu left a comment

xvrl Dec 18, 2018

mageshn Jan 15, 2019

rhauch Jan 15, 2019

cyrusv Feb 26, 2019

mageshn Feb 26, 2019

cyrusv Feb 26, 2019

yohei1126 May 17, 2019

yohei1126 commented May 7, 2019

cyrusv commented May 8, 2019

PraveenAdlakha commented May 14, 2019

yohei1126 commented May 17, 2019

PraveenAdlakha commented May 17, 2019

rhauch left a comment

rhauch Feb 27, 2019

rhauch Feb 27, 2019

rhauch Feb 27, 2019

rhauch commented Jul 22, 2019

rayokota left a comment

cyrusv commented Aug 5, 2019

cyrusv commented Sep 3, 2019

rayokota left a comment

rayokota Sep 3, 2019

fml2 commented Jan 30, 2020

maduris commented Feb 26, 2020

sergeiwaigant commented May 12, 2020

ddhanak commented May 4, 2021 •

edited

Ability to configure SSL truststore and keystore for SR Clients #957

Ability to configure SSL truststore and keystore for SR Clients #957

Conversation

mageshn commented Dec 14, 2018

jonchiu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yohei1126 commented May 7, 2019

cyrusv commented May 8, 2019

PraveenAdlakha commented May 14, 2019

yohei1126 commented May 17, 2019

PraveenAdlakha commented May 17, 2019

rhauch left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rhauch commented Jul 22, 2019

rayokota left a comment

Choose a reason for hiding this comment

cyrusv commented Aug 5, 2019

cyrusv commented Sep 3, 2019

rayokota left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fml2 commented Jan 30, 2020

maduris commented Feb 26, 2020

sergeiwaigant commented May 12, 2020

ddhanak commented May 4, 2021 • edited

ddhanak commented May 4, 2021 •

edited