unable to find valid certification path to requested target

[joettt]

Hello, I have set up a new solr server and configured to use TLS/SSL and have been successful in running Solr with https. I am also able to crawl the site using the Norconex crawler. But I am getting com.norconex.committer.core.CommitterException when the committer tries to index the documents. Looking at the attached logs, I see the message "unable to find valid certification path to requested target". In my attached config file (pol2.txt), I have even tried adding
<httpClientFactory> <trustAllSSLCertificates>true</trustAllSSLCertificates> </httpClientFactory> as suggested in https://github.com/Norconex/collector-http/issues/100
but I still get the same error.
Any advice will be greatly appreciated.
pol_32_crawler.log
pol2.txt

[joettt]

Revised config file
pol2.txt

[essiembre]

Are you using a valid/trusted certificate? The trustAllSSLCertificates option resolves a lot of cases, but in this case, I would try to add the certificate manually to your java key store:

https://docs.oracle.com/javase/tutorial/security/toolfilex/rstep1.html

If the communication between your consuming app and Solr is all behind a firewall, you could also drop the encryption, which should also improve performance slightly.

[joettt]

Yes I am using a valid certificate Following are the steps I have used to add the certificate manually.

I have created a keystore file and have enabled SSL on my solr server using the following procedures:

1. Created pkcs#12 file using the command:

Keytool –genkey –alias aliasname –keystore /solr-ssl.keystore.pfx –storetype PKCS12 –keyalg RSA –storepass password –ext SAN=dns:localhost,dns:solr-devapp01.devt1.restOfDomain –validity 730 –keysize 2048

2. Imported the pkcs keystore file into Trusted Root Certification Authority

3. Copied the pkcs file solr-ssl.keystore.pfx to the solr /server/etc folder

4. Modified solr.in.cmd file with the following:

set SOLR_SSL_ENABLED=true

set SOLR_SSL_KEY_STORE=etc/solr-ssl.keystore.pfx
set SOLR_SSL_KEY_STORE_PASSWORD=secret
set SOLR_SSL_TRUST_STORE=etc/solr-ssl.keystore.pfx
set SOLR_SSL_TRUST_STORE_PASSWORD=secret

set SOLR_SSL_NEED_CLIENT_AUTH=false
set SOLR_SSL_WANT_CLIENT_AUTH=false
set SOLR_SSL_KEY_STORE_TYPE=PKCS12
set SOLR_SSL_TRUST_STORE_TYPE=PKCS12

[joettt]

I wonder if there is anything else I need to do for the documents to be committed. WIth above steps, I am able to crawl but not commit.

[jdbranham]

@joettt
You may want to verify the httpcrawler is using the truststore that trusts your cert.
Are you using the same Java installation for both apps?
How did you import the cert into the truststore?

[joettt]

Hi Jeremy, yes I am using the same java installation for both apps. I imported the cert using Certificte Import Wizrd.

[joettt]

How do I verify that the httpcrawler is infact using the same truststore? In the httpcrawler configuration, I have used the trustAllSSLCertificates option.

[joettt]

The httpcrawler has no problem communicating with the trusted sites but the Norconex Committer is unable to recognize the certificate

[jdbranham]

I'm not familiar with the wizard, but typically you'll import your CA file into the truststore. Since you're using a self signed keypair, you'll probably need to export your cer.pem like this -
[be sure to use the same alias as when you generated the keypair] .

keytool -export -keystore ./solr-ssl.keystore.pfx -storetype pkcs12 -alias localhost -file ./cer.pem

Then import it into your CA trustore [typically JAVA_HOME/jre/lib/security/cacerts]

keytool -import -trustcacerts -keystore ./cacerts -alias localhost -file ./cer.pem

[jdbranham]

The default truststore is 'JAVA_HOME/jre/lib/security/cacerts' If no Java arguments have been provided to the app to change it.
You might be able to verify the JAVA_HOME and cacerts location in the application logs.

[joettt]

My truststore is at JAVA_HOME/lib/security/cacerts which for some reason committer is unable to detect.

[jdbranham]

You could add a java arg to tell the app which truststore to use -
-Djavax.net.ssl.trustStore=/path/to/truststore

Also, here is a gist [SSLPoke] that helps debug keystore/trust issues -
https://gist.github.com/4ndrej/4547029

[joettt]

Thanks again Jeremy.
Pascal, can I tell the committer which appstore to use?

[essiembre]

You would do it like @jdbranham suggests, on Java itself when it launches. Modify the collector-http.sh script (or .bat on Windows) and specify it there as an extra argument (right after the java command).

[joettt]

Thank you @jdbranham and Pascal. The issue is now resolved by following these steps

1. keytool -export -keystore ./solr-ssl.keystore.pfx -storetype pkcs12 -alias localhost -file ./cer.pem
2. keytool -import -trustcacerts -keystore ./cacerts -alias localhost -file ./cer.pem
3. Adding Djavax.net.ssl.trustStore=/path/to/truststore to collector-http.bat