TLS 1.2 client certificate authentification

[alexvrv]

Hi, is there a way for send the certificate when opening a page? The webpage that I try to open requests a certificate right at the opening (https://forexe.mfinante.gov.ro). I can get it done with HttpURLConnection but I need to enable javascript that's why I'm trying with htmlunit.

SSLContext sc = SSLContext.getInstance("TLS");
sc.init(new X509ExtendedKeyManager[] {km}, null, null);

URL url = new URL("https://webserviceapl.anaf.ro/prod/FCTEL/rest/stareMesaj?id_incarcare=" + msg.id_incarcare);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
if (connection instanceof HttpsURLConnection) {
((HttpsURLConnection) connection)
.setSSLSocketFactory(sc.getSocketFactory());
}
connection.setConnectTimeout(100000);
connection.setReadTimeout(100000);
connection.setInstanceFollowRedirects(true);
connection.setDoOutput(true);

I need to include that SSLContext in HtmlUnit somehow if it is even possible...

[rbri]

You can have a look at the class org.htmlunit.httpclient.HtmlUnitSSLConnectionSocketFactory to find all the details of the current impl.

The SSL Context it created by this code

final SSLContext sslContext = SSLContext.getInstance(protocol);
sslContext.init(getKeyManagers(options), new X509ExtendedTrustManager[] {new InsecureTrustManager()}, null);

and getKeyMangers looks like this

private static KeyManager[] getKeyManagers(final WebClientOptions options) {
    if (options.getSSLClientCertificateStore() == null) {
        return null;
    }
    try {
        final KeyStore keyStore = options.getSSLClientCertificateStore();
        final KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(
                KeyManagerFactory.getDefaultAlgorithm());
        keyManagerFactory.init(keyStore, options.getSSLClientCertificatePassword());
        return keyManagerFactory.getKeyManagers();
    }
    catch (final Exception e) {
        throw new RuntimeException(e);
    }
}

Means you can provide your own 'SSLClientCertificateStore'. Is that sufficient for you?

[alexvrv]

And how can I provide my own Store? I think I just need to replace the SSLContext instantiated by me, but I don't see how xD

webClient.getOptions().setSSLClientCertificateStore(); is not public...

[rbri]

try (InputStream certificateInputStream = getClass().getClassLoader()
        .getResourceAsStream("insecureSSL.pfx")) {
    final byte[] certificateBytes = new byte[4096];
    certificateInputStream.read(certificateBytes);

    try (InputStream is = new ByteArrayInputStream(certificateBytes)) {
        webClient.getOptions().setSSLClientCertificate(is, "nopassword", "PKCS12");
        webClient.getOptions().setUseInsecureSSL(true);

        final URL https = new URL("https://localhost:" + PORT2 + "/");
        loadPage("<div>test</div>", https);
    }
}

[alexvrv]

I can't use webClient.getOptions().setSSLClientCertificateStore(); is not public...
And a second problem would be that I don't have a PFX file. I have a USB stick that the browser need to acces and request the password. The certificate is installed in Windows.

[rbri]

The sample above uses setSSLClientCertificate and this is public.

You can create a your own certificate store and add the certificate to this one....

But yes you are right the was is a bit hard at the moment. Will have a look but this might require some time.

[rbri]

Can you please give me some more details about your use case - what do you have to do when using a real browser.

[alexvrv]

On this website https://forexe.mfinante.gov.ro/ there is a table with some files that I need to download once a month. I want to automate it. But the website require a Digital Signature Login, a certificate from a USB (same one that I use to sign PDFs for the government). I can't get past that login. And after the login the page is javascript generated and I can't use HttpsURLConnection...

The USB Signature has a software SafeNet Authentification Client. I think the login system is called TLSv1.2 authentification.

[rbri]

Let me google a bit over the weekend ...

[rbri]

can you please check the source code of the web site for some Applet or ActiveX plugin....

[alexvrv]

I think I got it to work but now i have a new problem. The USB contains 4 same signatures, 3 expired and 1 valid. I any way i send the certificate to setSSLClientCertificate, it picks the expired certificate xD

[rbri]

maybe you can use the Keytool to remove the expired ones?
https://docs.oracle.com/javase/8/docs/technotes/tools/unix/keytool.html

[alexvrv]

I think the problem is that the InputStream that I send to webClient.getOptions().setSSLClientCertificate is ignored somehow. I think it loads again all the certificates from "Windows-MY"...

Certificate cert = ks.getCertificate(aliasKey);
InputStream is = new ByteArrayInputStream(cert.getEncoded());
webClient.getOptions().setSSLClientCertificate(is, "", "Windows-MY");

[rbri]

any chance to debug it at your end?

[alexvrv]

If you mean TeamViewer or smth, no I can't... Company PC and stuff xD

[rbri]

no the idea is that you are debugging this ....

[alexvrv]

Yep it loads all the certificates again, doesn't matter what InputStream I send...

                Certificate cert = ks.getCertificate(aliasKey);

                InputStream is = new ByteArrayInputStream(cert.getEncoded());

                webClient.getOptions().setSSLClientCertificate(is, "", "Windows-MY");

                KeyStore ks2 = webClient.getOptions().getSSLClientCertificateStore();
                Enumeration<String> en2 = ks2.aliases();
                while (en2.hasMoreElements()) {
                    String aliasKey2 = en2.nextElement();
                    Date certExpiryDate2 = ((X509Certificate) ks.getCertificate(aliasKey2)).getNotAfter();
                    Date today2 = new Date();
                    long dateDiff2 = certExpiryDate2.getTime() - today2.getTime();
                    long expiresIn2 = dateDiff2 / (24 * 60 * 60 * 1000);

                    System.out.println(aliasKey2 + " - " + expiresIn2);
                }

returns all certificates that I have on windows ...

[rbri]

will check later today (i have a real job - this means i have to attend to some meetings :-D)

[rbri]

public void setSSLClientCertificate(final InputStream certificateInputStream, final String certificatePassword,
        final String certificateType) {
    try {
        sslClientCertificateStore_ = getKeyStore(certificateInputStream, certificatePassword, certificateType);
        sslClientCertificatePassword_ = certificatePassword == null ? null : certificatePassword.toCharArray();
    }
    catch (final Exception e) {
        throw new RuntimeException(e);
    }
}

private static KeyStore getKeyStore(final InputStream inputStream, final String keystorePassword,
        final String keystoreType)
                throws IOException, KeyStoreException, NoSuchAlgorithmException, CertificateException {
    if (inputStream == null) {
        return null;
    }

    final KeyStore keyStore = KeyStore.getInstance(keystoreType);
    final char[] passwordChars = keystorePassword == null ? null : keystorePassword.toCharArray();
    keyStore.load(inputStream, passwordChars);
    return keyStore;
}

public KeyStore getSSLClientCertificateStore() {
    return sslClientCertificateStore_;
}

Maybe you can write your own test options class with only these methods and a private KeyStore sslClientCertificateStore_; property. And then use this instead of the options in the code above to figure out what is going on here.

[rbri]

https://www.pixelstech.net/article/1452337547-Different-types-of-keystore-in-Java----Windows-MY

Maybe you have to do something in the setup to add support for this kind of keystore and maybe the input stream is ignored by the load method if you use this type????

[alexvrv]

I have cloned the repo, made some changes, but how can i compile it to JAR so i can test it on my app?

[rbri]

you need maven (or an ide supporting maven)
then

mvn package -DskipTests

jars are generated in the target directory

[alexvrv]

mvn package -DskipTests gives me a Node error wtf xD
node:internal/validators:440
throw new ERR_INVALID_ARG_TYPE(name, 'Function', value);
^

TypeError [ERR_INVALID_ARG_TYPE]: The "cb" argument must be of type function. Received undefined
at makeCallback (node:fs:198:3)
at Object.mkdir (node:fs:1360:14)
at target.init (C:\Users\User 12\AppData\Roaming\nvm\v18.17.0\node_modules\mvn\target.js:25:10)
at Object. (C:\Users\User 12\AppData\Roaming\nvm\v18.17.0\node_modules\mvn\target.js:39:8)
at Module._compile (node:internal/modules/cjs/loader:1256:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
at Module.load (node:internal/modules/cjs/loader:1119:32)
at Module._load (node:internal/modules/cjs/loader:960:12)
at Module.require (node:internal/modules/cjs/loader:1143:19)
at require (node:internal/modules/cjs/helpers:110:18) {
code: 'ERR_INVALID_ARG_TYPE'
}

Node.js v18.17.0

[BrillBay]

mvn - not npm :-D

[BrillBay]

Have you maven installed?

[alexvrv]

Yeah i did "npm i -g mvn" and "npm i -g maven" and with " mvn package -DskipTests" i get that Node error xD

[rbri]

Oh :-D
maven is not an npm package, there are universes outside of javascript / npm.

You have to install maven - maybe you can start here https://maven.apache.org/download.cgi

• and then maven has to be in your path
• and then open a command line in the root of your project directory
• and then mvn package -DskipTests

[alexvrv]

Yeah long weekend xD. Got it to work with intellij xD

[rbri]

Have you found something?

[alexvrv]

I think I have solved (atleast for my use-case) with this in the getKeyStore function from WebClientOptions.java :

    Enumeration<String> en = keyStore.aliases();
    List<String> aliases = new ArrayList<>();

    while (en.hasMoreElements()) {
        String aliasKey = en.nextElement();
        Date certExpiryDate = ((X509Certificate) keyStore.getCertificate(aliasKey)).getNotAfter();
        Date today = new Date();
        long dateDiff = certExpiryDate.getTime() - today.getTime();
        long expiresIn = dateDiff / (24 * 60 * 60 * 1000);

        if (expiresIn < 1) {
            aliases.add(aliasKey);
        }
    }

    aliases.forEach(a -> {
        try {
            keyStore.deleteEntry(a);
        } catch (KeyStoreException ignored) {
        }
    });

[rbri]

@alexvrv what do you think about adding a filter parameter to setSSLClientCertificate() to provide a lambda for doing the code you did?
And maybe providing a default impl of that filter that did the removal of expired certs

[alexvrv]

Is there a point for a lambda parameter? Who would need to use expired certificates? Maybe a function to explicitly set a Certificate not a keystore would help more.

[rbri]

ok, sounds reasonable, let me think a bit ;-)

[rbri]

@alexvrv , decided to go another was. Now (starting with 3.6.0-SNAPSHOT) there is a new method WebClientOptions.setSSLClientCertificateKeyStore(KeyStore, char[]) allowing you to provide your own keystore.

For your case you can decorate/wrap your real keystore and use this wrapped keystore then for HtmlUnit. Your wrapper then can filter out the outdated certificates (like you already did).

Will be great if you can try this solution with the latest snapshot build.
Hopefully this will make your code also a bit simpler and more readable (and you do not need to use a patched version of HtmlUnit).

[alexvrv]

Only the build.gradle.kts will be a bit simpler, the code will be more complex because I need to filter the keystore before sending it to the webClient. Anyway, the 3.6.0-SNAPSHOT needs to be manually compiled right?

[rbri]

no you don't need to build it yourself - look at https://github.com/HtmlUnit/htmlunit at the end of the page

[alexvrv]

I have tried this: implementation("org.htmlunit:htmlunit:3.6.0-SNAPSHOT") but I get the following error: Caused by: org.gradle.internal.resolve.ModuleVersionNotFoundException: Could not find org.htmlunit:htmlunit:3.6.0-SNAPSHOT.

[rbri]

The snapshots are in a separate repository - please have a look at the whole gray area on the page....

[alexvrv]

Yep, my bad sorry xD

[rbri]

no prob

[alexvrv]

Just tested it, the code works as intended. Thank you! :D

[rbri]

but do not close this, i like to spend some minutes to update the docu

[rbri]

some docu added, will close this.