Not able to establish MQTT connection

[ChetanPundhir]

• I've searched for previous similar issues and didn't find any solution

Platform/OS/Hardware/Device
Windows 10

Describe the question
I am trying to use JITP feature to onboard my client certificate.
Its working with aws iot device sdk for java old version(https://github.com/aws/aws-iot-device-sdk-java) using PublishSubscribeSample.java but with this new version the same I try to do using PubSub.java sample, its not working.

Issue 1 is that as mentioned for JITP (https://aws.amazon.com/blogs/iot/setting-up-just-in-time-provisioning-with-aws-iot-core), for first connection attempt, a concatenated(device + ca) certificate should be used. If I try to do that I get the error: TLS (SSL) negotiation failed
Issue 2 is that If I do the client JITP using old sdk for java and use this new sdk sample code to just connect with client certificate, I still get the error: TLS (SSL) negotiation failed.

If I use RSA certificates, issue one still appears but issue 2 does not and with rsa certificates connection is established.
If I use ECC certificates, I get both issue 1 and issue 2.

However, I get no issue if I do same steps with old sdk for java. With old sdk, it work fine for both rsa and ecc certificates.

What additional need to be done for ECC certificates to make the connection work and why concatenated certificate not work as explained here: https://aws.amazon.com/blogs/iot/setting-up-just-in-time-provisioning-with-aws-iot-core/.

I am using ats endpoint.

Please help.

[uberbinge]

We are also facing the same issue, we get TLS (SSL) negotiation failed if we don't provide an explicit root certificate. But that wasn't the case with the old SDK.

@ChetanPundhir you can try to provide the root cert as a String for this builder parameter: withCertificateAuthority

Certificate link: https://www.amazontrust.com/repository/AmazonRootCA1.pem

Can someone from AWS please pitch in and let us know why we need to provide a root certificate now? It will be extra effort to keep it up-to-date.

[ChetanPundhir]

It has been sometime but received no response form technical team. Request to share some input.

[TingDaoK]

You will need to manually provide the certificate for v2. TLS (SSL) negotiation failed indicates your certs are invalid or not set. So, in this case, you just need to manually set the certs.

[uberbinge]

@TingDaoK

you just need to manually set the certs

Yeah this is what exactly we don't want to do. In previous SDK it wasn't required and from the JavaDoc it overrides the default Trust Store.

/**
     * Overrides the default system trust store.
     * 
     * @param caRoot - Buffer containing all trust CAs, in PEM format
     */
    public AwsIotMqttConnectionBuilder withCertificateAuthority(String caRoot) {
        this.tlsOptions.overrideDefaultTrustStore(caRoot);
        resetLazilyCreatedResources = true;
        return this;
    }

Why can't the new SDK use the default TrustStore and if it doesn't work then use the default Certificate internally so the consumers of the SDK don't have to manage & communicate to their customers why we need to override default System trust store.

[JonathanHenson]

you shouldn't need to set a CA if it's already in the system CA store. The CA isn't for your client certificate anyways; it's for verifying the server's identify. There's no x.509 validation on the client of your MTLS certs.

If you're getting:

We are also facing the same issue, we get TLS (SSL) negotiation failed if we don't provide an explicit root certificate. But that wasn't the case with the old SDK.

This is a different scenario. If that's happening the CA isn't in your OS's CA store. You can add it, update the OS's store (those should be in the store by now), or specify it as described. As for why it's different now? The v2 SDKs go through the operating system's PKI system, not the java keystore. So for windows, this would go through CertManager, on Apple OS: keychain, and on Unix systems, the typical locations for that distribution (/etc/ssl/certs and friends).

[uberbinge]

As you can see in the screenshot attached at the end, the Amazon Root CA 1 cert is already part of trusted certificates for the OS.

But the SDK still throws ExecutionException: software.amazon.awssdk.crt.mqtt.MqttException: TLS (SSL) negotiation failed when we don't specify Amazon Root CA 1's content as String to following method:

public AwsIotMqttConnectionBuilder withCertificateAuthority(String caRoot) {

Device:Pixel 3a
OS: Android 11

[uberbinge]

We are also using the Java SDK version software.amazon.awssdk.iotdevicesdk:aws-iot-device-sdk:1.2.7 on MacOS to run some of our tests. That SDK works without passing any CA into method:

public AwsIotMqttConnectionBuilder withCertificateAuthority(String caRoot) {

Same would be the expected behaviour on Android as well since Amazon Root CA 1 is already trusted by the OS.

[JonathanHenson]

Well, if it's in the trust store, and the SDK isn't picking it up, that's definitely a bug we need to track down. Thanks for clarifying!

[ChetanPundhir]

@JonathanHenson

I am passing the rootCA certificate as well but it does not work with v2 for ECC certificates. With RSA, it works fine.

Are ECC certificates supported with v2?

I tested the ssl hadshake using below command:

openssl s_client -connect *-ats.iot.eu-west-1.amazonaws.com:8443 -CAfile rootCA.pem -cert deviceCert.pem -key deviceCert.key

which works well, means the certificates are correct.

Please suggest what could be the issue.

[JonathanHenson]

Is this on windows? I think we have an open issue for aws-c-io where ecc certs werent importing correctly.

[ChetanPundhir]

@JonathanHenson yes, I am testing on Windows. Do you mean it does not work on windows. Does it work with linux or any issues with that as well.

[JonathanHenson]

I mean, this code on windows:
https://github.com/awslabs/aws-c-io/blob/main/source/windows/windows_pki_utils.c#L321

doesn't explicitly handle ECC key pairs, where as the apple and unix integrations work transparently. When I read the dotnet core source code I see a separate branch and import for ECC.

[ilcannibale]

I have the same problem with implementation of JITR. I'm on Mac OS.

I implemented this process with aws-iot-device-sdk-js and it works.
With the same certificates (device cetificate, private key and Ca) it doesn't works with aws-iot-device-sdk-java-v2.
I tried both .withCertificateAuthority and .withCertificateAuthorityFromPath and the result doesn't changed.

The error change if i change the endpoint.
If i try with "-ats" endpoint the error message is "The connection was closed unexpectedly".
If i try with "not -ats" endpoint the error message is "TLS (SSL) negotiation failed".

I read also this https://aws.amazon.com/it/blogs/iot/aws-iot-core-ats-endpoints/ but my java version is 8u202 and it haven't jdk.security.caDistrustPolicies property set in java security options.

I validated the certificates with openssl s_client as suggested here https://docs.aws.amazon.com/iot/latest/developerguide/diagnosing-connectivity-issues.html and the result was good.

However, the error occurs only if the certificate in not active yet (first time to connect to mqtt client, the step 1 of JITR).
If i register thing implementing JITR with my javascript client and next i try to connect to mqtt client with java client, then the mqtt client connection works.

(sorry for my english)

EDIT: i enabled the crt log

[ERROR] [2020-11-30T15:56:16Z] [0000700010535000] [socket] - id=0x7fa8171497d0 fd=78: connect failed with error code 65.
[INFO] [2020-11-30T15:56:16Z] [0000700010535000] [dns] - id=0x7fa815570750: recording failure for record 2a01:578:13::12c0:a328 for -ats.iot.eu-central-1.amazonaws.com, moving to bad list
[DEBUG] [2020-11-30T15:56:16Z] [0000700010535000] [dns] - static: purging address 2a01:578:13::12c0:a328 for host -ats.iot.eu-central-1.amazonaws.com from the cache due to cache eviction or shutdown
[DEBUG] [2020-11-30T15:56:16Z] [0000700010535000] [socket] - id=0x7fa8171497d0 fd=78: is still open, closing...
[DEBUG] [2020-11-30T15:56:16Z] [0000700010535000] [socket] - id=0x7fa8171497d0 fd=78: closing
[ERROR] [2020-11-30T15:56:16Z] [0000700010535000] [channel-bootstrap] - id=0x7fa815570840: failed to create socket with error 1049

After some research on these errors, i discovered there is a similar bug in aws/aws-iot-device-sdk-js-v2#30

[jmklix]

There was a fix to correctly handle concatenated certs on windows here. This should no longer be an issue, but if you are still running into any problems please let us know.
You can follow this guide for the latest instructions on how to generate certs for JITP

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.