Regression: 1.1.1 fails to connect to AWS IOT over TLS; works with 1.1.0 #345

Typhlosaurus · 2017-03-24T14:22:52Z

[Yes ] Bug exists Release Version 1.1.1 ( Master Branch)
[ Yes] Bug exists in Snapshot Version
Version 1.1.2-SNAPSHOT (Develop Branch) (currently identical to 1.1.1)

MQTT newby here...

Using the amazon sdk to communicate with the AWS IoT server using MQTT over TLS in eu-west-1 has started to fails since 1.1.1 of paho.mqhtt.java was released. When attempting to connect a 403 is reported (actually reported as a "WebSocket Response header: Incorrect connection header" in org.eclipse.paho.client.mqttv3.internal.websocket.WebSocketHandshake due to the missing connection header rather than the status code).

Switching back to mqttv3 1.1.0 restores the ability to connect.

AWS SDK version: com.amazonaws:aws-iot-device-sdk-java:1.1.0

I'm afraid someone else is going to have to help with any tracing details - if they're needed could someone tell me how to enable them.

If this is a known/suspected AWS IoT issue I'd be grateful for some information to pass on to them - I've assumed it is not due to 1.1.0 still working.

WARNING: Connect request failure
MqttException (0) - java.io.IOException: WebSocket Response header: Incorrect connection header
	at org.eclipse.paho.client.mqttv3.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:38)
	at org.eclipse.paho.client.mqttv3.internal.ClientComms$ConnectBG.run(ClientComms.java:690)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: WebSocket Response header: Incorrect connection header
	at org.eclipse.paho.client.mqttv3.internal.websocket.WebSocketHandshake.receiveHandshakeResponse(WebSocketHandshake.java:144)
	at org.eclipse.paho.client.mqttv3.internal.websocket.WebSocketHandshake.execute(WebSocketHandshake.java:74)
	at org.eclipse.paho.client.mqttv3.internal.websocket.WebSocketSecureNetworkModule.start(WebSocketSecureNetworkModule.java:77)
	at org.eclipse.paho.client.mqttv3.internal.ClientComms$ConnectBG.run(ClientComms.java:676)
	... 1 more

The text was updated successfully, but these errors were encountered:

jpwsutton · 2017-03-24T14:42:27Z

Could this be related to: aws/aws-iot-device-sdk-java#5

Typhlosaurus · 2017-03-24T14:48:41Z

@jpwsutton I doubt it as the exception is different (my apologies: I added the exception details after you wrote your comment) and I do not need to disconnect/reconnect the network to generate the error - it happens whenever I try to connect using 1.1.1 (and has been reproduced in 3 environments - developer running on MacOs, developer on Windows and build server running linux).

sandyperez · 2017-03-29T07:43:55Z

Same problem here, downgrading to version 1.1.0 solves it. Best regards.

beersheba · 2018-06-10T11:21:41Z

AWS IoT is still incompatible with Paho > 1.1.0, throwing java.io.IOException: WebSocket Response header: Incorrect upgrade. There is 1.2.0 allready. Any advice how it can be resolved?

jpwsutton · 2018-06-15T12:19:16Z

Would it be possible with someone who has access to the AWS IoT service to copy and paste me a copy of a working example of their webockets header exchange please?

It looks like the upgrade part of the exchange is incorrect, but without seeing it I can't fix anything...

jpwsutton · 2018-08-28T19:54:17Z

@Typhlosaurus, @beersheba, @sandyperez, If anyone could send me a copy of the headers returned from AWS that would be great, otherwise I will try and recreate this when I get the opportunity.

desokroshan · 2019-01-09T13:54:43Z

@jpwsutton Following is snapshot of header returned by the AWS:

Please let me know if you need anything else.

desokroshan · 2019-02-20T14:03:45Z

Any updates on this?

EddieRingle · 2019-03-26T19:42:04Z

@jpwsutton @desokroshan I'm hitting this issue as well. Below are the request headers from both 1.1.0 as well as 1.2.1, where the latter gets the 403 response from AWS. Confidential details have been snipped, of course.

1.1.0 request headers:

GET /mqtt?X-Amz-Algorithm=<snip...> HTTP/1.1
Host: <snip...>.iot.us-east-1.amazonaws.com:443
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: bXF0dC0xNTUzNjI4MDg3
Sec-WebSocket-Protocol: mqttv3.1
Sec-WebSocket-Version: 13

1.2.1 request headers:

GET /mqtt?X-Amz-Algorithm=<snip...> HTTP/1.1
Host: <snip...>.iot.us-east-1.amazonaws.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: Y2Y2YzFhOGMtZTM4OS00ZA==
Sec-WebSocket-Protocol: mqtt
Sec-WebSocket-Version: 13

So, the differences from what I can see:

Host header has a port specified in 1.1.0, but not in 1.2.1
Sec-WebSocket-Protocol is using a non-standard 'mqttv3.1' in 1.1.0, and the standard 'mqtt' in 1.2.1 (See 3ddb38b)

EddieRingle · 2019-03-26T20:26:47Z

After modifying the latest develop branch to include the port number again, I made it past the Upgrade error but then entered a loop of connects/disconnects. (I'm using the AWS Java SDK: https://github.com/aws/aws-iot-device-sdk-java)

aliozgun · 2019-08-23T08:10:22Z

Is there any update on this issue? Using 1.1.0 at this point is not helping anymore. Iot-device-stk is brought down to 1.0.1

Thanks
Ali

desokroshan · 2019-09-18T22:42:12Z

@jpwsutton @icraggs Is there any update on this. AWS SDK for Android depends on paho and is unable to upgrade because of this error. This prevents us from using the bug fixes and features in the latest paho version. Can we have some idea of timeline for this?

icraggs · 2019-09-19T10:34:27Z

The latest release of the Paho Java client is 1.2.1. I presume this is still not working.

Does the same error occur if the connection attempt is made outside of Android, in a 'plain' Java environment? If so, that will make it easier to find and fix.

Can anyone supply a trace or other details of the protocol exchange? I'd hope not to have to create an AWS account to try this out. I could put some more debug info into the trace if necessary.

aliozgun · 2019-09-19T14:15:36Z

When I replayed the same update HTTP request from Postman I got signature error. It looks like the signature calculation is incorrect.

icraggs · 2019-09-19T15:14:16Z

Ah sorry, I hadn't seen the update request above posted.

aliozgun · 2019-09-19T15:34:30Z

Sorry I meant the HTTP upgrade request for websocket. That request fails with signature error and does not have the necessary response headers.

desokroshan · 2019-09-19T16:51:50Z

@icraggs I have tried with the 1.2.1 and am still getting the same error. Only version it works with is 1.1.0. Please let me know if you need anything else. This is blocking quite a few of our customers from using AWS IoT, so we would really appreciate a speedy resolution.

icraggs · 2019-09-19T20:39:23Z

Thanks. I think the post from @EddieRingle could contain the key in the handshake:

Sec-WebSocket-Protocol: mqttv3.1

We should hopefully be able to try it out. If anyone else can in the meantime, we could get a fix out quick.

desokroshan · 2019-09-20T02:36:24Z

The problem is not with Sec-WebSocket-Protocol being mqtt but with the missing port number in host header for websocket request. I have verified that adding the port back fixes the issue. I have issued a PR(referenced above) with the required fix.

icraggs · 2019-09-20T12:39:35Z

Ah yes, thanks. I've merged your fix, so we should be good to go in the next release.

desokroshan · 2019-09-20T17:33:08Z

Thanks! When can we expect next release to go out?

icraggs · 2019-09-20T19:28:38Z

I'll get one out next week - now there are two significant fixes.

By the way, once you were successfully connected, did things work as expected?

desokroshan · 2019-09-20T22:48:46Z

I'll get one out next week - now there are two significant fixes.

Thanks that would be great!

By the way, once you were successfully connected, did things work as expected?

Yes. I was able subscribe and publish messages successfully.

icraggs · 2019-09-23T17:32:30Z

1.2.2 release created.

jpwsutton added the triage label Mar 24, 2017

Typhlosaurus mentioned this issue Mar 28, 2017

Connect fails org.eclipse.paho:org.eclipse.paho.client.mqttv3:1.1.1 aws/aws-iot-device-sdk-java#23

Closed

timtay-microsoft mentioned this issue Jun 25, 2018

[Device Client - MQTT] Issue while connecting to IOT HUB Azure/azure-iot-sdk-java#264

Closed

This was referenced Jan 10, 2019

Changes from PR#554 on gradle #607

Closed

IOException: WebSocket Response header: Incorrect upgrade aws-amplify/aws-sdk-android#479

Closed

timmattison mentioned this issue Jun 5, 2019

Bump org.eclipse.paho.client.mqttv3 from 1.1.0 to 1.2.1 in /mqtt-over-websockets/java aws-iot-builder-tools/iot-reference-architectures#13

Closed

desokroshan mentioned this issue Aug 9, 2019

How to include custom authorization headers while connecting? aws-amplify/aws-sdk-android#429

Closed

This was referenced Aug 23, 2019

Client not connecting to Websocket when using Paho 1.2.0 library awslabs/aws-iot-fuse#4

Open

Upgraded paho library to 1.2.0 aws/aws-iot-device-sdk-java#65

Closed

icraggs self-assigned this Sep 19, 2019

icraggs added this to the 1.2.2 milestone Sep 19, 2019

icraggs added bug and removed triage labels Sep 19, 2019

This was referenced Sep 19, 2019

Keep port in the host websocket request header for port 443 #700

Closed

Keep port number in the host websocket request header for port 443 #701

Merged

icraggs added the fix submitted label Sep 20, 2019

icraggs closed this as completed Sep 23, 2019

KF5JWC mentioned this issue Aug 7, 2022

[Regression] Broken websockets support Vetru-Apps/mqtt-dashboard-documentation#17

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Regression: 1.1.1 fails to connect to AWS IOT over TLS; works with 1.1.0 #345

Regression: 1.1.1 fails to connect to AWS IOT over TLS; works with 1.1.0 #345

Typhlosaurus commented Mar 24, 2017 •

edited

jpwsutton commented Mar 24, 2017

Typhlosaurus commented Mar 24, 2017

sandyperez commented Mar 29, 2017

beersheba commented Jun 10, 2018 •

edited

jpwsutton commented Jun 15, 2018

jpwsutton commented Aug 28, 2018

desokroshan commented Jan 9, 2019

desokroshan commented Feb 20, 2019

EddieRingle commented Mar 26, 2019

EddieRingle commented Mar 26, 2019 •

edited

aliozgun commented Aug 23, 2019

desokroshan commented Sep 18, 2019 •

edited

icraggs commented Sep 19, 2019

aliozgun commented Sep 19, 2019

icraggs commented Sep 19, 2019

aliozgun commented Sep 19, 2019

desokroshan commented Sep 19, 2019 •

edited

icraggs commented Sep 19, 2019

desokroshan commented Sep 20, 2019

icraggs commented Sep 20, 2019

desokroshan commented Sep 20, 2019

icraggs commented Sep 20, 2019

desokroshan commented Sep 20, 2019

icraggs commented Sep 23, 2019

Regression: 1.1.1 fails to connect to AWS IOT over TLS; works with 1.1.0 #345

Regression: 1.1.1 fails to connect to AWS IOT over TLS; works with 1.1.0 #345

Comments

Typhlosaurus commented Mar 24, 2017 • edited

jpwsutton commented Mar 24, 2017

Typhlosaurus commented Mar 24, 2017

sandyperez commented Mar 29, 2017

beersheba commented Jun 10, 2018 • edited

jpwsutton commented Jun 15, 2018

jpwsutton commented Aug 28, 2018

desokroshan commented Jan 9, 2019

desokroshan commented Feb 20, 2019

EddieRingle commented Mar 26, 2019

EddieRingle commented Mar 26, 2019 • edited

aliozgun commented Aug 23, 2019

desokroshan commented Sep 18, 2019 • edited

icraggs commented Sep 19, 2019

aliozgun commented Sep 19, 2019

icraggs commented Sep 19, 2019

aliozgun commented Sep 19, 2019

desokroshan commented Sep 19, 2019 • edited

icraggs commented Sep 19, 2019

desokroshan commented Sep 20, 2019

icraggs commented Sep 20, 2019

desokroshan commented Sep 20, 2019

icraggs commented Sep 20, 2019

desokroshan commented Sep 20, 2019

icraggs commented Sep 23, 2019

Typhlosaurus commented Mar 24, 2017 •

edited

beersheba commented Jun 10, 2018 •

edited

EddieRingle commented Mar 26, 2019 •

edited

desokroshan commented Sep 18, 2019 •

edited

desokroshan commented Sep 19, 2019 •

edited