service/iot: Add MQTT over websockets client #820

dhubler · 2016-08-30T12:22:14Z

It only has Topic Publish.

This is most likely because SDK needs to use websocket protocol to get bidirectional communication.

Even if this is closed as "Will not implement", this should stand as notice API is incomplete compared with JS and Java SDKs feature set.

Also Note in source file
service/iotdataplane/service.go
We see comment:

// AWS IoT-Data enables secure, bi-directional communication between Internet-connected
// things (such as sensors, actuators, embedded devices, or smart appliances)

While AWS IoT API is bi-direction capable, the AWS IoT Golang SDK is not bi-directional

The text was updated successfully, but these errors were encountered:

xibz · 2016-08-30T17:00:10Z

Hello @dhubler, thank you for reaching out to us. Those are service specific SDKs, the JS and Java SDK, and currently are only supported by those two SDKs. This would be a great feature and I will mark this as a feature request.

dhubler · 2016-08-31T00:15:57Z

As a work around, If anyone is interested, I followed instructions in
http://docs.aws.amazon.com/iot/latest/developerguide/protocols.html#http
ported code to golang and using paho MQTT over websockets protocol from Golang SDK.

func AwsIotWsUrl(accessKey string, secretKey string, region string, endpoint string) string {
    host := fmt.Sprintf("%s.iot.%s.amazonaws.com", endpoint, region)

    // according to docs, time must be within 5min of actual time (or at least according to AWS servers)
    now := time.Now().UTC()

    dateLong := now.Format("20060102T150405Z")
    dateShort := dateLong[:8]
    serviceName := "iotdevicegateway"
    scope := fmt.Sprintf("%s/%s/%s/aws4_request", dateShort, region, serviceName)
    alg := "AWS4-HMAC-SHA256"
    q := [][2]string{
        {"X-Amz-Algorithm", alg},
        {"X-Amz-Credential", accessKey + "/" + scope},
        {"X-Amz-Date", dateLong},
        {"X-Amz-SignedHeaders", "host"},
    }
    query := awsQueryParams(q)

    signKey := awsSignKey(secretKey, dateShort, region, serviceName)
    stringToSign := awsSignString(accessKey, secretKey, query, host, dateLong, alg, scope)
    signature := fmt.Sprintf("%x", awsHmac(signKey, []byte(stringToSign)))

    return fmt.Sprintf("wss://%s/mqtt?%s&X-Amz-Signature=%s", host, query, signature)
}

func awsQueryParams(q [][2]string) string {
    var buff bytes.Buffer
    var i int
    for _, param := range q {
        if i != 0 {
            buff.WriteRune('&')
        }
        i++
        buff.WriteString(param[0])
        buff.WriteRune('=')
        buff.WriteString(url.QueryEscape(param[1]))
    }
    return buff.String()
}

func awsSignString(accessKey string, secretKey string, query string, host string, dateLongStr string, alg string, scopeStr string) string {
    emptyStringHash := "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
    req := strings.Join([]string{
        "GET",
        "/mqtt",
        query,
        "host:" + host,
        "", // separator
        "host",
        emptyStringHash,
    }, "\n")
    return strings.Join([]string{
        alg,
        dateLongStr,
        scopeStr,
        awsSha(req),
    }, "\n")
}

func awsHmac(key []byte, data []byte) []byte {
    h := hmac.New(sha256.New, key)
    h.Write(data)
    return h.Sum(nil)
}

func awsSignKey(secretKey string, dateShort string, region string, serviceName string) []byte {
    h := awsHmac([]byte("AWS4"+secretKey), []byte(dateShort))
    h = awsHmac(h, []byte(region))
    h = awsHmac(h, []byte(serviceName))
    h = awsHmac(h, []byte("aws4_request"))
    return h
}

func awsSha(in string) string {
    h := sha256.New()
    fmt.Fprintf(h, "%s", in)
    return fmt.Sprintf("%x", h.Sum(nil))
}

jeanbza · 2017-05-31T22:18:23Z

+1 this, we're really hurting not being able to pubsub from iot topics

aidansteele · 2017-11-09T21:24:21Z

FWIW I made a library to fill the gap until the AWS SDK provides support for this: https://github.com/glassechidna/awsiot

sess := session.Must(session.NewSessionWithOptions(sessOpts))
iot := awsiot.New(sess)
theUrl, _ := iot.WebsocketUrl("a1kxjqeyezkt7") // can be used with the eclipse paho mqtt library

swt2c · 2018-03-15T20:26:53Z

Just a note, as it took me a while to figure this out. It seems that @dhubler and @aidansteele solutions don't work (anymore?). IOT seems to want the X-Amz-Security-Token parameter, but it cannot be part of the canonical query parameters. It has to be added on after signing.

aidansteele · 2018-03-15T20:53:27Z

@swt2c I'll check that out. I had it working a couple of weeks ago, haven't tried since.

noliva · 2018-07-02T15:21:11Z

@swt2c @aidansteele Did you got this working? i've used @dhubler solution and appended the securityToken.. but i still get a 403 ..

swt2c · 2018-07-02T15:29:23Z

Yes, I did get it working at the time. You included the security token outside of the signature?

noliva · 2018-07-02T15:32:59Z

@swt2c yes, did you use @dhubler example of the library by @aidansteele ?

swt2c · 2018-07-02T15:36:07Z

Yes, but unfortunately, I'm not using it anymore and don't have access to the code I was using.

dhubler · 2018-07-02T16:25:12Z

One thing to know even if you get this working, websocket connection will reset every 24 hours. Ultimately using straight MQTT and TLS was most reliable.

…

On Mon, Jul 2, 2018, 8:36 AM Scott Talbert ***@***.***> wrote: Yes, but unfortunately, I'm not using it anymore and don't have access to the code I was using. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#820 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAgfip3PIZCg3P6uWC6cA8Jl1WVDpk7ks5uCj3ugaJpZM4Jwdvx> .

ytwig · 2019-05-03T23:31:17Z

The solution provided by @aidansteele is still working great.
Tested just now

Pitasi · 2020-07-17T14:32:03Z

I just had a hard time to get this working and want to share what I ended up doing.

aidansteele's solution worked on my laptop but not inside a lambda
dhubler's solution didn't work on my laptop but I got it working on my lambda (ensure IAM roles allows iot actions) editing it a little:

// usage:
// addr, err := AwsIotWsUrl(sess, "xxxxx-ats.iot.eu-west-1.amazonaws.com ")

func AwsIotWsUrl(p client.ConfigProvider, endpoint string) (string, error) {
	serviceName := "iotdevicegateway"
	config := p.ClientConfig(serviceName)
	region := *config.Config.Region
	creds, err := config.Config.Credentials.Get()
	if err != nil {
		return "", err
	}
	accessKey := creds.AccessKeyID
	secretKey := creds.SecretAccessKey
	sessionToken := creds.SessionToken

	// according to docs, time must be within 5min of actual time (or at least according to AWS servers)
	now := time.Now().UTC()

	dateLong := now.Format("20060102T150405Z")
	dateShort := dateLong[:8]
	scope := fmt.Sprintf("%s/%s/%s/aws4_request", dateShort, region, serviceName)
	alg := "AWS4-HMAC-SHA256"
	q := [][2]string{
		{"X-Amz-Algorithm", alg},
		{"X-Amz-Credential", accessKey + "/" + scope},
		{"X-Amz-Date", dateLong},
		{"X-Amz-SignedHeaders", "host"},
	}
	query := awsQueryParams(q)

	signKey := awsSignKey(secretKey, dateShort, region, serviceName)
	stringToSign := awsSignString(accessKey, secretKey, query, endpoint, dateLong, alg, scope)
	signature := fmt.Sprintf("%x", awsHmac(signKey, []byte(stringToSign)))

	return fmt.Sprintf("wss://%s/mqtt?%s&X-Amz-Signature=%s&X-Amz-Security-Token=%s", endpoint, query, signature, url.QueryEscape(sessionToken)), nil
}

func awsQueryParams(q [][2]string) string {
	var buff bytes.Buffer
	var i int
	for _, param := range q {
		if i != 0 {
			buff.WriteRune('&')
		}
		i++
		buff.WriteString(param[0])
		buff.WriteRune('=')
		buff.WriteString(url.QueryEscape(param[1]))
	}
	return buff.String()
}

func awsSignString(accessKey string, secretKey string, query string, host string, dateLongStr string, alg string, scopeStr string) string {
	emptyStringHash := "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
	req := strings.Join([]string{
		"GET",
		"/mqtt",
		query,
		"host:" + host,
		"", // separator
		"host",
		emptyStringHash,
	}, "\n")
	return strings.Join([]string{
		alg,
		dateLongStr,
		scopeStr,
		awsSha(req),
	}, "\n")
}

func awsHmac(key []byte, data []byte) []byte {
	h := hmac.New(sha256.New, key)
	h.Write(data)
	return h.Sum(nil)
}

func awsSignKey(secretKey string, dateShort string, region string, serviceName string) []byte {
	h := awsHmac([]byte("AWS4"+secretKey), []byte(dateShort))
	h = awsHmac(h, []byte(region))
	h = awsHmac(h, []byte(serviceName))
	h = awsHmac(h, []byte("aws4_request"))
	return h
}

func awsSha(in string) string {
	h := sha256.New()
	fmt.Fprintf(h, "%s", in)
	return fmt.Sprintf("%x", h.Sum(nil))
}

ffoysal · 2021-03-01T21:20:24Z

this kind of subscription does not work over webscoket. using goland with paho mqtt library
Steps I have done:

create aws iam role
create iot role-alias pointing to the IAM role

    {
      "Effect": "Allow",
      "Action": [
        "iot:Subscribe"
      ],
      "Resource": [
        "arn:aws:iot:region:********:topicfilter/$aws/things/${iot:Connection.Thing.ThingName}/jobs/notify-next",
        "arn:aws:iot:region:*****:topicfilter/$aws/things/${iot:Connection.Thing.ThingName}/jobs/start-next/accepted",
        "arn:aws:iot:region:******:topicfilter/$aws/things/${iot:Connection.Thing.ThingName}/jobs/start-next/rejected",
        "arn:aws:iot:region:******:topicfilter/$aws/things/${iot:Connection.Thing.ThingName}/jobs/*/update/accepted",
        "arn:aws:iot:region:*******:topicfilter/$aws/things/${iot:Connection.Thing.ThingName}/jobs/*/update/rejected"
      ]
    }

create a policy with role-alias and having

    {
      "Effect": "Allow",
      "Action": "iot:AssumeRoleWithCertificate",
      "Resource": "arn:aws:iot:REGION:***********:rolealias/my-thing-role-alias"
    }

I can subscribe to the jop topic but dont get any events from it.

If IAM role has something like this

        {
            "Effect": "Allow",
            "Action": [
                "iot:Subscribe"
            ],
            "Resource": [
                "arn:aws:iot:region:*****:topicfilter/$aws/things/${iot:ClientId}/jobs/notify-next",
                "arn:aws:iot:region:*****:topic/$aws/things/${iot:ClientId}/jobs/notify",
                "arn:aws:iot:region:*******:topic/$aws/things/${iot:ClientId}/jobs/get/accepted",
                "arn:aws:iot:region:******:topic/$aws/things/${iot:ClientId}/jobs/*/get/accepted",
                "arn:aws:iot:region:*******:topic/$aws/things/${iot:ClientId}/jobs/get/rejected",
                "arn:aws:iot:region:********:topic/$aws/things/${iot:ClientId}/jobs/*/get/rejected",
                "arn:aws:iot:region:********:topicfilter/$aws/things/${iot:ClientId}/jobs/start-next/accepted",
                "arn:aws:iot:region:********:topicfilter/$aws/things/${iot:ClientId}/jobs/start-next/rejected",
                "arn:aws:iot:region:*********:topicfilter/$aws/things/${iot:ClientId}/jobs/*/update/accepted",
                "arn:aws:iot:region:*********:topicfilter/$aws/things/${iot:ClientId}/jobs/*/update/rejected"
            ]
        }

Then subscribed and get evens from those topics.

vudh1 · 2022-06-03T19:07:32Z

Hi, our team has discussed and we decide that this feature is not implemented as this is a more cross SDK feature.

github-actions · 2022-06-03T19:07:59Z

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

yeraslan-96 · 2022-07-15T13:00:57Z

Anyone got this working thought MQTT over Websockets, still struggling with it...

yeraslan-96 · 2022-07-15T13:31:03Z

As a work around, If anyone is interested, I followed instructions in http://docs.aws.amazon.com/iot/latest/developerguide/protocols.html#http ported code to golang and using paho MQTT over websockets protocol from Golang SDK.

func AwsIotWsUrl(accessKey string, secretKey string, region string, endpoint string) string {
    host := fmt.Sprintf("%s.iot.%s.amazonaws.com", endpoint, region)

    // according to docs, time must be within 5min of actual time (or at least according to AWS servers)
    now := time.Now().UTC()

    dateLong := now.Format("20060102T150405Z")
    dateShort := dateLong[:8]
    serviceName := "iotdevicegateway"
    scope := fmt.Sprintf("%s/%s/%s/aws4_request", dateShort, region, serviceName)
    alg := "AWS4-HMAC-SHA256"
    q := [][2]string{
        {"X-Amz-Algorithm", alg},
        {"X-Amz-Credential", accessKey + "/" + scope},
        {"X-Amz-Date", dateLong},
        {"X-Amz-SignedHeaders", "host"},
    }
    query := awsQueryParams(q)

    signKey := awsSignKey(secretKey, dateShort, region, serviceName)
    stringToSign := awsSignString(accessKey, secretKey, query, host, dateLong, alg, scope)
    signature := fmt.Sprintf("%x", awsHmac(signKey, []byte(stringToSign)))

    return fmt.Sprintf("wss://%s/mqtt?%s&X-Amz-Signature=%s", host, query, signature)
}

func awsQueryParams(q [][2]string) string {
    var buff bytes.Buffer
    var i int
    for _, param := range q {
        if i != 0 {
            buff.WriteRune('&')
        }
        i++
        buff.WriteString(param[0])
        buff.WriteRune('=')
        buff.WriteString(url.QueryEscape(param[1]))
    }
    return buff.String()
}

func awsSignString(accessKey string, secretKey string, query string, host string, dateLongStr string, alg string, scopeStr string) string {
    emptyStringHash := "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
    req := strings.Join([]string{
        "GET",
        "/mqtt",
        query,
        "host:" + host,
        "", // separator
        "host",
        emptyStringHash,
    }, "\n")
    return strings.Join([]string{
        alg,
        dateLongStr,
        scopeStr,
        awsSha(req),
    }, "\n")
}

func awsHmac(key []byte, data []byte) []byte {
    h := hmac.New(sha256.New, key)
    h.Write(data)
    return h.Sum(nil)
}

func awsSignKey(secretKey string, dateShort string, region string, serviceName string) []byte {
    h := awsHmac([]byte("AWS4"+secretKey), []byte(dateShort))
    h = awsHmac(h, []byte(region))
    h = awsHmac(h, []byte(serviceName))
    h = awsHmac(h, []byte("aws4_request"))
    return h
}

func awsSha(in string) string {
    h := sha256.New()
    fmt.Fprintf(h, "%s", in)
    return fmt.Sprintf("%x", h.Sum(nil))
}

With which parameters I can call the AwsIotWsUrl function? I can not call that function

xibz self-assigned this Aug 30, 2016

xibz added the feature-request A feature should be added or improved. label Aug 30, 2016

jasdel changed the title ~~IoT API doesn't support Topic Subscribe~~ service/iot: Client doesn't support Topic Subscribe Apr 12, 2017

jasdel changed the title ~~service/iot: Client doesn't support Topic Subscribe~~ service/iot: Add MQTT over websockets client Apr 12, 2017

jasdel mentioned this issue May 30, 2017

Subscribe to IoT topic #1304

Closed

skmcgrail unassigned xibz Mar 19, 2021

vudh1 closed this as completed Jun 3, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

service/iot: Add MQTT over websockets client #820

service/iot: Add MQTT over websockets client #820

dhubler commented Aug 30, 2016

xibz commented Aug 30, 2016

dhubler commented Aug 31, 2016 •

edited

Loading

jeanbza commented May 31, 2017 •

edited

Loading

aidansteele commented Nov 9, 2017 •

edited

Loading

swt2c commented Mar 15, 2018

aidansteele commented Mar 15, 2018

noliva commented Jul 2, 2018

swt2c commented Jul 2, 2018

noliva commented Jul 2, 2018

swt2c commented Jul 2, 2018

dhubler commented Jul 2, 2018 via email

ytwig commented May 3, 2019

Pitasi commented Jul 17, 2020

ffoysal commented Mar 1, 2021

vudh1 commented Jun 3, 2022

github-actions bot commented Jun 3, 2022

yeraslan-96 commented Jul 15, 2022

yeraslan-96 commented Jul 15, 2022

service/iot: Add MQTT over websockets client #820

service/iot: Add MQTT over websockets client #820

Comments

dhubler commented Aug 30, 2016

xibz commented Aug 30, 2016

dhubler commented Aug 31, 2016 • edited Loading

jeanbza commented May 31, 2017 • edited Loading

aidansteele commented Nov 9, 2017 • edited Loading

swt2c commented Mar 15, 2018

aidansteele commented Mar 15, 2018

noliva commented Jul 2, 2018

swt2c commented Jul 2, 2018

noliva commented Jul 2, 2018

swt2c commented Jul 2, 2018

dhubler commented Jul 2, 2018 via email

ytwig commented May 3, 2019

Pitasi commented Jul 17, 2020

ffoysal commented Mar 1, 2021

vudh1 commented Jun 3, 2022

github-actions bot commented Jun 3, 2022

⚠️COMMENT VISIBILITY WARNING⚠️

yeraslan-96 commented Jul 15, 2022

yeraslan-96 commented Jul 15, 2022

dhubler commented Aug 31, 2016 •

edited

Loading

jeanbza commented May 31, 2017 •

edited

Loading

aidansteele commented Nov 9, 2017 •

edited

Loading