Enhancement Request: Add support for AWS IoT MQTT server

[msreynolds]

As a user of Indigo and this plugin, I want to use Amazon's AWS IoT Core as my MQTT server, instead of running my own.

Amazon's AWS IoT web console is here:
https://us-east-1.console.aws.amazon.com/iot/home

There is a specific python MQTT client class that Amazon provides (AWSIoTMQTTClient) for connecting to their backend. It is based on the Paho MQTT client. There are examples of using this client within the repo:
https://github.com/aws/aws-iot-device-sdk-python

This AWS IoT client requires

1. a Host (or Endpoint in AWS terms)
2. a Port
3. a Root CA certificate
4. a Private key certificate
5. a Public key certificate

This is what connecting with the AWS IoT client looks like using one of the examples from their repo. In this example, basicPubSub.py uses a default port if one is not passed in:

python basicPubSub.py -e xxxxxxxxxxxxxx-ats.iot.us-east-1.amazonaws.com -r root-CA.crt -c device.cert.pem -k device.private.key

Where xxxxxxxxxxxxxx-ats.iot.us-east-1.amazonaws.com is your personal AWS IoT Endpoint, found under the Settings of the AWS IoT web console. Settings can be accessed from the lower left navigation panel. The certs are generated from the AWS IoT web console when you Onboard a new device/thing.

From the AWS IoT web console:

1. Onboard a new Thing/Device that will represent Indigo. Pick your OS and Language - Linux/MacOS and Python in this case.
   The important part of this step is to generate the X.509 certificates that will be associated with your "device" aka Indigo. These certificates are required in order to get the AWS IoT MQTT client to talk to your personalized AWS IoT MQTT Endpoint.

2. Create a new Policy and attach it to your Certificate generated in step 1. The policy will look something like:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "iot:Publish",
        "iot:Receive"
      ],
      "Resource": [
        "arn:aws:iot:us-east-1:<your-account-id>:topic/x",
        "arn:aws:iot:us-east-1:<your-account-id>:topic/y",
        "arn:aws:iot:us-east-1:<your-account-id>:topic/z"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "iot:Subscribe"
      ],
      "Resource": [
        "arn:aws:iot:us-east-1:<your-account-id>:topicfilter/x",
        "arn:aws:iot:us-east-1:<your-account-id>:topicfilter/y",
        "arn:aws:iot:us-east-1:<your-account-id>:topicfilter/z"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "iot:Connect"
      ],
      "Resource": [
        "arn:aws:iot:us-east-1:<your-account-id>:client/<your-mqtt-client-id>"
      ]
    }
  ]
}

To find <your-account-id> try using their Policy wizard to 'Add a Statement' to your new Policy. It should automatically populate the ARN with your account Id when you pick an Action.

Think of the X.509 certificates as Authentication and the attached Policy as Authorization.

A single AWS IoT Device or Thing can publish to any topic on the global Endpoint, assuming it is allowed by the attached Policy. This should allow a single AWS IoT Thing representing Indigo to publish and subscribe to a sufficient number of unique topics to express all devices, actions, and variables, as needed.

It can be a little confusing to navigate the AWS IoT web console, but keep poking around and you'll find what you need.

[FlyingDiver]

Have you actually set up the cert and key needed for this, and attempted to connect using the sample scripts in the SDK? I'm having some problems there, and also in the plugin code that I've added.

Executing this works. I see messages from the test endpoint.

 python ./basicPubSub.py -e avv4sfoj382qz-ats.iot.us-east-1.amazonaws.com -r AmazonRootCA1.pem -c Indigo-Plugin.cert.pem -k Indigo-Plugin.private.key -p 8883

But as soon as I add a topic to the command:

 python ./basicPubSubAsync.py -e avv4sfoj382qz-ats.iot.us-east-1.amazonaws.com -r AmazonRootCA1.pem -c Indigo-Plugin.cert.pem -k Indigo-Plugin.private.key -p 8883 -t "foo"

It fails:

2020-01-25 21:25:57,127 - AWSIoTPythonSDK.core.protocol.mqtt_core - INFO - Performing async subscribe...
2020-01-25 21:25:57,128 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - Adding a new subscription record: foo qos: 1
2020-01-25 21:25:57,128 - AWSIoTPythonSDK.core.protocol.internal.clients - DEBUG - Filling in custom suback event callback...
2020-01-25 21:25:57,168 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - Produced [disconnect] event
2020-01-25 21:25:57,168 - AWSIoTPythonSDK.core.protocol.connection.cores - DEBUG - backOff: current backoff time is: 1 sec.
2020-01-25 21:25:57,171 - AWSIoTPythonSDK.core.protocol.internal.workers - DEBUG - Dispatching [disconnect] event

[FlyingDiver]

Separately, when I try to connect from the new plugin version, it just times out:

   MQTT Connector Debug            Amazon IoT: Broker __init__ address = avv4sfoj382qz-ats.iot.us-east-1.amazonaws.com:8883, ca_bundle = /Library/Application Support/Perceptive Automation/Indigo 7.4/Certs/AmazonRootCA1.pem, cert_file = /Library/Application Support/Perceptive Automation/Indigo 7.4/Certs/Indigo-Plugin.cert.pem, private_key = /Library/Application Support/Perceptive Automation/Indigo 7.4/Certs/Indigo-Plugin.private.key
   MQTT Connector Debug            Amazon IoT: Client is OffLine
   MQTT Connector Error            Amazon IoT: Exception while creating Broker object
Traceback (most recent call last):
  File "/Library/Application Support/Perceptive Automation/Indigo 7.4/Plugins/MQTT Connector.indigoPlugin/Contents/Server Plugin/aiot_broker.py", line 59, in __init__
    if not self.aIoTClient.connect():
  File "/Library/Application Support/Perceptive Automation/Indigo 7.4/Plugins/MQTT Connector.indigoPlugin/Contents/Server Plugin/AWSIoTPythonSDK/MQTTLib.py", line 513, in connect
    return self._mqtt_core.connect(keepAliveIntervalSecond)
  File "/Library/Application Support/Perceptive Automation/Indigo 7.4/Plugins/MQTT Connector.indigoPlugin/Contents/Server Plugin/AWSIoTPythonSDK/core/protocol/mqtt_core.py", line 199, in connect
    raise connectTimeoutException()
connectTimeoutException
   MQTT Connector Debug         

[FlyingDiver]

I'm currently using the non-Async calls, as I don't mind the plugin startup being delayed while the connection is made.

[FlyingDiver]

Nevermind. My Google-foo finally came back. Turns out that the default security policy that AWS creates is VERY restrictive. I fixed that and now both the sample app and plugin are working. I think.

[FlyingDiver]

Test release available. https://github.com/FlyingDiver/Indigo-MQTT/releases/tag/0.3.0

[FlyingDiver]

Released in 0.3.0.