Skip to content

[HELP] Connecting to Thingsboard and using rest connector #696

Description

@Hkrie

Describe the issue
I try to connect to thingsboard and use the rest connector without success.
I have a Thingsboard instance running on:
http://thingsboard.demo.local/
and a rest-api-endpoint:
http://localhost:8052/mmsp/status/systemStatus/get
==> These work correctly in browser.

I followed the installation guide:
https://thingsboard.io/docs/iot-gateway/install/source-installation/
and configured the tb_gateway.yml using the tb-gateway-configurator:
https://thingsboard.io/docs/iot-gateway/guides/how-to-configure-gateway-using-configurator/
After that I adjusted the rest.json.

To see that I can actually connect to the thingsboard and it's not a problem with my thingsboard instance I ran a simple node script:

const mqtt = require('mqtt')

const client = mqtt.connect('mqtt://thingsboard.demo.local/', {
    username: MY_GATEWAY_ACCESS_TOKEN
})

client.on('connect', function () {
    console.log('connected')
})

Configuration (Attach your configuration file)
tb_gateway.yaml

thingsboard:
  host: thingsboard.demo.local
  port: 1883
  remoteShell: false
  remoteConfiguration: false
  statsSendPeriodInSeconds: 3600
  minPackSendDelayMS: 0
  checkConnectorsConfigurationInSeconds: 30
  security:
    accessToken: MY_GATEWAY_ACCESS_TOKEN
  qos: 1
storage:
  type: memory
  read_records_count: 10
  max_records_count: 1000
grpc:
  enabled: false
  serverPort: 9595
  keepaliveTimeMs: 10000
  keepaliveTimeoutMs: 5000
  keepalivePermitWithoutCalls: true
  maxPingsWithoutData: 0
  minTimeBetweenPingsMs: 10000
  minPingIntervalWithoutDataMs: 5000
connectors:
    - 
    type: rest
    name: Rest Connector
    configuration: rest.json

rest.json

{
  "host": "127.0.0.1",
  "port": "8052",
  "SSL": false,
  "mapping":[
    {
      "endpoint": "/mmsp/status/systemStatus/get",
      "HTTPMethods": [
        "GET"
      ],
      "security":
      {
          "type": "anonymous"
      },
      "converter": {
        "type": "json",
        "deviceNameExpression": "Device Simulation",
        "deviceTypeExpression": "default",
        "attributes": [
          {
            "type": "string",
            "key": "systemStatusExpression",
            "value": "${name}"
          }
        ],
        "timeseries": [
          {
              "type": "string",
              "key": "systemStatus",
              "value": "${data}"
          }
        ]
      }
    }]
}

Connector name (If you need help with some connector/converter):
(REST Connector)

Error traceback (If it was raised):
complete command line output:

root@LT000985:~/thingsboard/example-gateway/thingsboard-gateway# thingsboard-gateway
''2022-01-14 12:07:41' - INFO - tb_gateway_service - 93 - Gateway starting...'
''2022-01-14 12:07:41' - DEBUG - tb_updater - 69 - Checking for new version'
''2022-01-14 12:07:41' - INFO - tb_gateway_service - 98 - ThingsBoard IoT gateway version: 3.0.1'
''2022-01-14 12:07:41' - DEBUG - tb_client - 124 - connecting to ThingsBoard'
''2022-01-14 12:07:41' - DEBUG - tb_device_mqtt - 172 - <paho.mqtt.client.Client object at 0x7f50afe7f3d0>'
''2022-01-14 12:07:41' - DEBUG - tb_device_mqtt - 173 - Disconnecting from ThingsBoard'
''2022-01-14 12:07:41' - DEBUG - tb_device_mqtt - 302 - Subscribed to * with id 1'
''2022-01-14 12:07:41' - INFO - tb_gateway_mqtt - 175 - Subscribed to *|* with id 1 for device *'
''2022-01-14 12:07:41' - DEBUG - tb_logger - 38 - Added remote handler to log service'
''2022-01-14 12:07:41' - DEBUG - tb_logger - 38 - Added remote handler to log extension'
''2022-01-14 12:07:41' - DEBUG - tb_logger - 38 - Added remote handler to log converter'
''2022-01-14 12:07:41' - DEBUG - tb_logger - 38 - Added remote handler to log connector'
''2022-01-14 12:07:41' - DEBUG - tb_logger - 38 - Added remote handler to log tb_connection'
''2022-01-14 12:07:41' - DEBUG - memory_event_storage - 27 - Memory storage created with following configuration: 
Max size: 100000
 Read records per time: 100'
''2022-01-14 12:07:41' - DEBUG - tb_gateway_service - 1014 - No device found in connected device file.'
''2022-01-14 12:07:41' - DEBUG - tb_gateway_service - 972 - Persistent keys file not found'
''2022-01-14 12:07:41' - DEBUG - tb_loader - 35 - Root path is: /usr/local/lib/python3.8/dist-packages/thingsboard_gateway-3.0.1-py3.8.egg/thingsboard_gateway'
''2022-01-14 12:07:41' - DEBUG - tb_loader - 57 - ModuleSpec(name='MqttConnector', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7f50af6303d0>, origin='/usr/local/lib/python3.8/dist-packages/thingsboard_gateway-3.0.1-py3.8.egg/thingsboard_gateway/extensions/mqtt/custom_mqtt_uplink_converter.py')'
''2022-01-14 12:07:41' - DEBUG - tb_loader - 57 - ModuleSpec(name='MqttConnector', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7f50af630460>, origin='/usr/local/lib/python3.8/dist-packages/thingsboard_gateway-3.0.1-py3.8.egg/thingsboard_gateway/connectors/mqtt/mqtt_connector.py')'
''2022-01-14 12:07:41' - INFO - tb_loader - 66 - Import MqttConnector from /usr/local/lib/python3.8/dist-packages/thingsboard_gateway-3.0.1-py3.8.egg/thingsboard_gateway/connectors/mqtt.'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'topicFilter' found in mapping handler: {"topicFilter": "/sensor/data", "converter": {"type": "json", "deviceNameJsonExpression": "${serialNumber}", "deviceTypeJsonExpression": "${sensorType}", "timeout": 60000, "attributes": [{"type": "string", "key": "model", "value": "${sensorModel}"}], "timeseries": [{"type": "double", "key": "temperature", "value": "${temp}"}, {"type": "double", "key": "humidity", "value": "${hum}"}]}}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'converter' found in mapping handler: {"topicFilter": "/sensor/data", "converter": {"type": "json", "deviceNameJsonExpression": "${serialNumber}", "deviceTypeJsonExpression": "${sensorType}", "timeout": 60000, "attributes": [{"type": "string", "key": "model", "value": "${sensorModel}"}], "timeseries": [{"type": "double", "key": "temperature", "value": "${temp}"}, {"type": "double", "key": "humidity", "value": "${hum}"}]}}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 162 - mapping handler has all mandatory keys => accepted: {"topicFilter": "/sensor/data", "converter": {"type": "json", "deviceNameJsonExpression": "${serialNumber}", "deviceTypeJsonExpression": "${sensorType}", "timeout": 60000, "attributes": [{"type": "string", "key": "model", "value": "${sensorModel}"}], "timeseries": [{"type": "double", "key": "temperature", "value": "${temp}"}, {"type": "double", "key": "humidity", "value": "${hum}"}]}}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'topicFilter' found in mapping handler: {"topicFilter": "/sensor/+/data", "converter": {"type": "json", "deviceNameTopicExpression": "(?<=sensor/)(.*?)(?=/data)", "deviceTypeTopicExpression": "Thermometer", "timeout": 60000, "attributes": [{"type": "string", "key": "model", "value": "${sensorModel}"}], "timeseries": [{"type": "double", "key": "temperature", "value": "${temp}"}, {"type": "double", "key": "humidity", "value": "${hum}"}]}}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'converter' found in mapping handler: {"topicFilter": "/sensor/+/data", "converter": {"type": "json", "deviceNameTopicExpression": "(?<=sensor/)(.*?)(?=/data)", "deviceTypeTopicExpression": "Thermometer", "timeout": 60000, "attributes": [{"type": "string", "key": "model", "value": "${sensorModel}"}], "timeseries": [{"type": "double", "key": "temperature", "value": "${temp}"}, {"type": "double", "key": "humidity", "value": "${hum}"}]}}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 162 - mapping handler has all mandatory keys => accepted: {"topicFilter": "/sensor/+/data", "converter": {"type": "json", "deviceNameTopicExpression": "(?<=sensor/)(.*?)(?=/data)", "deviceTypeTopicExpression": "Thermometer", "timeout": 60000, "attributes": [{"type": "string", "key": "model", "value": "${sensorModel}"}], "timeseries": [{"type": "double", "key": "temperature", "value": "${temp}"}, {"type": "double", "key": "humidity", "value": "${hum}"}]}}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'topicFilter' found in mapping handler: {"topicFilter": "/custom/sensors/+", "converter": {"type": "custom", "extension": "CustomMqttUplinkConverter", "extension-config": {"temperatureBytes": 2, "humidityBytes": 2, "batteryLevelBytes": 1}}}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'converter' found in mapping handler: {"topicFilter": "/custom/sensors/+", "converter": {"type": "custom", "extension": "CustomMqttUplinkConverter", "extension-config": {"temperatureBytes": 2, "humidityBytes": 2, "batteryLevelBytes": 1}}}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 162 - mapping handler has all mandatory keys => accepted: {"topicFilter": "/custom/sensors/+", "converter": {"type": "custom", "extension": "CustomMqttUplinkConverter", "extension-config": {"temperatureBytes": 2, "humidityBytes": 2, "batteryLevelBytes": 1}}}'
''2022-01-14 12:07:41' - INFO - mqtt_connector - 165 - Number of accepted mapping handlers: 3'
''2022-01-14 12:07:41' - INFO - mqtt_connector - 169 - Number of rejected mapping handlers: 0'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'deviceNameFilter' found in serverSideRpc handler: {"deviceNameFilter": ".*", "methodFilter": "echo", "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}", "responseTopicExpression": "sensor/${deviceName}/response/${methodName}/${requestId}", "responseTimeout": 10000, "valueExpression": "${params}"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'methodFilter' found in serverSideRpc handler: {"deviceNameFilter": ".*", "methodFilter": "echo", "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}", "responseTopicExpression": "sensor/${deviceName}/response/${methodName}/${requestId}", "responseTimeout": 10000, "valueExpression": "${params}"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'requestTopicExpression' found in serverSideRpc handler: {"deviceNameFilter": ".*", "methodFilter": "echo", "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}", "responseTopicExpression": "sensor/${deviceName}/response/${methodName}/${requestId}", "responseTimeout": 10000, "valueExpression": "${params}"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'valueExpression' found in serverSideRpc handler: {"deviceNameFilter": ".*", "methodFilter": "echo", "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}", "responseTopicExpression": "sensor/${deviceName}/response/${methodName}/${requestId}", "responseTimeout": 10000, "valueExpression": "${params}"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 162 - serverSideRpc handler has all mandatory keys => accepted: {"deviceNameFilter": ".*", "methodFilter": "echo", "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}", "responseTopicExpression": "sensor/${deviceName}/response/${methodName}/${requestId}", "responseTimeout": 10000, "valueExpression": "${params}"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'deviceNameFilter' found in serverSideRpc handler: {"deviceNameFilter": ".*", "methodFilter": "no-reply", "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}", "valueExpression": "${params}"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'methodFilter' found in serverSideRpc handler: {"deviceNameFilter": ".*", "methodFilter": "no-reply", "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}", "valueExpression": "${params}"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'requestTopicExpression' found in serverSideRpc handler: {"deviceNameFilter": ".*", "methodFilter": "no-reply", "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}", "valueExpression": "${params}"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'valueExpression' found in serverSideRpc handler: {"deviceNameFilter": ".*", "methodFilter": "no-reply", "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}", "valueExpression": "${params}"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 162 - serverSideRpc handler has all mandatory keys => accepted: {"deviceNameFilter": ".*", "methodFilter": "no-reply", "requestTopicExpression": "sensor/${deviceName}/request/${methodName}/${requestId}", "valueExpression": "${params}"}'
''2022-01-14 12:07:41' - INFO - mqtt_connector - 165 - Number of accepted serverSideRpc handlers: 2'
''2022-01-14 12:07:41' - INFO - mqtt_connector - 169 - Number of rejected serverSideRpc handlers: 0'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'topicFilter' found in connectRequests handler: {"topicFilter": "sensor/connect", "deviceNameJsonExpression": "${SerialNumber}"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 162 - connectRequests handler has all mandatory keys => accepted: {"topicFilter": "sensor/connect", "deviceNameJsonExpression": "${SerialNumber}"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'topicFilter' found in connectRequests handler: {"topicFilter": "sensor/+/connect", "deviceNameTopicExpression": "(?<=sensor/)(.*?)(?=/connect)"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 162 - connectRequests handler has all mandatory keys => accepted: {"topicFilter": "sensor/+/connect", "deviceNameTopicExpression": "(?<=sensor/)(.*?)(?=/connect)"}'
''2022-01-14 12:07:41' - INFO - mqtt_connector - 165 - Number of accepted connectRequests handlers: 2'
''2022-01-14 12:07:41' - INFO - mqtt_connector - 169 - Number of rejected connectRequests handlers: 0'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'topicFilter' found in disconnectRequests handler: {"topicFilter": "sensor/disconnect", "deviceNameJsonExpression": "${SerialNumber}"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 162 - disconnectRequests handler has all mandatory keys => accepted: {"topicFilter": "sensor/disconnect", "deviceNameJsonExpression": "${SerialNumber}"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'topicFilter' found in disconnectRequests handler: {"topicFilter": "sensor/+/disconnect", "deviceNameTopicExpression": "(?<=sensor/)(.*?)(?=/disconnect)"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 162 - disconnectRequests handler has all mandatory keys => accepted: {"topicFilter": "sensor/+/disconnect", "deviceNameTopicExpression": "(?<=sensor/)(.*?)(?=/disconnect)"}'
''2022-01-14 12:07:41' - INFO - mqtt_connector - 165 - Number of accepted disconnectRequests handlers: 2'
''2022-01-14 12:07:41' - INFO - mqtt_connector - 169 - Number of rejected disconnectRequests handlers: 0'
''2022-01-14 12:07:41' - ERROR - mqtt_connector - 142 - 'attributeRequests' section missing from configuration'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'deviceNameFilter' found in attributeUpdates handler: {"deviceNameFilter": "SmartMeter.*", "attributeFilter": "uploadFrequency", "topicExpression": "sensor/${deviceName}/${attributeKey}", "valueExpression": "{\"${attributeKey}\":\"${attributeValue}\"}"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'attributeFilter' found in attributeUpdates handler: {"deviceNameFilter": "SmartMeter.*", "attributeFilter": "uploadFrequency", "topicExpression": "sensor/${deviceName}/${attributeKey}", "valueExpression": "{\"${attributeKey}\":\"${attributeValue}\"}"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'topicExpression' found in attributeUpdates handler: {"deviceNameFilter": "SmartMeter.*", "attributeFilter": "uploadFrequency", "topicExpression": "sensor/${deviceName}/${attributeKey}", "valueExpression": "{\"${attributeKey}\":\"${attributeValue}\"}"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 154 - Mandatory key 'valueExpression' found in attributeUpdates handler: {"deviceNameFilter": "SmartMeter.*", "attributeFilter": "uploadFrequency", "topicExpression": "sensor/${deviceName}/${attributeKey}", "valueExpression": "{\"${attributeKey}\":\"${attributeValue}\"}"}'
''2022-01-14 12:07:41' - DEBUG - mqtt_connector - 162 - attributeUpdates handler has all mandatory keys => accepted: {"deviceNameFilter": "SmartMeter.*", "attributeFilter": "uploadFrequency", "topicExpression": "sensor/${deviceName}/${attributeKey}", "valueExpression": "{\"${attributeKey}\":\"${attributeValue}\"}"}'
''2022-01-14 12:07:41' - INFO - mqtt_connector - 165 - Number of accepted attributeUpdates handlers: 1'
''2022-01-14 12:07:41' - INFO - mqtt_connector - 169 - Number of rejected attributeUpdates handlers: 0'
''2022-01-14 12:07:41' - DEBUG - tb_gateway_service - 1014 - No device found in connected device file.'
''2022-01-14 12:07:41' - ERROR - mqtt_connector - 207 - [Errno 111] Connection refused'
''2022-01-14 12:07:41' - DEBUG - tb_gateway_service - 615 - Send data Thread has been started successfully.'
''2022-01-14 12:07:41' - INFO - tb_gateway_service - 180 - Gateway started.'
''2022-01-14 12:07:41' - DEBUG - tb_client - 82 - TB client <paho.mqtt.client.Client object at 0x7f50afe7f3d0> connected to ThingsBoard'
''2022-01-14 12:07:41' - ERROR - tb_device_mqtt - 146 - connection FAIL with error 3 server unavailable'
''2022-01-14 12:07:41' - DEBUG - tb_device_mqtt - 123 - Disconnected client: <paho.mqtt.client.Client object at 0x7f50afe7f3d0>, user data: None, result code: 5'
''2022-01-14 12:07:42' - DEBUG - tb_client - 124 - connecting to ThingsBoard'
''2022-01-14 12:07:42' - DEBUG - tb_client - 82 - TB client <paho.mqtt.client.Client object at 0x7f50afe7f3d0> connected to ThingsBoard'
''2022-01-14 12:07:42' - ERROR - tb_device_mqtt - 146 - connection FAIL with error 3 server unavailable'
''2022-01-14 12:07:42' - DEBUG - tb_device_mqtt - 123 - Disconnected client: <paho.mqtt.client.Client object at 0x7f50afe7f3d0>, user data: None, result code: 5'
''2022-01-14 12:07:43' - DEBUG - tb_client - 124 - connecting to ThingsBoard'
''2022-01-14 12:07:44' - DEBUG - tb_client - 124 - connecting to ThingsBoard'
''2022-01-14 12:07:45' - DEBUG - tb_client - 124 - connecting to ThingsBoard'
''2022-01-14 12:07:46' - DEBUG - tb_client - 124 - connecting to ThingsBoard'
''2022-01-14 12:07:47' - DEBUG - tb_client - 124 - connecting to ThingsBoard'
''2022-01-14 12:07:49' - DEBUG - tb_client - 124 - connecting to ThingsBoard'
''2022-01-14 12:07:50' - DEBUG - tb_client - 124 - connecting to ThingsBoard'
''2022-01-14 12:07:51' - ERROR - mqtt_connector - 207 - [Errno 111] Connection refused'
''2022-01-14 12:07:51' - DEBUG - tb_client - 124 - connecting to ThingsBoard'
''2022-01-14 12:07:52' - DEBUG - tb_client - 124 - connecting to ThingsBoard'
''2022-01-14 12:07:53' - DEBUG - tb_client - 82 - TB client <paho.mqtt.client.Client object at 0x7f50afe7f3d0> connected to ThingsBoard'
''2022-01-14 12:07:53' - ERROR - tb_device_mqtt - 146 - connection FAIL with error 3 server unavailable'
''2022-01-14 12:07:53' - DEBUG - tb_device_mqtt - 123 - Disconnected client: <paho.mqtt.client.Client object at 0x7f50afe7f3d0>, user data: None, result code: 5'
''2022-01-14 12:07:53' - DEBUG - tb_client - 124 - connecting to ThingsBoard'
''2022-01-14 12:07:54' - DEBUG - tb_client - 124 - connecting to ThingsBoard'
''2022-01-14 12:07:55' - DEBUG - tb_client - 124 - connecting to ThingsBoard'
''2022-01-14 12:07:56' - DEBUG - tb_client - 124 - connecting to ThingsBoard'

complete-command-line-output.txt

Versions (please complete the following information):

  • OS: Windows 10 Subsystem: Ubuntu 20.04
  • Thingsboard IoT Gateway version 3.0.1
  • Python version 3.8.10

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions