EMQ X integration to InfluxDB

[timhallinflux]

The current integration approach between EMQ X and InfluxDB is based on two things:

• UDP
• JSON formatted messages

InfluxDB OSS v1.x supports both of these. However, InfluxDB Cloud does not support either. InfluxDB OSS v2 has been released since November and also does not support UDP or JSON formatted messages.

Ideally, we could work together to upgrade the integration to leverage an HTTP-based integration between EMQ X which would allow for users to use either InfluxDB OSS v2 or InfluxDB Cloud.

[zmstone]

Hi @timhallinflux
Thank you for reporting this.
Is http api the only way forward with new version influxdb?

[timhallinflux]

Yes, for now. We are working on a native MQTT ingestion mechanism where you would configure InfluxDB Cloud to listen to the MQTT broker. That is planned for 2H21.

If you wanted to leverage Telegraf (our distributed agent which is open source), you could embed those bits into EMQ X and simply use your configuration approach to pull from the MQTT broker and then configure the appropriate InfluxDB output.

Happy to explain this approach in more detail... if you wish.

[zmstone]

Hi @timhallinflux @Benniu and I are interested to learn more details about how we can integrate influxDB.
Maybe we can have a chat over a call ? You can reach me at the email from my github commit history ;)

[zmstone]

sorry for the delayed reply.
we have been testing a bit with webhook.

converting this thread to a discussion.

[zibuyu1995]

We use EMQ X Webhook to persist data to the Influxdb Cloud:

Create Bucket

After logging in to the influxDB's console, go to Load Data page and create a new bucket.

Generate Token

Go back to Load Data page and generate a new token. At this time, we will generate a token with full access.

Create EMQ X Webhook Resource

Resource url: https://us-east-1-1.aws.cloud2.influxdata.com/api/v2/write

Create EMQ X Rule SQL

In this example we demonstrate how to store temperature and humidity sensor data to the influxdb cloud.

SELECT
  payload.location as location,
  payload.temp as temp,
  payload.hum as hum
FROM
  "emqx/test"

Add Rule Action

Add Request Parameters

Need to change {organizations} and {bucket}

• Method: POST
• Path: /api/v2/write?org={organizations}&bucket={bucket}&precision=ns

Add Request Headers

Authorization needs to be modified. Authorization is the token generated by Influxdb Cloud

• Content-Encoding: identity
• Content-Type: text/plain
• Accept: application/json
• Authorization: Token _kC8WcOxoxxjH0RAqHT-uHQD5T69RC-fvTEOX4WJMYbwNLH3pFyjQD-7JiEYAxGi84UubA9N9l0h4zkWJHB-T4w==
• User-Agent: emqx

Add Request Body

• Measurement: temp_hum
• Tags: location
• Fields: temp, hum

temp_hum,location=${location} temp=${temp},hum=${hum}

Publish temperature and humidity data

View EMQ X Rule Matched

View data in Influxdb Cloud

[timhallinflux]

super! Looks like it is all working. Perhaps the documentation needs to be updated to reflect this approach?

[zibuyu1995]

We provide video tutorials, see the https://www.youtube.com/watch?v=1S09epPWLpc&t=43s