Problem writting to DB

[PsySc0rpi0n]

Hi.
I just started trying TinyFlux to record sensors data into a DB.
I changed a few lines to match topic, mqtt broker, etc but when I run the script I get this errors trying to write the DB.

• Message received for topic "CCS811/caix1"...
 **Problem attempting to write: 'dCO2'
• Message received for topic "CCS811/caix1"...
 **Problem attempting to write: 'dCO2'
• Message received for topic "CCS811/caix1"...
 **Problem attempting to write: 'dCO2'
• Message received for topic "CCS811/caix1"...
 **Problem attempting to write: 'dCO2'
• Message received for topic "CCS811/caix1"...
 **Problem attempting to write: 'dCO2'
• Message received for topic "CCS811/caix1"...
 **Problem attempting to write: 'dCO2'
• Message received for topic "CCS811/caix1"...
 **Problem attempting to write: 'dCO2'
• Message received for topic "CCS811/caix1"...
 **Problem attempting to write: 'dCO2'

What do I need to change to make this work??

[citrusvanilla]

Hi @PsySc0rpi0n , you'll need to provide your full code. Can you zip the directory and drop it in here and I will take a look? Thanks. ~Justin

[PsySc0rpi0n]

I can paste the code and describe the contents of the folder. Zipping is harder for me.
But the setup is very simple. I have a gas sensor and in a near future I'll also have a load cell sensor.
I cloned your repo, created a Virtual Environment and installed what was needed in there.

The code is here:
https://pastebin.com/AQz0yrHn
I must also say that I didn't fully understood the variables used in run_tinyflux_worker() function. Namely, data, topic, payload and then the 3 other variables used in the IoT MQTT example code.

I have an ESP01 publishing messages from the sensors to a broker (Mosquitto) in a Rpi 4 where I'm also running TinyFlux.
The code above shows shows the topic and I'm sending 3 values. dCO2 and dtVOC from the sensors and then, I'm hardcoding a 3rd value which will be the load cell value when i add it to the system. For now, I'm just sending an hardcoded value.

So, the subscriber sees something like this:

{"CO2" : 400, "tVOC" : 0, "Peso" : 100}
{"CO2" : 1288, "tVOC" : 135, "Peso" : 100}
{"CO2" : 1309, "tVOC" : 138, "Peso" : 100}
{"CO2" : 1293, "tVOC" : 136, "Peso" : 100}
{"CO2" : 1262, "tVOC" : 131, "Peso" : 100}

[citrusvanilla]

It looks like a key error on this line: dCO2 = payload["dCO2"]

You have called the variable dCO2 and the key "dCO2" in your code with a zero, but in your comment you are using a capital letter 'O':

I'm sending 3 values. dCO2 and dtVOC from the sensors and then...

Check your zeroes and letter-Os.

Also to explain the code, you need to know that this process is multi-threaded. The mosquitto callback on_message is running in one thread, and writes a message to a shared inter-thread queue. The run_tinyflux_worker function is running in the main thread (separate from Mosquitto) and continuously checking the inter-thread queue for any message. When it sees one, it turns it into a TinyFlux point and writes it to the database. It is designed this way so that the Mosquitto callback thread does not spend any additional I/O time writing to the database, which is expensive. It makes the main thread do this work so it can immediately return to service any new MQTT messages.

Hope this helps, let me know if you need anything else.

[PsySc0rpi0n]

Hello.
I checked here

      dCO2 = payload["dCO2"]
      dtVOC = payload["dtVOC"]

and here and all of them are O's and not zeros.

fields={"CO2": dCO2, "tVOC": dtVOC},

I think I understand the concept of the code.
However, I'm not sure I understand the variables data, topic and payload.
Please confirm if my reasoning is correct about these 3 variables.
data is fetching the topic and payload (aka MQTT message) from the inter-thread queue (back from line 100 of my code).
topic saves the value of key topic from data variable, from the inter-thread.
payload saves the value of key payload from data variable, aka MQTT message from the inter-thread.

I think I got this part right.

Now, about the Point, I'm not sure what to put in tags. What are we supposed to put here? I just put something there as a placeholder!

After this, we will see what else can we do until I get tit working!

[citrusvanilla]

• topic is from the MQTT topic, its a string
• payload is just whatever is on the MQTT message body, in the script it assumes the the body is serialized JSON string in utf-8 (standard serialization for JSON)
• data is just the item from the queue, it is a python dict containing two key/values- the topic and payload key/values

From the tinyflux documentation- tags are for string key/values and fields are for numeric key/values:

https://tinyflux.readthedocs.io/en/latest/preparing-data.html

Tags are string/string key value pairs. The reason for having separate attributes for tags and fields in TinyFlux (and in InfluxDB) is twofold: It enforces consistency of types and data on the user’s side, and it allows the database to efficiently index on tags, which are attributes with low cardinality (compared to fields, which tend to have much higher variation across values).

[PsySc0rpi0n]

I got it. I understood what was wrong.

I was trying to read the wrong keys. I was trying to read dCO2 and dtVOC instead of CO2 and tVOC

But I still have something else to fix.
I'm getting this in the database:

2022-11-22T10:23:24.828596,CCS811/caix1,_tag_caix1,caix1,_field_CO2,400,_field_tVOC,_none,_field_Wheight,100

The last field, says _none. Why am I getting that? Is it because the value is zero?

[citrusvanilla]

This might be a bug. _none is supposed to be the serialized value for Python None type but i checked the serialization function and it will serialize the value 0 or 0.0 or other truthy None types to the string _none.

So yes, if you were trying to write a field value of 0.0 to the db it would appear in the CSV as the string _none. This is not the intended behavior. It should appear in the CSV as "0.0".

I create a bug report for this issue and will resolve it today:

#23

[citrusvanilla]

Issue has been resolved in v0.2.2 of TinyFlux (latest version).

To update using pip:

"pip install tinyflux --upgrade"