Kotori using Weewx (MQTT) - ERROR: Processing MQTT message failed from topic "weewx//loop": #5

RuiPinto96 · 2019-05-22T14:33:15Z

Hello!!

I have a weather station Vantage Pro 2 console and i'm trying to use Kotori and Weewx to publish data into grafana and store the weather data in InfluxDB.
I'm using an MQTT broker (Mosquitto) to send and receive the data from weewx to kotori.
But i have a problem processing MQTT messages from the topic. The topic that i used is 'weewx/#'.

I am getting this error from kotori.log:

ERROR   : Processing MQTT message failed from topic "weewx//loop"

If anyone could please help...

This is the error (kotori.log):

    2019-05-22T15:24:17+0100 [kotori.daq.services.mig            ] ERROR   : Processing MQTT message failed from topic "weewx//loop":

    [Failure instance: Traceback: <type 'exceptions.AttributeError'>: 'dict' object has no attribute 'slot'
/usr/lib/python2.7/threading.py:801:__bootstrap_inner
/usr/lib/python2.7/threading.py:754:run
/opt/kotori/lib/python2.7/site-packages/twisted/_threads/_threadworker.py:46:work
/opt/kotori/lib/python2.7/site-packages/twisted/_threads/_team.py:190:doWork
--- <exception caught here> ---
/opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:250:inContext
/opt/kotori/lib/python2.7/site-packages/twisted/python/threadpool.py:266:<lambda>
/opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:122:callWithContext
/opt/kotori/lib/python2.7/site-packages/twisted/python/context.py:85:callWithContext
/opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:234:process_message
/opt/kotori/lib/python2.7/site-packages/kotori/daq/services/mig.py:92:topology_to_storage
/opt/kotori/lib/python2.7/site-packages/kotori/daq/intercom/strategies.py:85:topology_to_storage

The text was updated successfully, but these errors were encountered:

RuiPinto96 · 2019-05-22T14:49:01Z

I am using a Raspberry Pi 2 with Debian stretch

amotl · 2019-05-22T18:39:15Z

Hi there,

thanks for writing in and for using Kotori with weeWX. It's probably just a configuration error you are experiencing.

As you can read on [1], the MQTT topic should look like weewx/097287c4-6fb0-4aeb-a095-00d65ecb15f7/Leoni/VantagePro2/loop, while your topic looks like weewx//loop - so there are some bits missing there.

The relevant configuration snippet for weewx.conf would be something like

[StdRestful]
    [[MQTT]]
        server_url = mqtt://username:password@mqtt.example.org:1883/
        topic = weewx/{USER}/{SITE}/{NODE}
        unit_system = METRIC

where you would replace {USER}/{SITE}/{NODE} by something reasonable which matches your own topology.

If you just want to address a single node without naming this thing at all,

topic = weewx/default/default/VantagePro2

should also be perfectly fine.

We hope this helps and are happy to hear if this works for you.

With kind regards,
Andreas.

[1] https://getkotori.org/docs/applications/weewx.html

RuiPinto96 · 2019-05-24T10:13:28Z

Hi!!
Thank you for the reply.
Because i don't always have access to the weather station console, I am using the Weewx Simulator which sends data replicating the console.
I'm trying in localhost first.
The parameters {USER}/{SITE}/{NODE} (097287c4-6fb0-4aeb-a095-00d65ecb15f7/Leoni/VantagePro2) are from where? the account of grafana or from a site that you created?

Best Regards

amotl · 2019-05-24T10:53:56Z

Hi there,

thanks for providing further information about your scenario.

I am using the Weewx Simulator which sends data replicating the console.

Funny thing, we haven't been aware there's even a weeWX-simulator out there. Good to know!

The parameters {USER}/{SITE}/{NODE} are from where?

The parameters can be chosen arbitrarily and designate the data ingress channel. As you can choose the values on your own, you are able to operate an arbitrary number of data ingress channels without configuring anything. For more details about the channel addressing scheme and its background, you might enjoy reading [1].

Recognizing you just want to operate a single data ingress channel, please just choose arbitrary values which might fit your scenario there - like outlined above

topic = weewx/default/default/VantagePro2

and you should be able ingest, process and visualize as intended. Please let us know if this works for you.

With kind regards,
Andreas.

[1] https://hiveeyes.org/docs/system/acquisition/index.html#addressing

RuiPinto96 · 2019-05-24T15:20:49Z

It worked!!
Thank you!!

amotl · 2019-05-24T17:49:21Z

Excellent. Enjoy the data acquisition and graphing!

If you like it, we are happy to receive stars on this project.

amotl · 2019-05-24T17:50:31Z

Let's close this issue but feel free to reopen if you have further questions.

RuiPinto96 · 2019-05-27T11:03:03Z

Hi there again!!
It stopped working...On friday i could see in localhost in grafana the weather data but after turn off the raspberry pi during the weekend it stopped working. Today (Monday) i started again and i couldn't received transactions to kotori. I used the topic that you told me to use (weewx/default/default/VantagePro2).

Here's the log's of kotori:

2019-05-27T11:45:16+0100 [kotori.daq.graphing.grafana.api    ] WARN    : Problem creating instant folder: HTTPConnectionPool(host='localhost', port=3000): Max retries exceeded with url: /api/folders/instagraf (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x72ceffd0>: Failed to establish a new connection: [Errno 111] Connection refused',))
2019-05-27T11:45:16+0100 [kotori.daq.services.mig            ] INFO    : Bootstrapping MqttInfluxGrafanaService. name=service-mig-weewx, channel={'realm': 'weewx', 'mqtt_topics': 'weewx/default/default/VantagePro2', 'enable': 'true', 'type': 'application', 'app_factory': 'kotori.daq.application.mqttkit:mqttkit_application'}
2019-05-27T11:45:16+0100 [kotori.daq.storage.influx          ] INFO    : Storage target is influxdb://localhost:8086
2019-05-27T11:45:16+0100 [kotori.daq.intercom.mqtt.base      ] INFO    : Starting PahoMqttAdapter. name=mqtt-weewx, broker=localhost:1883, object=<kotori.daq.intercom.mqtt.paho.PahoMqttAdapter object at 0x72cefe10>
2019-05-27T11:45:16+0100 [kotori.daq.services.mig            ] INFO    : [weewx       ] transactions: 0.00 tps
2019-05-27T11:45:16+0100 [kotori.core                        ] INFO    : Enabling vendors []
2019-05-27T11:45:16+0100 [kotori.daq.intercom.mqtt.paho      ] INFO    : Subscribing to topics [u'weewx/default/default/VantagePro2']. client=<paho.mqtt.client.Client object at 0x72c8c3b0>
2019-05-27T11:45:16+0100 [kotori.daq.intercom.mqtt.paho      ] INFO    : Subscribing to topic 'weewx/default/default/VantagePro2'
2019-05-27T11:46:16+0100 [kotori.daq.services.mig            ] INFO    : [weewx       ] transactions: 0.00 tps
2019-05-27T11:47:16+0100 [kotori.daq.services.mig            ] INFO    : [weewx       ] transactions: 0.00 tps
2019-05-27T11:48:16+0100 [kotori.daq.services.mig            ] INFO    : [weewx       ] transactions: 0.00 tps
2019-05-27T11:49:16+0100 [kotori.daq.services.mig            ] INFO    : [weewx       ] transactions: 0.00 tps
2019-05-27T11:50:16+0100 [kotori.daq.services.mig            ] INFO    : [weewx       ] transactions: 0.00 tps
2019-05-27T11:51:16+0100 [kotori.daq.services.mig            ] INFO    : [weewx       ] transactions: 0.00 tps

RuiPinto96 · 2019-05-27T11:18:06Z

Mqtt is publishing but i can't get the data to kotori.

May 27 12:16:17 raspberrypi weewx[607]: restx: MQTT: Published record 2019-05-27 12:16:00 WEST (1558955760)

RuiPinto96 · 2019-05-27T11:20:41Z

I also configured (change the Mqtt topic) the file Weewx.ini from kotori:

[weewx]
enable      = true
type        = application
realm       = weewx
mqtt_topics = weewx/default/default/VantagePro2
app_factory = kotori.daq.application.mqttkit:mqttkit_application

RuiPinto96 · 2019-05-27T11:29:01Z

SORRY!!
I fix it... it was the topic on weewx.ini, it should be mqtt_topics = weewx/default/default/VantagePro2/loop

amotl · 2019-05-27T11:44:06Z

Hi there,

good to hear you have been able to resolve the issue you have been observing.

Suffixing the designated topic with /loop in the Kotori configuration feels weird to me as I believe this will be handled internally. Maybe you've added an additional /loop suffix to the WeeWX MQTT Plugin configuration so the effective MQTT topic will look like /loop/loop?

As you can see with https://github.com/daq-tools/kotori/blob/0.22.7/etc/examples/vendors/weewx.ini#L14-L22, just setting the MQTT topic prefix to be the same as the realm with wildcarding will be totally sufficient for Kotori to pick up the data

[weewx]
enable      = true
type        = application
realm       = weewx
mqtt_topics = weewx/#
app_factory = kotori.daq.application.mqttkit:mqttkit_application

as long as the MQTT topic is correctly configured on the publisher side (here: WeeWX MQTT Plugin), which looks like that on our systems:

[StdRestful]
    [[MQTT]]
        server_url = mqtt://username:password@mqtt.example.org:1883/
        topic = weewx/MilkyWay/Earth/VantagePro2
        unit_system = METRIC

However, it always might happen that some semantics have changed due to more recent software versions, e.g. how the WeeWX MQTT Plugin is actually publishing data. If this is the case, we will be happy to find out.

With kind regards,
Andreas.

amotl · 2019-05-27T11:50:35Z

While your problem might have been from a different nature (as said: glad that you solved it!), I would also like to point out that warnings like

2019-05-27T11:45:16+0100 [kotori.daq.graphing.grafana.api    ] WARN    : Problem creating instant folder: HTTPConnectionPool(host='localhost', port=3000): Max retries exceeded with url: /api/folders/instagraf (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x72ceffd0>: Failed to establish a new connection: [Errno 111] Connection refused',))

indicate that Grafana is not running - at least it's not responding on localhost:3000. However, this should not have any impact on appropriate data acquisition.

RuiPinto96 · 2019-05-27T13:37:49Z

Yes, when i had the problem Grafana didn't work but now it works fine.
Thank You!!

RuiPinto96 · 2019-05-27T15:06:21Z

Hello there again!!
Just asking a question: Do you have packages for Centos 7?
I'm trying to install kotori on a Centos Machine instead of Raspberry pi.

Thank you

RuiPinto96 · 2019-08-06T16:40:14Z

Hello again!!
I just have a little question regarding the topic (weewx/default/default/VantagePro2/loop).
In here https://getkotori.org/docs/applications/weewx.html the topic in question has all variables of the station.
My question is:
The /loop is the representation of a multi-level wildcard, like /# ?

Best Regards.

amotl · 2019-08-06T17:38:36Z

Dear @TheOneWhoKnocks96,

The topic in question has all variables of the station.

That's correct.

Is /loop the representation of a multi-level wildcard, like /#?

I believe the /loop suffix automatically gets added by the WeeWX MQTT plugin, so we had to account for that. Kotori has a little specialization in its belly [1] by checking for topology.slot.startswith('loop') in order to designate the ingress package into the "sensors" domain.

In this manner, the /loop suffix just does the same as the regular /data suffix mentioned throughout the documentation [2].

With kind regards,
Andreas.

[1]

kotori/kotori/daq/intercom/strategies.py

Lines 84 to 89 in 5f5d951

    
           # data:     Regular endpoint 
        
           # loop:     WeeWX 
        
           # SENSOR:   Sonoff-Tasmota 
        
           if topology.slot.startswith('data') or topology.slot.startswith('loop') \ 
        
                   or topology.slot.endswith('SENSOR') or topology.slot.endswith('STATE'): 
        
               suffix = 'sensors'

[2] https://getkotori.org/docs/handbook/acquisition/protocol/mqtt.html#basic-examples

RuiPinto96 · 2019-08-09T14:44:34Z

Thanks for the response!!

amotl closed this as completed May 24, 2019

amotl mentioned this issue May 27, 2019

Packages for CentOS #6

Open

daq-tools deleted a comment from RuiPinto96 Aug 12, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Kotori using Weewx (MQTT) - ERROR: Processing MQTT message failed from topic "weewx//loop": #5

Kotori using Weewx (MQTT) - ERROR: Processing MQTT message failed from topic "weewx//loop": #5

RuiPinto96 commented May 22, 2019 •

edited by amotl

RuiPinto96 commented May 22, 2019

amotl commented May 22, 2019 •

edited

RuiPinto96 commented May 24, 2019

amotl commented May 24, 2019 •

edited

RuiPinto96 commented May 24, 2019

amotl commented May 24, 2019 •

edited

amotl commented May 24, 2019

RuiPinto96 commented May 27, 2019 •

edited by amotl

RuiPinto96 commented May 27, 2019 •

edited by amotl

RuiPinto96 commented May 27, 2019 •

edited by amotl

RuiPinto96 commented May 27, 2019 •

edited by amotl

amotl commented May 27, 2019 •

edited

amotl commented May 27, 2019

RuiPinto96 commented May 27, 2019

RuiPinto96 commented May 27, 2019

RuiPinto96 commented Aug 6, 2019

amotl commented Aug 6, 2019 •

edited

RuiPinto96 commented Aug 9, 2019

Kotori using Weewx (MQTT) - ERROR: Processing MQTT message failed from topic "weewx//loop": #5

Kotori using Weewx (MQTT) - ERROR: Processing MQTT message failed from topic "weewx//loop": #5

Comments

RuiPinto96 commented May 22, 2019 • edited by amotl

RuiPinto96 commented May 22, 2019

amotl commented May 22, 2019 • edited

RuiPinto96 commented May 24, 2019

amotl commented May 24, 2019 • edited

RuiPinto96 commented May 24, 2019

amotl commented May 24, 2019 • edited

amotl commented May 24, 2019

RuiPinto96 commented May 27, 2019 • edited by amotl

RuiPinto96 commented May 27, 2019 • edited by amotl

RuiPinto96 commented May 27, 2019 • edited by amotl

RuiPinto96 commented May 27, 2019 • edited by amotl

amotl commented May 27, 2019 • edited

amotl commented May 27, 2019

RuiPinto96 commented May 27, 2019

RuiPinto96 commented May 27, 2019

RuiPinto96 commented Aug 6, 2019

amotl commented Aug 6, 2019 • edited

RuiPinto96 commented Aug 9, 2019

RuiPinto96 commented May 22, 2019 •

edited by amotl

amotl commented May 22, 2019 •

edited

amotl commented May 24, 2019 •

edited

amotl commented May 24, 2019 •

edited

RuiPinto96 commented May 27, 2019 •

edited by amotl

RuiPinto96 commented May 27, 2019 •

edited by amotl

RuiPinto96 commented May 27, 2019 •

edited by amotl

RuiPinto96 commented May 27, 2019 •

edited by amotl

amotl commented May 27, 2019 •

edited

amotl commented Aug 6, 2019 •

edited