Too much traffic with Easymeter Q3B (push)

[hvorragend]

Hallo,
ich lasse mir diverse Werte von einem Easymeter Q3B per MQTT liefern.

Ich habe die verschiedensten Konfigurationen ausprobiert. Der Zähler pusht alle 2 Sekunden. Und ich kann es nicht verzögern. Auch nicht mit "avg" oder "max".

aggtime: 10
interval: 0
read_timeout: 10

aggtime: -1
interval: -1
read_timeout: 10

Gibt es irgendeine Möglichkeit, die Last zu minimieren und den Traffic zu drosseln?

Vielen Dank.

[r00t-]

@hvorragend: mind sharing the full config?

[hvorragend]

This is my full config:

verbosity: 10
mqtt_broker: core-mosquitto
mqtt_port: 1883
mqtt_username: xxxx
mqtt_password: xxxx
mqtt_topic: vzlogger/data
mqtt_timestamp: "false"
meter1_protocol: d0
meter1_parity: "7E1"
meter1_baudrate: 300
meter1_baudrate_read: 300
meter1_pullseq: 2F3F210D0A
meter1_ackseq: 063030300d0a
meter1_aggtime: -1
meter1_interval: 60
meter1_read_timeout: 30
meter1_use_local_time: "false"
meter1_device: /dev/serial/by-id/usb-FTDI_FT232R_USB_UART_cxxx-if00-port0
meter1_channels:
  - identifier: 1-1:1.7.0
    aggmode: none
  - identifier: 1-1:2.7.0
    aggmode: none
  - identifier: 1-1:1.8.0
    aggmode: none
  - identifier: 1-1:1.8.1
    aggmode: none
  - identifier: 1-1:1.8.2
    aggmode: none
  - identifier: 1-1:2.8.0
    aggmode: none
  - identifier: 1-1:2.8.1
    aggmode: none
  - identifier: 1-1:2.8.2
    aggmode: none
meter2_protocol: sml
meter2_parity: 8N1
meter2_baudrate: 9600
meter2_baudrate_read: 9600
meter2_pullseq: ""
meter2_ackseq: ""
meter2_aggtime: 10
meter2_interval: 0
meter2_read_timeout: 10
meter2_use_local_time: "false"
meter2_device: >-
  /dev/serial/by-id/usb-Silicon_Labs_CP2104_USB_to_UART_Bridge_Controller_cxxx-if00-port0
meter2_channels:
  - identifier: 1-0:1.8.0*255
    aggmode: none
  - identifier: 1-0:1.8.1*255
    aggmode: none
  - identifier: 1-0:1.8.2*255
    aggmode: none
  - identifier: 1-0:2.8.0*255
    aggmode: none
  - identifier: 1-0:2.8.1*255
    aggmode: none
  - identifier: 1-0:2.8.2*255
    aggmode: none
  - identifier: 1-0:1.7.0*255
    aggmode: none
  - identifier: 1-0:21.7.0*255
    aggmode: none
  - identifier: 1-0:41.7.0*255
    aggmode: none
  - identifier: 1-0:61.7.0*255
    aggmode: none
  - identifier: 1-0:32.7.0*255
    aggmode: none

[hvorragend]

I am now using a workaround with a time_throttle Filter (window_size = 1min) in Home Assistant.
And I have excluded the original sensor from the recorder.

[r00t-]

sorry for the delayed response.

what is that config for?!
vzlogger uses json, not yaml:
https://github.com/volkszaehler/vzlogger/blob/master/src/Config_Options.cpp#L77

in general, the aggregation feature was made specifically to reduce the logging rate for meters that log data at a fixed rate.
either you configured this incorrectly,
or maybe it doesn't work correctly with the mqtt integration.

[hvorragend]

This is the entire configuration of your HA-addon. You can get it if you switch from GUI editor to YAML editor.

[r00t-]

this is not "our" addon, but a third-party product:
https://github.com/markussiebert/homeassistant-addon-vzlogger

i'd actually recommend to check the generated json vzlogger.conf, just to rule out errors in their config-generation.

[hvorragend]

Oh, now I'm back on topic. It's been a while since I've dealt with it more intensively.

I had asked the question in the HomeAssistant addon and I was directed to vzlogger. The addon really only passes the data through. And that's how the data comes in.

The addon has a YAML configuration and properly writes the values in vzlogger.conf.
I just didn't think to include the JSON formatting in this ticket. Sorry.

But in the end the parameters baudrate, aggtime, ackseq etc. are identical. I was able to understand this in vzlogger.conf.

I'm just wondering whether there are any options in vzlogger to throttle the data.

[r00t-]

i'm sorry, but if you want support from us, you should really provide the configuration in the format our program actually uses, and that we can test, and not expect us to do some conversation for you.

but.. (see next comment)

[r00t-]

the option in vzlogger is to use aggregation, as i wrote.
but it appears that the mqtt intrgration bypasses the regular aggregation process.
this is a design error of the mqtt integration of vzlogger, i'll create a ticket to fix it, but i'm afraid there won't be quick fix.

[r00t-]

i'll close this issue in favor or the more technical issue now created.

[r00t-]

actually i missed this,
aggregation is supported,
the aggregated values are published in a separate topic(?).
https://github.com/mbehr1/vzlogger/blob/a3434834dc4344bf3c29a4dd201a9ec8d838c1d2/src/mqtt.cpp#L240

[hvorragend]

i'm sorry, but if you want support from us, you should really provide the configuration in the format our program actually uses, and that we can test, and not expect us to do some conversation for you.

Sorry. It couldn't be faster. I didn't have quick access to the docker container.

---

{
    // General Settings
    "retry" : 10,        // http retry delay in seconds
    "daemon": true,
    "verbosity" : 10,     // (0=log_alert, 1=log_error, 3=log_warning, 5=log_info, 10=log_debug, 15=log_finest)
    "log" : "/vzlogger.log",
 
    // Build-in HTTP server
    "local" : {
         "enabled" : false,   // enable local HTTPd for serving live readings
         "port" : 8081,
         "index" : true,      // index listing of channels (if no UUID)
         "timeout" : 30,
         "buffer" : 600
    },

    // mqtt client support (if ENABLE_MQTT set at cmake generation)
    "mqtt": {
        "enabled": true,  // enable mqtt client. needs host and port as well
        "host": "core-mosquitto", // mqtt server addr
        "port": 1883, // 1883 for unencrypted, 8883 enc, 8884 enc cert needed,
        "cafile": "", // optional file with server CA
        "capath": "", // optional path for server CAs. see mosquitto.conf. Specify only cafile or capath
        "certfile": "", // optional file for your client certificate (e.g. client.crt)
        "keyfile": "", // optional path for your client certficate private key (e.g. client.key)
        "keypass": "", // optional password for your private key
        "keepalive": 30, // optional keepalive in seconds.
        "topic": "vzlogger/data", // optional topic dont use $ at start and no / at end
        "user": "******", // optional user name for the mqtt server
        "pass": "******", // optional password for the mqtt server
        "retain": false, // optional use retain message flag
        "rawAndAgg": false, // optional publish raw values even if agg mode is used
        "qos": 0, // optional quality of service, default is 0
        "timestamp": false // optional whether to include a timestamp in the payload
    },

    // information on the (up to two) meters and their channels
    "meters" : [{
        "protocol" : "d0",
        "enabled" : true,
        "device" : "/dev/serial/by-id/usb-FTDI_FT232R_USB_UART_XXXXXXX-if00-port0",
        "parity" : "7E1",
        "baudrate" : 300,
        "baudrate_read" : 300,
        "pullseq" : "2F3F210D0A",
        "ackseq" : "063030300d0a",
        "aggtime" : -1,    // aggregate readings for <aggtime> seconds
        "interval" : 60,
        "read_timeout" : 30,
        "aggfixedinterval" : true,       // round timestamps to nearest <aggtime>
        "duplicates" : 60,
        "use_local_time" : false,
        "channels": [{
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-1:1.7.0", 
           "aggmode": "none" },
         {
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-1:2.7.0", 
           "aggmode": "none" },
         {
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-1:1.8.0", 
           "aggmode": "none" },
         {
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-1:1.8.1", 
           "aggmode": "none" },
         {
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-1:1.8.2", 
           "aggmode": "none" },
         {
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-1:2.8.0", 
           "aggmode": "none" },
         {
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-1:2.8.1", 
           "aggmode": "none" },
         {
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-1:2.8.2", 
           "aggmode": "none" }]
        },{
        "protocol" : "sml",
        "enabled" : true,
        "device" : "/dev/serial/by-id/usb-Silicon_Labs_CP2104_USB_to_UART_Bridge_Controller_XXXXXX-if00-port0",
        "parity" : "8N1",
        "baudrate" : 9600,
        "baudrate_read" : 9600,
        "pullseq" : "",
        "ackseq" : "",
        "aggtime" : 10,
        "interval" : 0,
        "read_timeout" : 10,
        "aggfixedinterval" : true,
        "duplicates" : 60,
        "use_local_time" : false,
        "channels": [{
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-0:1.8.0*255", 
           "aggmode": "none" },
         {
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-0:1.8.1*255", 
           "aggmode": "none" },
         {
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-0:1.8.2*255", 
           "aggmode": "none" },
         {
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-0:2.8.0*255", 
           "aggmode": "none" },
         {
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-0:2.8.1*255", 
           "aggmode": "none" },
         {
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-0:2.8.2*255", 
           "aggmode": "none" },
         {
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-0:1.7.0*255", 
           "aggmode": "none" },
         {
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-0:21.7.0*255", 
           "aggmode": "none" },
         {
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-0:41.7.0*255", 
           "aggmode": "none" },
         {
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-0:61.7.0*255", 
           "aggmode": "none" },
         {
           "uuid": "1", 
           "api": "null", 
           "identifier": "1-0:32.7.0*255", 
           "aggmode": "none" }]
    }]
}

[hvorragend]

actually i missed this, aggregation is supported, the aggregated values are published in a separate topic(?). https://github.com/mbehr1/vzlogger/blob/a3434834dc4344bf3c29a4dd201a9ec8d838c1d2/src/mqtt.cpp#L240

Yes, I am aware of that. But that doesn't change the fact that the topics are updated too frequently.