Support MQTT

[ZuinigeRijder]

@playbackandrewind asked a question:

You're supporting Domoticz, which is not so common here.
Is there a chance to see an MQTT connector or similar to store the data in a SQL Database (Homeassistant, ioBroker etc)?

Currently I do not have experience with implementing MQTT, but of course I can figure out the best way to support this. Therefore this discussion topic is opened.

[ZuinigeRijder]

I assume you want to run a MQTT broker on Raspberry Pi or another server. Question of course, which MQTT Broker should be used.

I found this article: Getting started with MQTT on Raspberry Pi using Python.

The following questions arise:

• Is "Mosquitto" a good choice? Or another implementation? Or configurable broker?
• Or should the python script just connect to an existing MQTT Broker, e.g. configured in HomeAssistent, ioBroker, etc? So script only needs to connect to a MQTT boker and assumption (if configured) that it is available?
• Probably for the MQTT client Paho MQTT library dependency is the best client library for Python implementation?

Any other ideas?

[ZuinigeRijder]

I have played around locally in python to be possible to send data to a MQTT broker.
Following configuration added to soliscloud_to_pvoutput.cfg:

[MQTT]
send_to_mqtt = True
mqtt_broker_hostname = test.mosquitto.org
mqtt_broker_port = 1883
mqtt_main_topic = soliscloud2pvoutput
mqtt_power_generated_id = power_generated
mqtt_ac_volt_id = ac_volt
mqtt_inverter_temp_id = inverter_temp
mqtt_volt_id = volt
mqtt_solarpower_id = solarpower
mqtt_energygeneration_id = energygeneration
mqtt_batterypower_id = batterypower
mqtt_gridpower_id = gridpower
mqtt_familyloadpower_id = familyloadpower
mqtt_homeconsumption_id = homeconsumption

And the output when running:

20240112 14:13:31: INFO: body: {"id":"12345678912345","sn":"12345678901234"}
20240112 14:14:32: INFO: data=20240112,14:12,300,50,,240.1,21.9,164.7
20240112 14:14:32: INFO: topic: soliscloud2pvoutput/ac_volt, msg: 240.1
20240112 14:14:32: INFO: topic: soliscloud2pvoutput/inverter_temp, msg: 21.9
20240112 14:14:32: INFO: topic: soliscloud2pvoutput/volt, msg: 164.7
20240112 14:14:32: INFO: topic: soliscloud2pvoutput/solarpower, msg: 50
20240112 14:14:32: INFO: topic: soliscloud2pvoutput/energygeneration, msg: 300
20240112 14:14:32: INFO: topic: soliscloud2pvoutput/batterypower, msg: 0
20240112 14:14:32: INFO: topic: soliscloud2pvoutput/gridpower, msg: 0
20240112 14:14:32: INFO: topic: soliscloud2pvoutput/familyloadpower, msg: 0
20240112 14:14:32: INFO: topic: soliscloud2pvoutput/homeconsumption, msg: 50

Not sure how the MQTT topic should be for HomeAssistent or ioBroker. And of course mqtt_broker_hostname will be probably localhost, but now used a test MQTT Broker server.

[playbackandrewind]

My MQTT is password protected, so we maybe need additional parameters for mqtt_username and mqtt_password

[ZuinigeRijder]

If you have multiple inverters, then you want to distinguish them, so I added the soliscloud inverter serial number as subtopic.

Example output:

20240114 16:01:31: INFO: body: {"id":"1308675217946123456","sn":"1001020218123456"}
20240114 16:02:32: INFO: data=20240114,16:01,200,20,,237.4,22.0,158.2
20240114 16:02:32: INFO: topic: SolisCloud2PVOutput/1001020218123456/last_update, msg: 20240114 16:01
20240114 16:02:32: INFO: topic: SolisCloud2PVOutput/1001020218123456/power_generated, msg: 20;200
20240114 16:02:32: INFO: topic: SolisCloud2PVOutput/1001020218123456/ac_volt, msg: 237.4
20240114 16:02:32: INFO: topic: SolisCloud2PVOutput/1001020218123456/inverter_temp, msg: 22.0
20240114 16:02:32: INFO: topic: SolisCloud2PVOutput/1001020218123456/volt, msg: 158.2
20240114 16:02:32: INFO: topic: SolisCloud2PVOutput/1001020218123456/solarpower, msg: 20
20240114 16:02:32: INFO: topic: SolisCloud2PVOutput/1001020218123456/energygeneration, msg: 200
20240114 16:02:32: INFO: topic: SolisCloud2PVOutput/1001020218123456/batterypower, msg: 0
20240114 16:02:32: INFO: topic: SolisCloud2PVOutput/1001020218123456/gridpower, msg: 0
20240114 16:02:32: INFO: topic: SolisCloud2PVOutput/1001020218123456/familyloadpower, msg: 0
20240114 16:02:32: INFO: topic: SolisCloud2PVOutput/1001020218123456/homeconsumption, msg: 20

Tested with MQTT-explorer and the ouput there:

SolisCloud2PVOutput
    10010202182123456
        last_update = 20240114 16:01
        power_generated = 20;200
        ac_volt = 237.4
        inverter_temp = 22.0
        volt = 158.2
        solarpower = 20
        energygeneration = 200
        batterypower = 0
        gridpower = 0
        familyloadpower = 0
        homeconsumption = 20

To add the new data to Home Assistant, new MQTT sensors should be added to the configuration.yaml, e.g.:

# Start SolisCloud2PVOutput 

- platform: mqtt
  state_topic: "SolisCloud2PVOutput/1001020218123456/solarpower"
  name: "Solar power"
  unit_of_measurement: W
  scan_interval: 60

- platform: mqtt
  state_topic: "SolisCloud2PVOutput/1001020218123456/energygeneration"
  name: "Energy generated today"
  unit_of_measurement: W
  scan_interval: 60

- platform: mqtt
  state_topic: "SolisCloud2PVOutput/1001020218123456/volt"
  name: "Panel Voltage"
  unit_of_measurement: V
  scan_interval: 60

- platform: mqtt
  state_topic: "SolisCloud2PVOutput/1001020218123456/ac_volt"
  name: "AC Voltage"
  unit_of_measurement: V
  scan_interval: 60

- platform: mqtt
  state_topic: "SolisCloud2PVOutput/1001020218123456/inverter_temp"
  name: "Inverter temperature"
  unit_of_measurement: °C
  scan_interval: 60

....

I do not have a Home Assistant installation, so not actually tested.

[ZuinigeRijder]

Anyone interested in the code and willing to test it with HomeAssistant and/or another MQTT aware implementation.

[playbackandrewind]

Yes, of course! Please publish a beta when you're ready ...

[ZuinigeRijder]

@playbackandrewind See release R1.15.0
Let me know if it works for you and if other improvements are needed.....

[playbackandrewind]

Works ! But i had to change the MQTT-Topic from random to fixed:

// f"{MQTT_MAIN_TOPIC}-{SOLISCLOUD_INVERTER_SN}-{random.randint(0, 1000)}"
f"{MQTT_MAIN_TOPIC}-{SOLISCLOUD_INVERTER_SN}"

The object-name must be fixed. If we add a random number, it creates every start a new device name, that's uncommon.

I grab now some data from the MQTT and forward them into a SQL database, tomorrow we know more how it runs.

I think it will run perfect ...

[playbackandrewind]

Aaahhhhh .. i see there a litte problem ....

386: If datetime_now.hour < 5 or datetime_now.hour > 22:

MQTT should run 24/365 - only the posting to pvoutput.org could/should be limited to 2-22 I think.

[ZuinigeRijder]

There are good reasons to start this once a day, with a crontab or something else:

• Between 22:00 and 5:00 the inverter will not have updates
• if somehow the script aborts, it will at least start the next day fresh
• you have to make sure the script starts automatically anyway when the system reboots
• the logfiles/output can be written once per day and removed afterwards
• if there is a memory leak (python garbage collection problem?), this will not be a problem because of the restart
• the today string is now only fetched once

So I would still suspect, that you start the script once per day, also when it writes to MQTT only.

What system/OS are you running the python script?
And are you using Home Assistant, ioBroker or something else?

[playbackandrewind]

Win2k22 Server running ioBroker etc.

[ZuinigeRijder]

Works ! But i had to change the MQTT-Topic from random to fixed:

// f"{MQTT_MAIN_TOPIC}-{SOLISCLOUD_INVERTER_SN}-{random.randint(0, 1000)}" f"{MQTT_MAIN_TOPIC}-{SOLISCLOUD_INVERTER_SN}"

The object-name must be fixed. If we add a random number, it creates every start a new device name, that's uncommon.

I grab now some data from the MQTT and forward them into a SQL database, tomorrow we know more how it runs.

I think it will run perfect ...

Ok, makes sense to remove the random number, especially because the inverter serial number is unique.

[ZuinigeRijder]

@playbackandrewind I removed the random number, see this release R3.15.1

[playbackandrewind]

Thanks, works perfect!

Is it possible to add some battery data to the MQTT output? Like charging/discharging state/voltages/level and so on?

It would be useful for many people to store this data into a SQL database for monitoring and comparing the whole system.

[ZuinigeRijder]

I do not have a battery, so it is more difficult for me to test it. Maybe you could make a proposal, based on the SolisCloud API in this PDF and test something. Thereafter I can incorporate these in the official version.

You have to look for page 21-27, some battery related information, e.g.:

3.6. /v1/api/inverterDetail (Inverter Details)
....
batteryCapacitySoc          Battery SOC
batteryHealthSoh            Battery SOH
batteryVoltage              Battery voltage

You can get the data with for example:

inverter_detail["batteryCapacitySoc"]

I do not see directly for all the battery related information you asked for, e.g. charging/discharging, although this would mean a positive or negative batteryPower, which is already reported?