Add support for Shelly Pro3EM using Modbus TCP

[Jalle19]

The https://github.com/victronenergy/dbus-shelly implementation has several shortcomings, among others:

• it only updates ever so often (whenever the device itself broadcasts a NotifyStatus to all connected clients)
• it's flaky, sometimes device information is missing (Shelly Pro 3EM product ID, firmware version etc. missing dbus-shelly#9)
• all phases are for some reason not shown in the remote console or VRM (Shelly PRO 3EM values not showing up in "Pages" View and VRM dbus-shelly#4)

I realized that there is already great boilerplate and plumbing available for Modbus devices, so why not use that since the new Shelly Pro3EM supports Modbus TCP.

Compared to dbus-shelly, this:

• updates every second (this is as fast as the Shelly internally updates itself, reading register more frequently won't produce more granular readings)
• device information is registered correctly every time
• device shows up in VRM
• readings for all phases are shown on the main page

Some caveats:

• Shelly Pro3EM uses input registers for everything, not holding registers
• Firmware version, device name, generic information, none of that is available via Modbus (at least for now)
• Energy readings (forward, reverse) are not available from the device itself
• Device probing is very rudimentary and may need further improvements
• Device name gets BEU added to the end for some unknown reason. Might be that we need a custom method for reading strings from these devices. This is apparently correct, see e.g. https://device.report/allterco-robotics/spem-003cebeu
• Error codes/alarms not yet supported

Would be great to get some feedback on this!

Screenshots:

[Jalle19]

Implemented energy readings

[Jalle19]

Doesn't seem to make any sense to implement /ErrorCode, it's hard to test as well since you need to introduce a serious fault on purpose

[Jalle19]

@mansr anything I can do to get this merged?

[Jalle19]

pinging @mpvader as well

[xsven80x]

Hi Jalle19,

I would like to try your script, unfortunately I have no idea how to install the script.

Do you perhaps have a few pointers for me here? What do I have to do to install your script on my Venus OS? Are configurations necessary?

Or did I just miss these tips?

Thank you in advance!!

greeting
Sven

[amadirnjul]

This would be great to test, but please add som details about how to implement this

[Jalle19]

I'll have to rebase this branch onto the latest changes, I can write a short guide after that

[Jalle19]

@xsven80x @amadirnjul here are instructions on how to test this:

Updated 2024-04-05

1. Make sure you're on VenusOS 3.30 (the newest version as of this writing)
2. SSH to your VenusOS device and run the follow commands:

mount -o remount,rw /
cd /opt/victronenergy/dbus-modbus-client
cp device.py device.py.bak
cp dbus-modbus-client.py dbus-modbus-client.py.bak
cd /tmp
wget https://codeload.github.com/Jalle19/dbus-modbus-client/zip/refs/heads/shelly-pro3em -O shelly-pro3em.zip
unzip shelly-pro3em.zip
cd dbus-modbus-client-shelly-pro3em
cp shelly.py device.py dbus-modbus-client.py /opt/victronenergy/dbus-modbus-client

3. Reboot your device
4. Go to Settings -> Modbus TCP/UDP devices -> Saved devices -> Add, enter the IP address of your Shelly Pro3EM.
5. That's it, the device should now be visible in the device list and working as expected

[amadirnjul]

That’s awesome. Can this also be used for the Shelly Plus 1PM Mini, as well?

[Jalle19]

@amadirnjul no, since it doesn't support Modbus

[amadirnjul]

I have two Shelly's installed in my sailboat. I thought it was the Plus 1PM mini and the Pro 3EM, but the last one is the Pro 1PM. This does also not support Modbus as far as I know. I better make the electrician to swap it out or I also might dig into the Pro 1 PM support for the Venus. Anyway, thanks alot

[Jalle19]

It would be fairly trivial to make a Modbus TCP proxy for various Shelly's that would expose them as a supported Shelly energy meter (currently only the Pro3EM since it's the only device with Modbus support).

Would be nice if someone who maintains this repository could comment here on the status of this pull request, @mansr ?

[Jalle19]

Rebased on top of the latest master (used by VenusOS 3.30).

I'm really disappointed with Victron Energy for not responding at all to this pull request. @mansr keeps releasing new versions while completely ignoring all pull requests here. Rebasing after every release is painstaking and error-prone, especially when the lower levels of the project are in constant motion.

I would really appreciate some feedback on this change.

[Jalle19]

Connection seems wonkier than with VenusOS 3.14, not really sure what's going on.
Turns out the issue was that I had two instances polling the same meter - the meter can barely keep up with a single consumer

[mpvader]

Hi, Matthijs here from Victron. The decision to includes this or not lies with me. I don’t have time to look into this the coming 2 weeks but will look into it thereafter.

One concern I have is that by making things work out of the box, customers also expect it to work out of the box and turn to us if it doesn’t.

and while shelly support is nice, we already support lots of energy meters and more and more types doesn’t bring extra functionality for users. However I also see the advantages..

Talk to you later! Matthijs

[Jalle19]

@mpvader thanks for getting back to me!

I appreciate your concerns about maintainability, however, most of the implementation is the same regardless of energy meter - it's just the particular registers that differ. Up until quite recently there was no genuine Victron-branded energy meter on the market, so to me it seemed logical to add support for new third-party meters.

Now that there's an official (and most likely superior, technically) product these third-party meters could perhaps me supported in the sense that the software supports them, but there is no guarantee that it will work.

Either way, Shelly products are quite popular and I believe this addition would greatly benefit the community.

Looking forward to further feedback from you.

[Jalle19]

Been using this in a three-phase ESS setup for over a week now without any issues. @mpvader have you had time to consider this PR?

[Marv2190]

Would be nice to have this added.