Generic SML based power meters support #146

qubeck · 2023-03-27T16:15:46Z

Add support for generic reading of SML based power meters by taking OBIS 1.8.0 & 2.8.0 lists for import & export of energy and OBIS 16.7.0 list for signed power value.

To not conflict with any upcoming extensions of the power meter class the power meter source was set to 99 for this extension.

…king OBIS 16.7.1 for power (using mod. SML Parser lib. by olliiiver)

Adminius · 2023-03-28T07:39:26Z

For my undesrtanding, is it for IR Power Merers, like "Volkszähler"?

In general, I'would switch UART-based-PowerMeters, like SML and SDM to software serial.
they don't need high bitrate, 9600-19200 is good possible with SoftwareSerial.
The reason is: I will add support for JK BMS (maybe JBD also), and thoose needs 115200bps UART. In this case only HW Serial ist possible, but no one is free :/

Maybe Victron 19200bps is also possible with SW-Serial. More than one Victron is also not really possible because of UARTs

helgeerbe · 2023-03-28T07:59:37Z

I want to extend viltron interface to use more than one charger and that hard and software serial could be used.

Adminius · 2023-03-28T08:11:41Z

@berni2288 which power meter ID are you using fot HTTP(S)?
I would say, we can change IDs to sort:
0: MQTT
1: HTTP(S)
20: SDM 1ph
21: SDM 3ph
22: SML

something like that, so we can add more PowerMeters in beetween.

berni2288 · 2023-03-28T08:14:09Z

I've taken 3, but I don't really care :) maybe change this to an enum so this can be changed easily at one place.

qubeck · 2023-03-28T08:34:26Z

@Adminius Exactly. It's aimed for powermeters that fan out their data using SML via 9600 8N1 serial - IR signal.
I choosed the OBIS 16.7.0 since the majority of them are supporting it see also:
compatibility list of: "ELV ES-IEC, Stromzählerausleser"

Regarding the usage of the SoftwareSerial: Yes my first intention was also to use this, since only rx is required and the baudrate is low. Unfortunately I got unexpected byte errors from time to time in real operation when hoymiles module was active. But the main reason to go with HardwareSerial for now was that since the update of the new compile settings the EspSoftwareSerial doesn't compile anymore if 'gnu++11' is disabled via 'build_unflags' in platformio.ini.

Adminius · 2023-03-28T08:44:25Z

EspSoftwareSerial doesn't compile anymore if 'gnu++11' is disabled via 'build_unflags' in platformio.ini

yeah... this is what I already found with SDM integration...
There newer version (8.0.1) available, we have to test it

qubeck · 2023-03-28T08:56:22Z

There newer version (8.0.1) available, we have to test it

Thanks for the hint!, Just tried that and it compiles now again. But the unexpected byte problem remains a thing.
One solution would be to keep polling for the next SML byte for the time period the whole SML list is expected to arrive. I'm on it...

Adminius · 2023-03-28T09:16:33Z

I see, SML in the list... D0 would be next possiblity.
1000 impulses/kWh can be also made for some cheap power meters... but it es very inaccurate:
http://ssb22.user.srcf.net/elec/imp.html

qubeck · 2023-03-28T13:54:47Z

The switch to software serial usage is complete. I set its buffer sizes to a value that should make sense. Which in turn reduced the observed SML unexpected state significantly.

Adminius · 2023-03-28T14:04:17Z

@qubeck look at my comments about _powerMeterTotalPower = _powerMeter1Power + _powerMeter2Power + _powerMeter3Power;

qubeck · 2023-03-28T14:30:21Z

@Adminius sorry but I can't see any, at least there are none visible to me. Strange...

Adminius · 2023-03-28T14:51:23Z

you don't need _powerMeterTotalPower = _powerMeter1Power + _powerMeter2Power + _powerMeter3Power;
that's done in float PowerMeterClass::getPowerTotal().
_powerMeterTotalPower isn't used (we have to remove it)

qubeck · 2023-03-28T15:02:15Z

That's why I changes also "float PowerMeterClass::getPowerTotal()" to only return "_powerMeterTotalPower"

And all of the powers are applied source individually in the loop. That way it's much more cleaner.
At least it appears to me. But feel free to change it if you really want.

In SML 16.7.0 case there are also no phase specific powers so putting it in to one of the phase powers to get responded by getPowerTotal() would be a little bit confusing.

Adminius · 2023-03-28T15:10:25Z

That's why I changes also "float PowerMeterClass::getPowerTotal()" to only return "_powerMeterTotalPower"

PowerMeter is from me and I had it this way at the beginning (that's why _powerMeterTotalPower is there).
In my opinion it was better to separate it in dedicated function.
The idea behind it:
Each time if at least one of 3 power values is changed, we caluclute a sum and it is not relavant if need this sum an not.
With functinon, we build a sum only if we really need it (on function call).
I did it, because I tested it with my SmartPI MQTT PowerMerer. It sends all 3 power values each second via MQTT. That means, we build as sum 3 times per second.
With a dedicated function only 1-2 times per 10 second (power limiter + mqtt output)

qubeck · 2023-03-28T15:16:10Z

I see your point, well may be in that case an alternative would be something like:
float PowerMeterClass::getPowerTotal() { return (_powerMeterTotalPower > 0) ? _powerMeterTotalPower : _powerMeter1Power + _powerMeter2Power + _powerMeter3Power; }

And total power would be only applied by meter sources that do not have individual readings.

Adminius · 2023-03-28T15:24:39Z

_powerMeterTotalPower > 0

If you have a real "zero export" ("Nulleinspeisung"), _powerMeterTotalPower = 0 and also _powerMeterTotalPower < 0 are valid values...

qubeck · 2023-03-28T15:36:16Z

Yes, of course... So we need to flag that case by something like:
_powerMeterTotalPowerOnly = true; at init

Also I see that the mqtt subscriptions there are also source specific and need to be turned of in that case.

Adminius · 2023-03-28T15:57:32Z

kW theoretically only... never heard that Power meters in home applications usung kW

But you also see why I'm using first power 1 und other 2 are 0 in single phase SDM implementation. Easiest solution, without flags and so on ;)

qubeck · 2023-03-28T16:04:29Z

Well since it gets also published via mqtt it can become confusing eventually. Especially in the case where powermeeters could have pos. and neg. readings for each phase but do not present that power transparently.

Adminius · 2023-03-28T16:22:49Z

Well, mqtt publishes power1, power2, power3 and powertotal.
power1 should not be power L1, power2 ist not L2 and so on 😉
The number represents power source number and not phase number ;)

…ither the sum of phase powers or explicit total power provided by meter

qubeck · 2023-03-28T20:55:56Z

Should not be power L1 ...L3? Sure they doesn't have to be... but the calls to the SDM lib for the _powerMeter...Power are:
_powerMeter1Power = static_cast<float>(sdm.readVal(SDM_PHASE_1_POWER, _address)); _powerMeter2Power = static_cast<float>(sdm.readVal(SDM_PHASE_2_POWER, _address)); _powerMeter3Power = static_cast<float>(sdm.readVal(SDM_PHASE_3_POWER, _address));

So they are implicit phase power readings. But yeah that can be any power reading of course and is physically determined eventually. In the end they are added up like anyone would do with phase power readings...

Adminius · 2023-03-29T05:48:32Z

In case of SDM this are 3 phases=3 power sources.
In MQTT case, theoretically, it can be up to 3 different powermeters...

…toring

qubeck · 2023-03-29T20:00:53Z

I simplified the code to make it more smoother to integrate into the existing module.

Btw. I observed some interesting behavior at trying to configure zero export with the power limiter. It appears that the inverter internally distributes the applied power limit across all DC channels. e.g. with only 1x producing DC channel (in my case @ HM-600) this results in the power limit gets applied with long delay over time, since only a fraction of the requested power limit becomes active on the actual producing DC channel when power limiter kicks in.

This can be improved by taking the number of actually producing DC channels into account to increase the applied power limit artificially. Just saying... I'll experiment with a slight modification to the power limiter as soon the weather becomes better. But this is of course a topic for an other discussion...

berni2288 · 2023-04-02T11:25:56Z

Btw. I observed some interesting behavior at trying to configure zero export with the power limiter. It appears that the inverter internally distributes the applied power limit across all DC channels. e.g. with only 1x producing DC channel (in my case @ HM-600)

It would be interesting if this PR fixes your problem: #154
Maybe you can try it out :)

madmartin · 2023-04-04T09:08:40Z

@Adminius Exactly. It's aimed for powermeters that fan out their data using SML via 9600 8N1 serial - IR signal. I choosed the OBIS 16.7.0 since the majority of them are supporting it see also: compatibility list of: "ELV ES-IEC, Stromzählerausleser"

Regarding the usage of the SoftwareSerial: Yes my first intention was also to use this, since only rx is required and the baudrate is low. Unfortunately I got unexpected byte errors from time to time in real operation when hoymiles module was active. But the main reason to go with HardwareSerial for now was that since the update of the new compile settings the EspSoftwareSerial doesn't compile anymore if 'gnu++11' is disabled via 'build_unflags' in platformio.ini.

Hi,
my personal opinion is that implementing this into here is absolutely waste of time. You can connect your serial IR reader to an esp8266 or esp32 and use tasmota - this works wonderful and is well maintained.
This needs hardware for less than 10 Euro and IMHO not worth the time to be implemented here.
Read tasmota doc and you see that implementing the SML stuff is a lot of code - so why repeat this here?

https://tasmota.github.io/docs/Smart-Meter-Interface/

Finally, since #153, the powermeterclass retrieves the values from tasmota via http(s).

Adminius · 2023-04-04T09:32:49Z

@madmartin

you need additional ESP8266/ESP32 + power supply
you need stable WiFi connection for both ESPs
With this implementation you can setup one ESP32 as stand alone and all-in-one solution, completely offline.

qubeck · 2023-04-04T20:20:29Z

aligned the code with current dev, with slight change on condition with "config.PowerMeter_Enabled" in onMqttMessage().
added SOURCE_SML (==4)
added explicit return if "!config.PowerMeter_Enabled" and more source dependent init/begin calls to modules in PowerMeterClass::init()

I think now would be the best time for a review and merge before it diverge any further...

berni2288 · 2023-04-05T17:31:13Z

src/PowerMeter.cpp

@@ -15,6 +15,15 @@ PowerMeterClass PowerMeter;

 SDM sdm(Serial2, 9600, NOT_A_PIN, SERIAL_8N1, SDM_RX_PIN, SDM_TX_PIN);

+#define USE_SW_SERIAL


I wonder... what's the point of this constant that can't be overriden via build parameters?

The decision about using the HW/SW serial wasn't made back then. So I left the option. But yeah I will remove it.

berni2288 · 2023-04-05T18:03:26Z

include/PowerMeter.h

+    float _powerMeterImport = 0.0;
+    float _powerMeterExport = 0.0;
+
+    bool _powerMeterOnlyTotalPowerAvailable = false;


I'm not fully sure if that's a good idea. It's inconsistent behavior with the other Power meters where you currently can only set the power per phase.
Also I don't understand why all member variables are starting with _powerMeter? PowerMeter is already part of the class name.

I guess we will need some refactoring of the Power meters, it's becoming a bit messy already (not your fault).

Well, I introduced that flag because SML power reading obtained from OBIS 16.7.0 list can not be related to any specific phase. It's the signed sum of all and I didn't want to confuse anything an put it into one of the phases.
Especially, because that powers are getting published...

…n better with existing code

berni2288 · 2023-04-07T18:17:31Z

webapp/src/views/PowerMeterAdminView.vue

@@ -215,10 +215,11 @@ export default defineComponent({
            dataLoading: true,
            powerMeterConfigList: {} as PowerMeterConfig,
            powerMeterSourceList: [
-            { key: 3, value: this.$t('powermeteradmin.typeHTTP') },


It might not've looked aesthetic in the code, but it was put on top on purpose because typeHTTP is probably the most used one :) But it's ok, we can also order it by number if you would like to.

github-actions · 2024-04-07T04:12:42Z

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new discussion or issue for related concerns.

helgeerbe and others added 2 commits March 27, 2023 10:59

Merge branch 'development'

7ba2058

add support for energy & power readings on SML based power meters, ta…

b5bb86d

…king OBIS 16.7.1 for power (using mod. SML Parser lib. by olliiiver)

qubeck and others added 2 commits March 28, 2023 15:37

switched SML read to use software serial

1bcd465

Merge branch 'development' into generic_sml_powermeter_support

35d9038

qubeck added 2 commits March 28, 2023 18:44

made total power meter response controled by meter source to obtain e…

95a97e2

…ither the sum of phase powers or explicit total power provided by meter

made mqtt subscriptions to power meter topics meter source dependend

6f84c31

qubeck force-pushed the generic_sml_powermeter_support branch from 9b58c28 to 6f84c31 Compare March 28, 2023 16:45

simplified SML read loop and OBIS handler registration, + minor refac…

40cd2a7

…toring

Merge branch 'development' into generic_sml_powermeter_support

e8ea278

berni2288 added the enhancement New feature or request label Apr 2, 2023

qubeck and others added 3 commits April 4, 2023 17:24

minor cleanup/style changes and optim. PowerMeter

260b3ab

Merge branch 'development' into generic_sml_powermeter_support

3de5372

fixed build, add SOURCE_SML == 4

d06196a

qubeck force-pushed the generic_sml_powermeter_support branch from d1640f8 to d06196a Compare April 4, 2023 20:01

berni2288 reviewed Apr 5, 2023

View reviewed changes

qubeck added 3 commits April 6, 2023 22:40

removed optional usage of HW serial for SML power meter

bd9db57

Merge branch 'development' into generic_sml_powermeter_support

86ecc25

switched to usage of _powerMeter1Power for SML power reading to allig…

80e5196

…n better with existing code

berni2288 reviewed Apr 7, 2023

View reviewed changes

berni2288 merged commit 00def1d into helgeerbe:development Apr 7, 2023

github-actions bot locked as resolved and limited conversation to collaborators Apr 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generic SML based power meters support #146

Generic SML based power meters support #146

qubeck commented Mar 27, 2023

Adminius commented Mar 28, 2023

helgeerbe commented Mar 28, 2023

Adminius commented Mar 28, 2023

berni2288 commented Mar 28, 2023

qubeck commented Mar 28, 2023

Adminius commented Mar 28, 2023

qubeck commented Mar 28, 2023

Adminius commented Mar 28, 2023

qubeck commented Mar 28, 2023

Adminius commented Mar 28, 2023

qubeck commented Mar 28, 2023

Adminius commented Mar 28, 2023

qubeck commented Mar 28, 2023

Adminius commented Mar 28, 2023

qubeck commented Mar 28, 2023

Adminius commented Mar 28, 2023

qubeck commented Mar 28, 2023 •

edited

Loading

Adminius commented Mar 28, 2023

qubeck commented Mar 28, 2023

Adminius commented Mar 28, 2023 •

edited

Loading

qubeck commented Mar 28, 2023 •

edited

Loading

Adminius commented Mar 29, 2023

qubeck commented Mar 29, 2023 •

edited

Loading

berni2288 commented Apr 2, 2023

madmartin commented Apr 4, 2023 •

edited

Loading

Adminius commented Apr 4, 2023

qubeck commented Apr 4, 2023

berni2288 Apr 5, 2023

qubeck Apr 5, 2023

berni2288 Apr 5, 2023

qubeck Apr 6, 2023

berni2288 Apr 7, 2023

github-actions bot commented Apr 7, 2024

		@@ -15,6 +15,15 @@ PowerMeterClass PowerMeter;

		SDM sdm(Serial2, 9600, NOT_A_PIN, SERIAL_8N1, SDM_RX_PIN, SDM_TX_PIN);

		#define USE_SW_SERIAL

Generic SML based power meters support #146

Generic SML based power meters support #146

Conversation

qubeck commented Mar 27, 2023

Adminius commented Mar 28, 2023

helgeerbe commented Mar 28, 2023

Adminius commented Mar 28, 2023

berni2288 commented Mar 28, 2023

qubeck commented Mar 28, 2023

Adminius commented Mar 28, 2023

qubeck commented Mar 28, 2023

Adminius commented Mar 28, 2023

qubeck commented Mar 28, 2023

Adminius commented Mar 28, 2023

qubeck commented Mar 28, 2023

Adminius commented Mar 28, 2023

qubeck commented Mar 28, 2023

Adminius commented Mar 28, 2023

qubeck commented Mar 28, 2023

Adminius commented Mar 28, 2023

qubeck commented Mar 28, 2023 • edited Loading

Adminius commented Mar 28, 2023

qubeck commented Mar 28, 2023

Adminius commented Mar 28, 2023 • edited Loading

qubeck commented Mar 28, 2023 • edited Loading

Adminius commented Mar 29, 2023

qubeck commented Mar 29, 2023 • edited Loading

berni2288 commented Apr 2, 2023

madmartin commented Apr 4, 2023 • edited Loading

Adminius commented Apr 4, 2023

qubeck commented Apr 4, 2023

berni2288 Apr 5, 2023

Choose a reason for hiding this comment

qubeck Apr 5, 2023

Choose a reason for hiding this comment

berni2288 Apr 5, 2023

Choose a reason for hiding this comment

qubeck Apr 6, 2023

Choose a reason for hiding this comment

berni2288 Apr 7, 2023

Choose a reason for hiding this comment

github-actions bot commented Apr 7, 2024

qubeck commented Mar 28, 2023 •

edited

Loading

Adminius commented Mar 28, 2023 •

edited

Loading

qubeck commented Mar 28, 2023 •

edited

Loading

qubeck commented Mar 29, 2023 •

edited

Loading

madmartin commented Apr 4, 2023 •

edited

Loading