Add function to connect reed contact as a counter for reading out a gas meter

[1coderookie]

Hey guys,
there is a user who is actually trying to connect a reed contact and the belonging custom code to BSB-LAN to read out his gas meter. He isn't really a hardcore programmer though, so he got stuck at certain points. Taking in account that more users are asking me about the internal function 'gas energy meter' (see below) recently that in the past, I thought this might be a good function which could be added to BSB-LAN in general.

In general:

• We already have the opportunity to connect relays, buttons etc to BSB-LAN an control them or even use the setup itself as a room unit.
• Within certain LMS series (afaik LMS14 since SW 4.2 and LMS15 since SW 3.8) there is the internal funtion "gas energy metering", parameters 2550 (activation) & 2551 (adjustment). Parameters 8378-8383 will then display the values. However, this function only is available within the mentioned controller types and (afaik) only supported within Brötje sytems and it's deactivated by default.
• Gas meters (only the ones which support it of course) can be read out using a reed contact. It's placed at a certain position of the meter and then counts every 0,01m³ (iirc) by closing the contact. There aren't much more of electronic components needed, maybe an additional resistor or so and of course one have to pay attention to the debouncing if connected to an Arduino/ESP.

So my thoughts about it:

• Just as with the buttons and relays we offer the possibility to connect a reed contact to a pin of the Arduino/ESP and activate it via (web)config. Therefore we would need a "counter" function in first place.
• Then the user has to type in his value from the gas meter (let's say 1234,56m³) for adjustment and the basic start-setting of the counter function. It would be good if this parameter could be adjusted via runtime also (so without the need of flashing the whole software again), so that (for example after) this value could be adjusted after an update while the gas burner was running and gas was used.
• Then BSB-LAN counts the reed contact (closed) and sums up internally every registered closed-contact as 0,01m³ (maybe it would be smart to also be able to set this value, just in case some gas meters differ here).
• Then, as a special parameter, one could add this parameter to the list of the logging parameters and the measured/calculated value could be logged on SD card, sent via UDP/MQTT etc in the given log interval.
  For this we maybe could use the function of the burner runtime or burner start calculation function, which also sums up the value(s) and can be logged/sent within the desired log interval - but that's just an idea from my non-programmer mind, so I don't know if this really is already something we/you could use as a template for this new function or not.. ;)

Of course one could use an extra setup for this, for example using an ESP8266 with Tasmota. But I think especially for users who don't use an extra home automation software, who don't want to use extra hardware (which also uses electricity again) and/or who are using BSB-LAN in terms of optimizing the whole heating system this additional feature of BSB-LAN could/would be very nice to have. Especially in these times where the prices are exploding and where more and more users seem to care about the amount of gas they are using.

So @fredlcore and @dukess - what do you guys think about it?

[fredlcore]

I think it's a nice idea in general - what kind of gas meters will this work on? I have one of the "classic" ones still, so probably I won't be able to test it, right?
Other than that, since gas heaters are just one of several heating systems supported by BSB-LAN, I would suggest we first implement this using _custom.h. Since we already have user-definable parameter numbers which can be set and queried, the whole thing could then be shared with others and at the same time would not generally consume memory on those installations which simply do not have this specific kind of gas meter (or no gas at all). In that respect I see this different compared to relays, buttons etc. which can be used on all BSB-LAN settings.
The big question is what it means that he got stuck at certain points. I can certainly give him pointers on how to implement this, but I guess I won't have the time to implement this myself in the near future.

[1coderookie]

Gas meters: I don't know if it's printed onto them somewhere or if you'd have to check the manual of it to see if it supports the reed contact - probably the same as with the water meters, where the really old ones don't support it but the newer ones mostly have it already. If the manufacturer offers some kind of a solution to add a certain peace of reading-out-hardware, it's most likely supported then. Maybe send me a photo your meter and I can check if I find something for your model.

Implementing vs custom code:
Well, looking at the fact that the biggest amount of users seem to use gas heating systems, I think it would be a nice feature to have implemented in BSB-LAN in general and I could imagine that more users would use this function than probably adding a relay or a button. But anyway - maybe you could do it in a way that it can be activated/deactivated like with several functions in the past where Mega-users were able to include/exclude functions from compiling? In that way the users who want to use it won't have to deal with the custom code part..

I think he got stuck at the point how he can get (cuz of the lack of a counter function), sum and transmit the value, also he didn't yet manage to add the function to type in the given value present at the meter and so on. So based on the fact that we already have most of this kind of functionality already (see above), I thought it would be smart to ask you guys if you maybe could take care of that so he doesn't have to re-invent the wheel which would lead to bigger (and probably cluttered) code.. ;)
I googled a bit about these kind of projects for arduino in general and (talking about the counter functionality) it doesn't seem to be a lot of code, so at the end I think it wouldn't really be that memory consuming.

Let's see what @dukess says about it, maybe he can take care about it since the relay is somewhat like a button (closed contact) with the necessity of having a counter function for it (and the above mentioned additional features)..?

[dukess]

hi, this idea is interesting. I read somewhere about the implementation of counters. the main technical problem, it seems to me, is the need to constantly save the state of the counter in case of a reboot or power outage, which requires regular overwriting of eeprom cells. we can cache values in memory and write only integer values, but in this case errors can accumulate quite quickly. "U. D." ***@***.***> 9 октября 2022 г. 19:53:42 написал:

…

Gas meters: I don't know if it's printed onto them somewhere or if you'd have to check the manual of it to see if it supports the reed contact - probably the same as with the water meters, where the really old ones don't support it but the newer ones mostly have it already. If the manufacturer offers some kind of a solution to add a certain peace of reading-out-hardware, it's most likely supported then. Maybe send me a photo your meter and I can check if I find something for your model. Implementing vs custom code: Well, looking at the fact that the biggest amount of users seem to use gas heating systems, I think it would be a nice feature to have implemented in BSB-LAN in general and I could imagine that more users would use this function than probably adding a relay or a button. But anyway - maybe you could do it in a way that it can be activated/deactivated like with several functions in the past where Mega-users were able to include/exclude functions from compiling? In that way the users who want to use it won't have to deal with the custom code part..I think he got stuck at the point how he can get (cuz of the lack of a counter function), sum and transmit the value, also he didn't yet manage to add the function to type in the given value present at the meter and so on. So based on the fact that we already have most of this kind of functionality already (see above), I thought it would be smart to ask you guys if you maybe could take care of that so he doesn't have to re-invent the wheel which would lead to bigger (and probably cluttered) code.. ;) I googled a bit about these kind of projects for arduino in general and (talking about the counter functionality) it doesn't seem to be a lot of code, so at the end I think it wouldn't really be that memory consuming. Let's see what @dukess says about it, maybe he can take care about it since the relay is somewhat like a button (closed contact) with the necessity of having a counter function for it (and the above mentioned additional features)..? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[fredlcore]

Just a quick note upfront: Please don't start working on this issue right now everybody. Currently, we have other issues we need to deal with first to get the several major changes we're working on bug-free :).

Maybe gas heaters are the majority, but if you look at the microcontrollers most of the people using, the number of people who can effectively use this without a significant effort, the number becomes much smaller:
About 1/3 of the users use the Olimex which hardly has any free pins available. And even if you want to use them, people would have to solder the wires to the connector because there are no pin headers attached. The same goes for the NodeMCU where our adapter plugs into all the pins. You can use sensors more easily because there are still a lot of them free, but again, you'd have to solder them yourself. Only with the Arduino (which makes up less than 1/3 of new installations nowadays) you could do it solderless. So even if 2/3 of the user base use gas heaters, the number who have a gas heater and could use it without much of a hassle is at best 1/3, most likely less.

Projects like this which caters to a (more or less) special interest group is exactly one of the reasons why we introduced the custom code. I don't know what you mean with "In that way the users who want to use it won't have to deal with the custom code part", @1coderookie, because if someone/we comes up with a working _custom.h (and maybe custom_setup.h etc.), all you'd need to do is copy these files to the code directory, compile and run it. What's the trouble with that?
Plus, we wanted to get away from the fact that users would have to deal with enabling/disabling features via definements. We want to reduce these definements rather than create more of them ;)...

Therefore I see not enough benefits to include this into the main code and even if his code is cluttered, it won't harm the main code if it comes as a _custom.h "plugin".

Regarding his programming issues (in case he reads this here): He can just use one of the custom_float or custom_long variables. These can be set straight away from the webinterface, so he could just enter his current gas meter value there and then just increment for example the variable custom_longs[0] accordingly with each impulse. Given that the library is supposed to be easy to include, this could be done with a few dozen lines of code, leaving out debouncing, storing to SD card etc.

But as mentioned above, we have more pressing issues right now. If he has something to show and needs help, I'd be happy to do so, but do all this from scratch is not possible for me at the moment...

[1coderookie]

@fredlcore : Whatever you guys think would be the best way to integrate it would be fine I guess..

@dukess : I'm not sure if I understood what you meant by "the need to constantly save the state of the counter". From what I know about how counters work in general is that they registrate the changed state from "normally open" to "open-closed-open again" which then counts as 1 count. The count itself is triggered by the changed state back to "open" from previously "closed" to avoid a constant counting during the moment that its closed, just in case that the meter stops at a position where the reed contact would be closed.
"we can cache values in memory and write only integer values, but in this case errors can accumulate quite quickly." -> and for being able to correct this error it would be good to have the possibility to adjust the value within BSB-LAN during runtime I think - right?

Btw: Another use case for a counter function like this would be the possibility to connect an inductive proximity sensor (instead of or -even better- in addition to a reed contact), so one could also use this function to read out a water meter (which often is located at the same place down in the basement where the heater is).
I built this for my own water meter using a solution from "MySensors", but there I have to use an additional home automation system again for getting the values which kinda sucks for me cuz my FHEM stopped working again and I have to have an extra server running for this also, which is the reason I never managed to set it up again (besides the fact that both units consume electric energy again and I don't really need an home automation system besides this kind of stuff).. ;)

[fredlcore]

From what I meant with "counter" (and I guess @dukess, too), is that you have to save the current gas counter value (e.g. "3214,643 m3" as displayed on the gas meter somewhere in the EEPROM regularly so that the counter can continue from there in case of a reboot/reset. Otherwise it would just start from zero, and then it's not so much use ;).

[1coderookie]

Exactly, that's what I meant with my second point in the initial post at 'my thoughts about it':
"Then the user has to type in his value from the gas meter (let's say 1234,56m³) for adjustment and the basic start-setting of the counter function. It would be good if this parameter could be adjusted via runtime also (so without the need of flashing the whole software again), so that (for example after) this value could be adjusted after an update while the gas burner was running and gas was used."
But I don't think it would be necessary to continously write the new values to the EEPROM - in case of a blackout or so the user then would have to type in the new actual value of the meter.

[fredlcore]

Yes, I read that, but do you really mean that you want the user to run down to the basement after each reboot (which btw also occurs scheduled upon changing some of the settings that require a restart)? I would not find that really convenient, and why should we do it differently from storing the avereage values on SD-card/EEPROM across a reboot?

[dukess]

I mean "should to write new value to eeprom on every change" "U. D." ***@***.***> 10 октября 2022 г. 13:43:57 написал:

…

@fredlcore : Whatever you guys think would be the best way to integrate it would be fine I guess.. @dukess : I'm not sure if I understood what you meant by "the need to constantly save the state of the counter". From what I know about how counters work in general is that they registrate the changed state from "normally open" to "open-closed-open again" which then counts as 1 count. The count itself is triggered by the changed state back to "open" from previously "closed" to avoid a constant counting during the moment that its closed, just in case that the meter stops at a position where the reed contact would be closed. "we can cache values in memory and write only integer values, but in this case errors can accumulate quite quickly." -> and for being able to correct this error it would be good to have the possibility to adjust the value within BSB-LAN during runtime I think - right?Btw: Another use case for a counter function like this would be the possibility to connect an inductive proximity sensor (instead of or -even better- in addition to a reed contact), so one could also use this function to read out a water meter (which often is located at the same place down in the basement where the heater is). I built this for my own water meter using a solution from "MySensors", but there I have to use an additional home automation system again for getting the values which kinda sucks for me cuz my FHEM stopped working again and I have to have an extra server running for this also, which is the reason I never managed to set it up again (besides the fact that both units consume electric energy again and I don't really need an home automation system besides this kind of stuff).. ;)— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[1coderookie]

I as a not-programmer don't really know how you guys do that stuff with the averages or the burner runtime for example in detail. But exactly that kind of functionality/procedure I had in mind when I wrote: "For this we maybe could use the function of the burner runtime or burner start calculation function, which also sums up the value(s) and can be logged/sent within the desired log interval - but that's just an idea from my non-programmer mind, so I don't know if this really is already something we/you could use as a template for this new function or not.. ;)" - I just forgot to mention the averages functionality, but for me it's more or less the same kind of procedure. So: no, of course it wouldn't be convenient if one has to adjust the value after each reboot - but if the updated value after each count or in a given interval has to be written to the EEPROM (which could lead to faster wearout I guess) continously or if you handle it like averages with a values.txt-file or whatsoever I don't know of course.. ;)

[fredlcore]

Burner runtime and averages are different things: One is not kept across reboots (burner runtime and all the others between 20000 and 20005), the other one (averages) is stored on EEPROM or SD-card for keeping it across reboots. Storing a value across reboots is not the same as logging a value to a logfile, even if it uses the same storage medium. Because you have to keep track of every change and not just store the value every x minutes or so. Otherwise you will have a significant drift in no time.

So if someone wants to implement this function, one has to choose which way to go, so it's not so much about how to do it as to whether its deemed necessary or not to keep the values across a reboot. And if yes, where the values should be stored on (not at all/SD-Card/EEPROM)

As you correctly say, writing constantly to the EEPROM will wear the EEPROM out sooner or later. That's not so much a problem with SD cards which are much larger than the EEPROM storage, so wear-levelling will be much more efficient. Another notable difference compared to the average calculation is that if you just choose not to write the averages to SD card or flash, the worst that will happen is that the averages will calculated without those values collected during the 24 hours before the restart. After that, it won't be noticeable again.
That is different to such a meter value where it is essential that the absolute value is kept across reboots.

But don't get me wrong, all of these aspects are not preventing us to do this function. However, taking all these aspects into consideration, I would advise standard NodeMCU users (without an external SD card reader) against using such a function because it will certainly wear their EEPROMs sooner than later. (And just for clarity: If we add a function to omit storing to EEPROM, then users would have to constantly keep an eye on the value to check if a reboot has happened.)

[1coderookie]

Thanks for that explanation! :)

The problem with the EEPROM wear out leads me to the question though, if it might be better to just transmit every count directly via MQTT for example without saving any summed value in the EEPROM and leaving the whole 'summing up' topic to the home automation system. Sure, the downside of this would be the necessity for a home automation system and for example the usage of MQTT, so users who don't have that wouldn't be able to use this new function, but at least we could avoid the problem with the wear out.
But that's just a thought - if you guys can really integrate everything so it could be used even with only having the BSB-LAN setup and still being on the 'safe side' somehow regarding the wear out problem, then that would be better of course.. ;)

[fredlcore]

Yes, that's the reason why I no longer use the averages feature from BSB-LAN. It is of course more reliable to let the home automation system do all the logging and calculation. But the same applies to logging on SD card in general. It's just a very simple alternative to having the logging done on a home automation system (and also the reason I don't use SD card logging anymore). But then you can contiune this thought to the question of integrating such additional features as well because these sensors could also be done from the home automation system as well etc.

I nevertheless think it's good to have these functions as part of BSB-LAN because it lowers the bar to get started, and you're less dependent on other systems (such as maintaining a Raspberry Pi and the software on it). But if you want to go that way, you should be aware that some of the microcontrollers are less ideal than others in our setup. The NodeMCU is great if you have a home automation system and/or don't need (long-term) logging on the device. The Arduino and the Olimex bring the SD card with it, so they're much more suitable for such a task, albeit more expensive and the Due without WiFi. So that's why people should think about what they want and need and then pick the most suitable device for running BSB-LAN...

[DE-cr]

Personally, I would not save such a counter across reboots, and also would not include a user supplied base value in it:

1. Providing an absolute value suggests reliability that I think cannot be guaranteed with such a setup, e.g. due to user error in supplying the base value, reed switch missing some events, BSB-LAN hardware offline for an extended period of time, ...
2. Providing just a counter of BSB-LAN registered reed events after startup, a BSB-LAN reboot would be easy to notice: the counter goes back to zero (if implemented reasonably).
3. How frequent are BSB-LAN reboots, anyway? With my setup, I've only noticed reboots on firmware updates that I triggered myself.

Btw, as much as I would love to have a sensor counting revolutions of the conveyor screw feeding pellets to my boiler (as an approximation of the fuel consumption), I wouldn't expect such a functionality in a general purpose BSB product, especially not when it requires additional hardware. To me, custom code (which can be shared) seems to be the best place for that.

[DE-cr]

P.S.:
4. Lacking an SD card, frequent saves of such a value would wear out the EEPROM, as already mentioned.

[fredlcore]

Just one quick thing about reboots: BSB-LAN reboots upon changing values in the "settings" menu. Not with all of them, but with quite a few. One probably doesn't change these settings all too often, but if you do, you may not realize that you lose certain calculated values.