How to cope with schedule in thermostat?

[andriej]

It was the easiest to describe the issue by sharing the gif-video:

As you can see, while testing the OTGW integration, I've set temperature to 22.
Meanwhile the schedule from iSense thermostat kicked in and set the temperature for 24,5 and started heating.

Right now the situation with 'jumping' value will remain until the value will go to previous value.
It will make those values be recorded (database getting bigger too) and mess with the chart a little.

The 'jump' happens every time value from thermostat is reported to HA (so like every ~20-30 seconds)

[andriej]

Basically current setpoint/heating temperature is set on termostat schedule and it's 24,5.
22 is value from HA. I'd like to see same values in both places. :-)

("Grzanie" is heating, "obecnie" current room temp.)

[andriej]

@mvn23 any ideas how I can deal with setting the data either on termostat (wall-mounted) and in HA so they are showing the same and acting the same without the 'jumps' recorded in the gif?

[mvn23]

Like #4, this is an issue with the HA integration, not with pyotgw. Please report it as such (in the home-assistant repo) and maybe @mention me there.

[andriej]

I'm not so sure if it's only HA side, as it's component not pushing some specific parameter and termostat still sends the 'old' value?
Which one of parameters TC or TT sent by pyotgw? Maybe it's the cause.

[mvn23]

pytogw supports both TT and TC. The HA integration sends TT by default, so it should allow for programmed changes. Please show me what pyotgw is sending to HA by running the usage example when this is happening and posting the output here. Change PORT to wherever you have connected your OpenTherm Gateway.

edit: make sure HA is not using the port while running the script.

[andriej]

@mvn23 here you go - HA right now shows '22' (as I was lowering the temperature during the day) and schedule from thermostat is currently while heating phase (meaning 24.5)

[mvn23]

OK, it seems there's a conflict between the otgw and the thermostat. The room_setpoint_ovrd disappears and then reappears between messages. To see what's happening I would need a log of the raw data between the lib and the gateway. Can you intercept the raw communication somehow? If not, I can make a branch with raw logging to facilitate this.

[andriej]

I can. For how long do you need the dump with 'raw'? 1-2 minutes will be enough?

[mvn23]

More than enough. Probably 10 seconds will do the trick, but make it ~30 to be sure.

[andriej]

Raw dump for like less than minute for sure. Is the format enough?
If you need anything more, please drop me an info.

[mvn23]

The problem here is that the thermostat is changing the room setpoint (data id 16) but the gateway does not clear the room setpoint override (data id 9).
The gateway does report the override mode correctly - both manual and programmed changes are allowed.
In which of the following situations does the problem occur?

• Gateway reset, manual change of room setpoint on thermostat.
• Gateway reset, thermostat programmed change of room setpoint.
• Gateway reset, manual change of room setpoint on thermostat, then change of room setpoint on the gateway (or HA).
• Gateway reset, thermostat programmed change of room setpoint, then change of room setpoint on the gateway (or HA).
• Gateway reset, change of room setpoint on the gateway (or HA), then manual change on thermostat.
• Gateway reset, change of room setpoint on the gateway (or HA), then programmed change on thermostat.

[andriej]

I will test those scenarios probably today's evening. Will get back to you, but what I remember is it fits scenario when: I set setpoint in HA and then, some time later, schedule kicks in.
I'm almost sure that py-otgw-mqtt which I was using earlier didn't had such issue (it's just the clarification that it may be solvable problem).

[andriej]

@mvn23 Way to reproduce:

• reset gw -> temperature changed on thermostat (i.e. from 22.5 to 23) -> temperature change via UI (i.e. from 23 to 21) -> temperature changed on thermostat [manually or scheduled] (i.e. from 21 to 22)

The value will jump between 21 and 22 like in gif in 1st post.

[mvn23]

OK, just one more thing to diagnose the exact cause of the issue then: can you provide a raw dump during a manual (or programmed) change on the thermostat after a change via the gateway? Make it ~30s both before and after the change of the thermostat.

[andriej]

No problem, just wondering - how should I do this? Reset via HA, stop HA, dump RAW, start HA, change temp, stop HA, dump raw further, change from thermostat?

[mvn23]

Try this:

• Reset via HA
• Change temp in HA
• Stop HA
• Start dump
• Wait 30s
• Make manual change
• Wait 30s
• Stop dump
  That should be enough to work with.

[andriej]

reset -> changed temp in HA from 22.5 (default) to 21.0 -> stopped HA -> [begin dump, 30s] -> changed thermostat to 22 -> [30s, end dump]
and after HA restart the issue remains (21 showing, '22' blinking for nanosecond)

link to the dump

[mvn23]

I notice the following in the dumps you provided:

The cause of the quick jumps is that your boiler responds to message id 9 (room setpoint override) with a value of 0 (override disabled), but then the gateway sends the current override value to the thermostat. This part can be fixed as the value from the boiler can be ignored while the override is active on the gateway.

The gateway does not seem to respond to a change in setpoint made on the thermostat while the override is active. In my setup, once the room setpoint (message id 16) is no longer equal to the room setpoint override (9), the gateway cancels the override and responds to future requests for this override with value 0. In your case it keeps on sending the override value even after the room setpoint was changed on the thermostat.
My best guess for the cause of this is that the gateway is trying to work around some quirks of the iSense thermostat. There's an old discussion from 2012 about the bugginess of the iSense thermostats on domoticaforum and some changes were made to the opentherm gateway firmware since to try to deal with it. Which firmware version does your thermostat use? It may not need the fixes that were applied. Try playing with the FT command on the gateway to see if that changes anything.

As far as the library is concerned: apart from the quick jumps it shows exactly what the current state of the gateway is when the problem occurs. The gateway still considers the override active, so the library provides an override value in its status dict.

[mvn23]

The cause of the quick jumps is that your boiler responds to message id 9 (room setpoint override) with a value of 0 (override disabled), but then the gateway sends the current override value to the thermostat. This part can be fixed as the value from the boiler can be ignored while the override is active on the gateway.

Should be fixed in a051dc5

[andriej]

Ok, I've just set the parameter FT from iSense to 'general' (FT=D).
Now the behaviour is more normal, but ends in strange way.
Temperature set on UI -> is visible on thermostat -> changed on thermostat is visible in UI and no jumps. But if right now it will be changed in UI, both thermostat and UI will 'reset' itself back to 'standard' temperature set in schedule.
On the thermostat panel I see the 'hand' meaning manual override (UI or device itself) disappearing.

[andriej]

So from scheduled 22,5 -> UI set to 21 -> thermostat change to 21.5 [till now everything is ok] -> UI change to 21 and wait few seconds -> it resets back to 22.5

[mvn23]

What is the response to command PR=O when the original problem occurs?

[andriej]

I will be able to use OTMonitor in the evening to check that. What I have read on thread you've mentioned is the 'iSense' hack parameter changes behaviour of the overriding this value.

[andriej]

@mvn23 Ok, just did back the steps to reproduce and:

19:34:15.255050	T00090000	Read-Data 	Remote override room setpoint: 0.00
19:34:15.378514	BC0090000	Read-Ack  	Remote override room setpoint: 0.00
19:34:15.397300	AC0091400	Read-Ack  	Remote override room setpoint: 20.00
19:34:16.255906	T00630000	Read-Data 	Message ID 99: 0
19:34:16.367743	BC0630000	Read-Ack  	Message ID 99: 0
19:34:16.795008	Command: PR=O
19:34:16.826660	PR: O=N

[mvn23]

Right, so the messages to the thermostat don't match with the internal state of the gateway when it's using the workaround for the iSense. I can make a workaround for this case, but it will be slower than usual.

[mvn23]

Can you try the isense_quirks branch?

[andriej]

@mvn23: I've changed the protocol.py with the file from isense_quirks branch and restarted HA.
Then did the usual test:

• from scheduled 22,5
• UI set to 21
• thermostat change to 21.5
• UI changed itself to to 21.5 and stood stable (no jumping)
• UI set to 22
• thermostat shows 22 and no jumps in UI
• on thermostat I've clicked 'continue program' [so get back to default 22,5] and in UI it changed to 22,5 with no jumps.

During whole this operation the scheduled value was 21,5 so also iSense didn't kick in with resetting anything like it would when I changed from 'iSense mode' to generic in gateway.

Long story short: it seems fixed!

[mvn23]

Fixed in dac9d51
For future reference, note that this fix only works if the thermostat is detected as an iSense by the gateway. According to the docs this isn't always successful. Use command FT=I to force the gateway into behaving like an iSense thermostat is connected.

[andriej]

Right, of course. Maybe it's worth noticing that in docs.
Thanks for all the effort!

[andriej]

@mvn23 - one thing regarding new code - there is no jumps, but there are some value changes like:

marked in yellow. it's chaning between 22 and 22.5 in random intervals of time.

Didn't see that earlier as it needs some time to happen, but you can clearly see the moment of implementation.

[andriej]

Now scheduled heating kicked in:

[mvn23]

What is the 22 value that it's jumping back to? Manually set on the thermostat, set via HA (override) or something else?
I can't tell from the picture where the values are coming from. Please provide another log from the usage example when the problem occurs.

[andriej]

22 is the continuation from this post - I didn't change anything in meanwhile.
Of course if needed I can provide current 'usage example' log as it's during heating

[mvn23]

I can provide current 'usage example' log

Please do so.

[andriej]

Here it is

[mvn23]

As you can see, only the first few messages contain a false room_setpoint_override. I assume this value keeps disappearing and reappearing from time to time, causing the issues with the target temp jumping in pyotgw.
The iSense quirk fix discussed earlier basically checks the value of otgw_thermostat_detect every time a message with ID 9 (room setpoint override) with a value other than 0 is received from the boiler or sent to the thermostat. If it finds that otgw_thermostat_detect == 'I' (iSense thermostat detected) it will issue a PR=O command to verify the setpoint override value as known in the gateway. The return value of this command determines the value for room_setpoint_ovrd.
As you can see from the above, if the override value is still present in the status dict after the iSense quirks fix has been applied, then that's simply the value that's known in the gateway at that time.
I personally don't see a way to get more accurate information at this point, but suggestions are welcome.

[andriej]

Currently have no real idea. Just wondering how it was done in py-otgw-mqtt, but I think it could be different somehow as there are topics to set and to receive temperatures.
Maybe from your perspective there's any info that could help inside the code...

[mvn23]

Now that I think about it, I implemented the room_setpoint_ovrd usage in the HA platform because of the speed advantages it has - room_setpoint_ovrd is updated before room_setpoint copies the setting.
The room_setpoint value in the status reports seems to be correct in all your logs (could you verify that?), so maybe the HA integration should stop using the room_setpoint_ovrd and use the room_setpoint instead.
Turns out it's a HA integration bug after all ;)

edit: Problem with this solution is that the HA UI will not show the new room setpoint until it is accepted by the thermostat...

edit2: proposed solution here

[andriej]

Just seen the edits, will test them in few hours.

[andriej]

@mvn23 I've tried (the solution from edit2) and:

• after reset 22.5
• UI change to 21 [ok]
• thermostat change to 21.5 [ok, UI ok]
• UI change to 22 - 1 (or max 2 - it was hard to notice, just seconds after change) jumps between 22,5 [base value?] then went straight to showing 22 and is stable on that.
• in meanwhile scheduled 24.5 kicked in and messed the wait I wanted to give the chart to be drawn ;-)
• UI change to 25 from 24.5, still 25 without jumps
• UI change to 21,5 [turn off heating] -> state of heating went off and stable 21,5 in UI.
• opentherm_gw.reset_gateway - made everything 0 [should it? why not unavailable? ;-))
• UI and thermostat/boiler went back to heating with scheduled 24.5

I know I already wrote it once, but: seems to be fixed! :-)

[mvn23]

* UI change to 22 - 1 (or max 2 - it was hard to notice, just seconds after change) jumps between 22,5 [base value?] then went straight to showing 22 and is stable on that.

That should not happen, after a change in HA and a change on the thermostat have been propagated between the thermostat and the gateway the 22.5 should not be present in the system anymore. Were all changes visible on both the thermostat and in HA before making the next change?

* opentherm_gw.reset_gateway - made everything 0 [should it? why not unavailable? ;-))

Upon reset, pyotgw immediately requests the last known values from the gateway like it does when connecting. The gateway reports 0, so the library does the same.

[andriej]

That should not happen, after a change in HA and a change on the thermostat have been propagated between the thermostat and the gateway the 22.5 should not be present in the system anymore. Were all changes visible on both the thermostat and in HA before making the next change?

Yes. I've made it repeat two times now, the 'jump' was short - the value of 22.5 lasted for like 1-2 seconds before going for final one and being stable.
Yes, the current value was the same both UI and thermostat before doing change.
I've recorded video (repeating the same steps I've described) - when I'll make it smaller gif I'll try to upload it just the same as initial.

[andriej]

Here find the record - value 22 is after click (where the movie starts), no further clicks done during the video.
(edit; the click was from 21,5 to 22)
I don't really consider this as something wrong anyway, it lasts for so short that it won't matter.
What I've observed also there were no green marked area during the heating scheduled.

[andriej]

In terms of the 'heating' greened area: HA was drawing it while the heating. When heating stopped, area disappeared:

Except that everything during the day - works as it should, I could accept the value showing for 2-3 seconds.

[mvn23]

I am 99% sure that the jumps are actually reported as such by your thermostat. If that's the case I will leave it as is. Can you confirm by providing a log with protocol.py from the debug_raw branch and the proposed solution earlier in this thread? Please enable debug logging in HA for the following items:

• homeassistant.components.opentherm_gw
• pyotgw.protocol

What I've observed also there were no green marked area during the heating scheduled.

I have seen this as well, but I suspect a graphing issue. If you put your mouse over the graph you will see that the information is still there when the system is heating.

[andriej]

Here find the dump of logfile.
Same 'workflow' -> 22,5 base -> 21 UI -> 21,5 thermostat -> 22 UI [and here 22,5 showed for few seconds] and then again from UI to 21,5 and again 22,5 showed for few seconds.
For me of course it's not the issue, the value is not repeated any more after and is properly changed in both places.

In my opinion it can be released further.

[mvn23]

It's actually caused by the OpenTherm gateway explicitly canceling the setpoint override before setting a new one. This seems to cause the thermostat to revert to its programmed setpoint for a few seconds before the new setpoint override becomes effective. I think I may leave this as-is since it is showing exactly what the thermostat is doing.

[andriej]

That's also what I thought - as it shows only when value is changed from both places.
Anyway - the 'workaround' for iSense works so it can be left 'as is'.

IMO it's ok to be pushed further. Except for the graph issues, whole the rest is more than ok:

Hope to see it in next release. :-)