The 01FF payload is not fully understood

[VAVons]

Don't know if this can be properly named an issue and if this the proper way of contacting you, but I'll do it anyway. Thanks for the great work you guys all did on ramses_rf

I've been analyzing my system with a indalo reciever on a raspberry pi using client.py. The system is a Spider (opentherm) room thermostat (21:033244), an Itho CVE box with built in CO2 and humidity sensor (37:053261) and a Spider gateway (18:010629). This consists of an HGI80 connected to an fifthplay gateway (home area manager) which in turn communicates with an external server from Itho Daalderop . (reason I'm doing this by the way is that itho daalderop will eventually end support, likely ~2024, ending time planning and remote control possibilities)

Using the gateway it is possible to have a weekplanner on the thermostat, I played around with the system through the app, changing the temperature and turning the time planning off and on. The gateway then sends a W|01FF message, the thermostat responds with an I|01FF message. (clearly this 01FF is not recognized by ramses_rf). I also logged the normal changes according to time planning (logged for 20+ hours). Finally, the thermostat sends a RQ|01FF with the same payload every ~9.5 min and gets a RP|01FF response.

I managed to figure out the following about this 01FF message:

Thermostat to gateway message:
057 RQ --- 21:033244 18:010629 --:------ 01FF 026 008026262A90008000143C28400000010480800280FF80070000
In the payload:

• The third byte (26) is the currently displayed temperature x 2 (19 degrees C here)
• Fourth byte (26) the set temperature - can be in/decreased by 0.5 degrees at a time
• Fifth byte (2A) appears to be the max temp
  (because it is consistently the same value as reported in 22C9 setpoint_bounds - temp high)
• The sixth byte determines the state the thermostat is in - 90 when the time planning is active, B0 when the time planning is active, but the temperature was temporarily adjusted, and D0 when the time planning is deactivated

Gateway to thermostat message:
029 RP --- 18:010629 21:033244 --:------ 01FF 026 008080262692000000143C80800000310080800280FF80040000
In the payload:

• Both the fourth and fifth byte appear to denote the temperature setpoint for the thermostat
• The sixth byte determines the state the thermostat is to be in. It is 92 when the time planning is active, B2 when the time planning is active, but the temperature was temporarily adjusted and D2 when the time planning is deactivated.

Did not see any changes in any of the other bytes yet.

Hope you guys have time somewhere in the future to add this 01FF to the code of ramses_rf?

Attached the log of almost 24 hours. Also found a couple of other interesting messages:

• The percentage in 3110 messages always increases when the central heating is on - I think it denotes the modulation of the heater
• I also got these messages on the command line (I logged the command line output):

06:22:19.124  I 125 37:053261 --:------ 37:053261 31D9 017 00863C0E20202020202020202020202000 < Support the development of ramses_rf by reporting this packet (byte 3: 0E)
06:22:23.189  I --- 37:053261 --:------ 37:053261 3120 007 00F0B0010003FF < Support the development of ramses_rf by reporting this packet (byte 1: F0)
06:23:12.928  I --- 37:053261 --:------ 37:053261 3120 007 0070B00000F0FF < Support the development of ramses_rf by reporting this packet (byte 5: F0)
17:25:15.847  I --- 21:033244 63:262142 --:------ 10E0 038 00000100220B0001FEFFFFFFFFFF140107DF4343552D31325432300000000000000000000000 < Support the development of ramses_rf by reporting this packet, with the make/model of device: 21:033244 (device type 21 not known to have signature: 000100220B0001FEFF)

test5.log

[zxdavb]

Thanks for your work - adding a parser for a new code class is straightforward enough, and you have supplied useful information...

The sixth byte determines the state the thermostat is in:

• 90 when the time planning is active,
• B0 when the time planning is active, but the temperature was temporarily adjusted, and
• D0 when the time planning is deactivated

The sixth byte determines the state the thermostat is to be in:

• 92 when the time planning is active,
• B2 when the time planning is active, but the temperature was temporarily adjusted and
• D2 when the time planning is deactivated.

These appear to be state flags:

• x1xx xxxx is inactive, (x0xx xxxx is activate), and
• xx1x xxxx is temporarily adjusted (xx0x xxxx is not)

[zxdavb]

This is what I have (so far):

07:57:07.499 ||  21:033244 |            |  I | ufh_setpoint     |      || {'temp_low': 16.0, 'temp_high': 20.0, '_unknown_0': '01'}
07:57:07.585 ||  21:033244 |  18:010629 | RQ | message_01ff     |      || {'temperature': 18.5, 'setpoint_min': 16.0, 'setpoint_max': 20.0, '_flags': '90'}
07:57:07.639 ||  18:010629 |  21:033244 | RP | message_01ff     |      || {'temperature': None, 'setpoint_min': 16.0, 'setpoint_max': 16.0, '_flags': '92'}

08:00:49.204 ||  18:010629 |  21:033244 |  W | message_01ff     |      || {'temperature': None, 'setpoint_min': 19.0, 'setpoint_max': 19.0, '_flags_10': '92'}
08:00:49.244 ||  21:033244 |  18:010629 |  I | message_01ff     |      || {'temperature': 18.5, 'setpoint_min': 19.0, 'setpoint_max': 21.0, '_flags': '90'}

08:00:53.918 ||  21:033244 |            |  I | ufh_setpoint     |      || {'temp_low': 19.0, 'temp_high': 21.0, '_unknown_0': '01'}
08:06:33.733 ||  21:033244 |  18:010629 | RQ | message_01ff     |      || {'temperature': 18.5, 'setpoint_min': 19.0, 'setpoint_max': 21.0, '_flags': '90'}
08:06:33.788 ||  18:010629 |  21:033244 | RP | message_01ff     |      || {'temperature': None, 'setpoint_min': 19.0, 'setpoint_max': 19.0, '_flags': '92'}

[zxdavb]

And now (sorry, is in a different format):

2023-02-17T14:04:14.517457 ...  W --- 18:010629 21:033244 --:------ 01FF 026 0080802424D2000000143C80800000310080800280FF80040000  # {'temperature': None, 'setpoint_min': 18.0, 'setpoint_max': None, 'time_planning': False, 'temp_adjusted': False, '_flags_10': 'D2'}
2023-02-17T14:04:14.551409 ...  I --- 21:033244 18:010629 --:------ 01FF 026 0080262428D0008000143C28400000010480800280FF80070000  # {'temperature': 19.0, 'setpoint_min': 18.0, 'setpoint_max': 20.0, 'time_planning': False, 'temp_adjusted': False, '_flags_10': 'D0'}

2023-02-17T13:42:25.195749 ... RQ --- 21:033244 18:010629 --:------ 01FF 026 008026262A90008000143C28400000010480800280FF80070000  # {'temperature': 19.0, 'setpoint_min': 19.0, 'setpoint_max': 21.0, 'time_planning': True,  'temp_adjusted': False, '_flags_10': '90'}
2023-02-17T13:42:25.249490 ... RP --- 18:010629 21:033244 --:------ 01FF 026 008080262692000000143C80800000310080800280FF80040000  # {'temperature': None, 'setpoint_min': 19.0, 'setpoint_max': None, 'time_planning': True,  'temp_adjusted': False, '_flags_10': '92'}

2023-02-17T13:43:44.164411 ...  W --- 18:010629 21:033244 --:------ 01FF 026 0080802626B2000000143C80800000310080800280FF80040000  # {'temperature': None, 'setpoint_min': 19.0, 'setpoint_max': None, 'time_planning': True,  'temp_adjusted': True,  '_flags_10': 'B2'}
2023-02-17T13:43:44.203371 ...  I --- 21:033244 18:010629 --:------ 01FF 026 008026262AB0008000143C28400000010480800280FF80070000  # {'temperature': 19.0, 'setpoint_min': 19.0, 'setpoint_max': 21.0, 'time_planning': True,  'temp_adjusted': True,  '_flags_10': 'B0'}

[VAVons]

Great, really fast!

With this it might now be worthwhile to get a home assistant setup to eventually replace the gateway. I'm not so familiar with home assistant, would it be possible now not only to read the thermostat, but eventually control it by configuring it so that it sends setpoints using this message? (if so I'll probably figure out how to do it, but I'd like to know if it's possible before getting into installing and exploring HA)

I'm thinking about eventually taking the HGI80 (so the 18:010629) and connect it to a raspberry pi instead of to the fifthplay gateway, to replace the gateway and keep all control local. Do I then need to bind it again to the CVE and the spider thermostat, or is it still bound, even if connected to another controller? In other words, do you know if the binding is to the RF transmitter or the controller behind it?

Probably more people might benefit from this, as I believe Itho Daalderop will not continue the support for the gateway very long. Only problem is that they will not all have the separate HGI80, as a different transmitter (probably the CCU-12T20) was eventually integrated into newer versions of the gateway. For their benefit, can an indalotech or other RF transmitter also be bound to the thermostat and CVE?

Thanks again for the (fast!) work.
P.S. let me know if there is any other (better) place to ask all these questions

[zxdavb]

This is the forum for you: https://community.home-assistant.io/t/itho-orco-nuaire-fan-metrics-remote-control-sensor-faking-via-rf/451296/

... but it is relatively inactive, so maybe here is better:
https://community.home-assistant.io/t/honeywell-ch-dhw-via-rf-evohome-sundial-hometronics-chronotherm/151584

OK - dirty secret reveal...

ramses_rf is actually two distinct pieces of code merged together - a bit like (say) TCP/IP merged with sockets.

The 'lower' layer (a la TCP/IP) is what we've been fiddling with here: it is concerned with validating and decoding stateless packets. As of now (I just pushed a commit), ramses_rf has sufficient support for the 01FF packet for your needs.

But it is the the 'upper' layer (a la sockets) deals with devices (like your spider gateway) and maintaining a state for these things.

[ For completeness, evofw3 (a la WiFi) sits under ramses_rf, and ramses_cc (i.e. HA, a la web server) sits on top. ]

Presently, ramses_rf struggles with HVAC (it's not as consistent as CH/DHW) and has little support for Spider gateways - it treats them like generic devices.

So there is much work to do at the upper layer to support Spider gateways. Spider thermostats are better supported, as thermostats, but what you probably want to do is:

• set it up as a remote - then you can send packets, or
• set it up a thermostat, and use the send_packet service call

You've got an evofw3 device, so start playing - ask on the forum for further help.

If I see the need to add functionality to the upper later, we can do that.

[zxdavb]

With this it might now be worthwhile to get a home assistant setup to eventually replace the gateway. I'm not so familiar with home assistant, would it be possible now not only to read the thermostat, but eventually control it by configuring it so that it sends setpoints using this message?

Is possible. The real question is: how to do it.

I'm thinking about eventually taking the HGI80 (so the 18:010629) and connect it to a raspberry pi instead of to the fifthplay gateway, to replace the [spider] gateway and keep all control local. Do I then need to bind it again to the CVE and the spider thermostat, or is it still bound, even if connected to another controller? In other words, do you know if the binding is to the RF transmitter or the controller behind it?

What you need to do is to re-bind everything and capture it in packet logs.

Please don't say 'it' - be ore exact.

Probably more people might benefit from this, as I believe Itho Daalderop will not continue the support for the gateway very long.

Someone else has said this, before.

Only problem is that they will not all have the separate HGI80, as a different transmitter (probably the CCU-12T20) was eventually integrated into newer versions of the gateway.

Yes, I've only seen the RF/Internet gateway and controller as one unit.

For their benefit, can an indalotech or other RF transmitter also be bound to the thermostat and CVE?

Yes, any evofw3-compatible device can do full impersonation.

(The indalotech guy is (essentially) the author of evofw3)

[zxdavb]

@VAVons Can you provide some more 24-72h packet logs for me to review?

[VAVons]

@zxdavb I still have no home assistant (didn't have the time/urge yet to figure that out), I just have the CLI utility running indefinitely on my raspi with

nohup python client_2.py -c configuration.json listen /dev/ttyACM0 &

so no log files from HA.
What I do have for you is any errors and strange messages, which are still being registered in nohup.out, see attached file (had to add the .log extension to allow me to upload it).
nohup.out.log

client_2.py is slightly modified version of your client.py just to write the (temperature, CO2, fan speed, mode etc) data in a sqlite database, which I then poll for my own home built website. Although you're welcome to that database data, it is simply the temperature, humidity, CO2 etc. in our house, so not much use to you. But the CLI utility has been running without fail since 28-03-2023, and data is being registered still. The Ramses_rf version dates from ~11-03-2023 (when I downloaded it from Github).

I have to say I hadn't looked at the files for a while, didn't realize there were so many errors registered, otherwise I would have let you know earlier. Quite a few error messages appear to be due to my sloppy programming in client_2.py. You're welcome to that file too of course, although I don't think you want 680 lines of code here in this comment, where do I best post this?

To be honest, although Itho Daaldrop said that they would stop supporting the thermostat (which was the main reason for me to dive into this) it has been running still all winter, and since I'm working on a different heating system altogether (switching to a Vaillant heatpump), didn't have the time and urge to put more effort in, sorry.

If you need anything else, let me know.

[zxdavb]

I'm seeing some interest in Spiders with ramses_cc (HA).

Because the 01FF payload is not fully understood - I thought I'd have another look at it.

You should be able to add -o packet.log to your nohup, to get the packet log in the format that is most useful for test/dev.

I have to say I hadn't looked at the files for a while, didn't realize there were so many errors registered,

There have been a lot of changes since then - well worth upgrading

[VAVons]

Haven't upgraded yet, would have to adapt my changes to the newest client.py, but started the packet logging now in the older version. Is logging this older version sufficient for you?

[zxdavb]

Yes - the packet logs are the same - just that - for you - a lot more 01FF packets will fail the asserts.

[zxdavb]

@VAVons @Onl1ne1373 My apologies for pinging you.

The 01FF payload is not fully understood, but is potentially very useful.

This is what I have:

    setpoint_bounds = (
        int(payload[6:8], 16) / 2,  # as: 22C9[2:6] and [6:10] ???
        None if msg.verb in (RP, W_) else int(payload[8:10], 16) / 2,
    )

    return {
        SZ_TEMPERATURE: None if msg.verb in (RP, W_) else int(payload[4:6], 16) / 2,
        SZ_SETPOINT_BOUNDS: setpoint_bounds,
        "time_planning": not bool(int(payload[10:12], 16) & 1 << 6),
        "temp_adjusted": bool(int(payload[10:12], 16) & 1 << 5),
        "_flags_10": payload[10:12],  #
    }

In your experimenting, have you discovered any more than the above?

[Onl1ne1373]

@VAVons The Spider Thermostat is not discontinued as far as I know, only the Spider Connect Gateway.

I have Home Assistant running with RAMSES_CC integration. And I have created a Blueprint for Spider communication.
This blueprint supports binding and setpoint communication. For me an easy way to play with the Spider communication.
So it responds to 01FF RQ messages and sends 01FF W messages. It is a work in progress.
The Blueprint: https://github.com/Onl1ne1373/home-assistant-blueprints/blob/main/spider.yaml

I still have a problem that some Spiders seem so send a different 01FF RQ message. It sends the 01FF RQ message to itself i.s.o. the gateway. And to make it worse, my HGI80 does not receive those messages, my evofw3 nanocul does.
It is not a range problem, because it does receive the other messages from the same spider.
But my nanocul does not send messages. So I ordered a SSM-D2 Rev B this morning.

Normal RQ:
RQ --- 21:033620 18:010547 --:------ 01FF 026 008029282AD0008000143C28320000C10280800280FF80040020
Problem RQ, is not received by HGI80:
RQ --- 21:033394 21:033394 --:------ 01FF 026 00802E282AD0008000243028320000C10280800280FF80040020

Both spiders are bound to my HGI:80 by the house symbol.

I am searching the logs for fields for Heating, Cooling and displayed error.

[VAVons]

First of all the output from the packet logging as requested. It's still running, so I can get you more later this week.

packet.log

Now that I had a bit more time to look at it, most parsing errors in my nohup.out posted earlier appear to be in the 3120 and 31D9 messages, are those improved in the newer versions? In that case it's really worth upgrading...

Regarding the payload, I haven't decoded any other variables. Initially I thought it might relate to the screen of the spider (since you see exactly the same screen online in the webportal) and the state of the different icons that are present on that screen. But now that I look at it, when the heating starts, the flame icon lights up but I don't see any of the payload changing. On the other hand, the modulation (contained in the 3110 messages the thermostat also puts out) also implicitly contains this flame icon information off course, so maybe it the payload does relate to the icons?

And I know it's only the gateway, not the thermostat that was discontinued. But I have a rather old Spider thermostat with old firmware (yes, there are different firmware versions!) - without the online portal this one is basically as dumb as it gets. Apparently with newer firmware it's possible to have a simple time program for lower temperatures at night etc, but not for mine.

Itho actually did do a decent job compensating clients hit by stopping the online portal (it was their Belgian partner Niko that provided the online portal and pulled the plug on it btw). Itho sent me a Plugwise Adam HA as a replacement hub (with HA integration!). I haven't connected this yet since the old system still functions, so no need yet, and my focus is on the new heating system.

[Onl1ne1373]

I have a newer firmware Spider Thermostat and the simple time program is blocked. They say it is like that because it is connected to a heat pump. So my newer one is probably just as dumb. I wanted the Spider Connect Gateway, but it was not available anymore. But now I connected my 6 spiders to Home Assistant and I can create everything I want thanks to ramses_cc.

In my previous home I had a Plugwise Adam HA system with Lisa Thermostats and it really liked it.

[Onl1ne1373]

According to a log of user 12345caravan on the tweakers forum, the next sequence changes the thermostat from heating to cooling.

Index 31 goes from 5 to 4:

076  W --- 18:011649 21:059180 --:------ 01FF 026 0080802828D0000000143C80800000B50080800280FF80040000
065  I --- 21:059180 18:011649 --:------ 01FF 026 008028282AD0008000143C28400000C50280800280FF80040084
077  W --- 18:011649 21:059180 --:------ 01FF 026 0080802828D0000000143C80800000B40080800280FF80040000
065  I --- 21:059180 18:011649 --:------ 01FF 026 008028282AD0008000143C28400000C40280800280FF80040084

In my log I have it at 4, while it is in heating modus:

045  I --- 21:041873 18:010547 --:------ 01FF 026 00802A2A2CD0008000143C28320000440280800280FF800400A4

So maybe he changed to cooling and back to heating.

In my RQ messages from the Spider to the gateway index 31 is 1:

RQ --- 21:033620 18:010547 --:------ 01FF 026 008028282AD0008000143C28320000C10280800280FF80040020