Firmware download fails - IndexError: list index out of range

[pgauret]

pethublocal fails to download firmware with the following error. Does not seem linked to DNS / network issue as I tried from two different hosts on different networks.

Download Credentials and Firmware for Hub (highly recommended)? Y/N y
INFO:pethublocal:Current Hub Firmware 233.364
INFO:pethublocal:Downloading Current Firmware for H008-xxxxxxx
INFO:pethublocal:SureHub Host hub.api.surehub.io IP Address: 35.172.246.108
INFO:pethublocal:Downloading first firmware record to get header information
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 35.172.246.108:80
DEBUG:urllib3.connectionpool:http://35.172.246.108:80 "POST /api/firmware HTTP/1.1" 200 0
Traceback (most recent call last):
  File "/home/ubuntu/test/bin/pethublocal", line 8, in <module>
    sys.exit(cli())
  File "/home/ubuntu/test/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/ubuntu/test/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/ubuntu/test/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ubuntu/test/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ubuntu/test/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/ubuntu/test/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/ubuntu/test/lib/python3.10/site-packages/pethublocal/cli/__init__.py", line 72, in setup
    pethubconfig = config_load(True, force)
  File "/home/ubuntu/test/lib/python3.10/site-packages/pethublocal/functions.py", line 379, in config_load
    log.info(download_firmware(serial_number, force))
  File "/home/ubuntu/test/lib/python3.10/site-packages/pethublocal/functions.py", line 104, in download_firmware
    recordcount = int(byte[2], 16) + 6
IndexError: list index out of range

[heisenberg2980]

Same issue here, did anyone find a solution for this or is it the end of the road for this amazing project?

[Ryanmt2]

Also seeing the same thing, seems to return a 500 response

[plambrechtsen]

This seems to be because SureFlap are returning a 0 byte response based on the request. The new firmware 233.364 must have bumped the bootloader and I am worried they have moved over to HTTPS for retrieving the firmware which will break everything unfortunately. I don't yet have my hands on a hub that has the new firmware to confirm, but am contacting a friend today to see if I can borrow theirs.

[heisenberg2980]

Thanks @plambrechtsen, really appreciate your effort to try to bring this amazing project back to life. I guess getting your hands in one of these hubs is the best idea as you can play around with it and see what the new firmware is doing, but just in case if there is anything we can do to help just shout.

[Ryanmt2]

Last week sureflap decided to block the surepy library by its user agent and bricked everyones integrations - I think that has renewed a desire for local control (it has for me at least!). Hopefully it can be fixed.

[jacotec]

Last week sureflap decided to block the surepy library by its user agent and bricked everyones integrations - I think that has renewed a desire for local control (it has for me at least!). Hopefully it can be fixed.

I did not notice an issue in my PHL integration - or are you talking about their own Sure Pet app (another shiny outage for all their users)?

[Ryanmt2]

Neither in this case, surepy is a open source library someone wrote to interact with the Sure Pet api without needing to use the Sure Pet app.

This thread explains it better - https://github.com/home-assistant/core/issues/93814

Sure Pet wont engage with the community in any way and have now shown they are willing to take additional measures to block 3rd party access meaning the solution most people are using is looking like it might not have much of a future.

[jacotec]

@Ryanmt2 Read through the issue ... that's insane :-(
I really have no clue why they're fu**ing up their user base.

Does anybody know what wireless protocol is used between the hub and the flaps (@plambrechtsen )?
In case it's some standard (i.e. Zigbee) I hope someone will create or start funding a project of an independent hub replacement on a Raspberry ...

In case I need to replace a flap one day I'm really, really concerned to even shortly connect my hub back to their cloud to get a proper new config. And I hope my hub never dies. PHL is running so great here (apart from the message-of-death issue of the hub, but my watchdog gets it back to life then after max 70 minutes), we'll never want to miss the local installation anymore!

[flyize]

Here's hoping all of us on the new firmware aren't screwed, but I'm guessing we are.

[heisenberg2980]

Does anybody know what wireless protocol is used between the hub and the flaps (@plambrechtsen )?

According to some comments in the thread @Ryanmt2 mentioned (i.e. https://community.home-assistant.io/t/local-deployment-for-sureflap-surepetcare-connect-using-only-local-mqtt-broker/274494/295) it seems the communication between the hub and the flap is zigbee, but I am not sure if it is standard zigbee and with a known key so it can be reproduced. Obviously being able to connect the flap to our zigbee network so the hub is not required anymore would be a dream. @plambrechtsen might have more details about this.

[Ryanmt2]

https://github.com/plambrechtsen/pethublocal

The original repo details some alterative hub options that were investigated before parking the idea in favour of the current solution. It is indeed zigbee

The wemos d1 hub sounds like a dream if there is ever any suitable firmware created!

[heisenberg2980]

Perfect, then if it is zigbee, why don´t we just try to connect the flap to a zigbee coordinator like any other zigbee device so it becomes part of the zigbee network? why is another hub (i.e. wemos d1 hub) required?

For instance this guide explains how to add support for new devices to zigbee2mqtt: https://www.zigbee2mqtt.io/advanced/support-new-devices/01_support_new_devices.html

Just asking because if nobody has explored this option before, it might be worth a try.

[evil-tom]

If I recall correctly, it is essentially ZigBee but with enough of a change to the packet headers (or something like that) so that it's not recognised as ZigBee and thus needs it's own transceiver to receive the data.
Obviously previously the docker software stack and the firmware downgrade was easier than setting up custom transceiver hardware, but if sure pet are actively trying to prevent that then that may no longer be the case.
I guess the advantage of a hardware solution would be that in theory the OEM hub and app could work alongside the home-made one and just sniff the data.
As long as they can't remotely update device firmware.....

[heisenberg2980]

You are right, I have done a small test disconnecting the sure hub, then enabling the "permit join" in my zigbee2mqtt and then putting the flap in pairing mode (which according to the Sure app just requires to click the settings button located at the left of the flap), and unfortunately the flap has not joined the zigbee network (1st step of the guide https://www.zigbee2mqtt.io/advanced/support-new-devices/01_support_new_devices.html), so it seems they have changed the protocol enough so that it is not recognised by the zigbee coordinator, which is a pity because it would have been the easiest solution.

[rfenouil]

Hello, I can't find the link back but at some point people mentioned a microchip implementation of this 2.4GHz zigbee-like protocol. There is probably some documentation available for the communication protocol but figuring the binary representation of states and commands would probably be a challenge.... I also think bypassing the hub is the way to go though... This cloud thing and their anti-open-source politics is ridiculous. Another (challenging) option is to go lower level and plug an esp32 right to the sensors/actuators... When mounting the cat flap I could see a motor to control the lock, a magnetic sensor on the door hinge (probably to infer the cat direction), and an antenna coupled to a chip-reader chip (?)... Not sure how difficult (probably a lot) but depending on the people that are motivated to give it a try, there are different layers to attack. Not complaining about the physical build, which is a good base, otherwise it might be simpler to just start a DIY project from scratch ;)

[flyize]

It's going to be a heavy lift to replace the hub. I doubt that's likely to happen. And it may not even matter if we can't downgrade the firmware. It's possible they've locked us out of everything.

[evil-tom]

Another (challenging) option is to go lower level and plug an esp32 right to the sensors/actuators... When mounting the cat flap I could see a motor to control the lock, a magnetic sensor on the door hinge (probably to infer the cat direction), and an antenna coupled to a chip-reader chip (?)... Not sure how difficult (probably a lot) but depending on the people that are motivated to give it a try, there are different layers to attack. Not complaining about the physical build, which is a good base, otherwise it might be simpler to just start a DIY project from scratch ;)

I had a go at this before I came to this project.
I've got a regular dual scan cat flap that I pulled apart. As you say, the lock, magnetic sensor, and the IR are all fairly easy. But I couldn't get anywhere with the RF coil. The apparently have to be carefully tuned and I couldn't get close.

I think the way forward is with working on intercepting the packets in flight with an appropriate ZigBee sniffer that will recognise the modded packets. At the moment (thanks to this project) we can see read the actions that each transmission performs so hopefully it's a just a case of process of elimination.

As far as I know, the end devices can't be updated remotely, so whatever they do with the hub won't matter, it will still have to speak unencrypted to the devices.

[flyize]

In case anyone may have missed it, @plambrechtsen's response about rolling your own hub is towards the bottom of this post. It lists some of the things that would need to be done.

https://community.home-assistant.io/t/local-deployment-for-sureflap-surepetcare-connect-using-only-local-mqtt-broker/274494/429

[Pirateguybrush]

I was planning to buy a Cat Flap Connect + hub soon, but I'm pretty dedicated to local-only control. Would a new device likely be impacted by this if it's never been connected to the cloud/can I set it up without having to connect it? Does this mean I should wait and see how this resolves?

The thread linked below indicates the issue is resolved for the cloud-based HA integration.

home-assistant/core#93814

[Flachzange]

My understanding: it is quite a risky endavaour for two reasons:

1. The hub you are buying might already have a newer firmware that won't allow the downgrade
2. You must ensure that the hub will never ever get an internet connection

And watch out for the three-headed monkey ;)