POW Notifications only works if HomeApp is opened first #3
Comments
|
Of course, that is not the expected behavior. There is an issue with mDNS protocol from HomeKit framework (https://github.com/maximkulkin/esp-homekit), used to announce to network that an accessory is connected. When a HomeKit device boots, it announces its presence using mDNS (Bonjour Protocol), and iDevices (iPhones, iPads and ATVs) ask it about its characteristics and states, but this happens only some times; other times, iDevices ignores the HomeKit device until you enters into Home App or use Siri to ask about it (It's required an user action). And when a HomeKit device is connected for a long time, sometimes it is unreachable even in Home App. I'm in contact with Max Kulkin to solve this issue, because is annoying that, in my case, all Ikea and KooGeek lights turn on when a power outage occurs. |
|
Thanks for your clarification!, I have been taking Debug logs in order to see whats going on, I found that with the ATV after some time I get a connection and from there the automations start working and notifications are received by the iPhone is in 4G, I have a similar problem than you but with HUE lights turning all ON after a power outage |
|
After talking with developer of HomeKit framework about a possible fix but without anything concrete to solve it, I have tried to improve the fake motion sensor by myself and I have modified Power Outage Warning code to ensure that ATV is connected, and now it works as follow:
When toggle ON (at 60 sec), you can receive the notification (If ATV is connected), and you can configure automation too. But I recommend to configure automation when toggle OFF, so you will have 2 chances to trigger it, first at 30 sec, and second at 90 sec. Another way could be toggling between ON and OFF every 5 seconds during a couple of minutes, for example, but that is so crazy for notifications. Feel free to try it and make any comment or suggestion. |
RavenSystem
changed the title
|
Thanks for the issue follow up, I compiled and installed your latest changes but no luck, something is not working properly upon boot after power lost that notifications aren´t received and automations don`t work. I changed the example to allow send the POW event by pressing the button in the Sonoff and notifications/automations start working after a time. There is any function to rebroadcast the accessory? or a way to verify the accessory is already associated with a client? |
|
There is not public functions to rebroadcast accessories or verify if it's already connected. You can open an issue in framework side at https://github.com/maximkulkin/esp-homekit However, I will continue working on a fix by myself. |
|
Hi, been working on this issue and taking networks logs and reading official docs to try to catch were is the problem, to me the framework is working quite close as expected, the HAP specs mention: "Delivering notifications to controllers requires the controller to establish an HAP session with the accessory", in my case the ATV establish this session 5 to 15 min after first Bonjour advertise is sent from the accessory, I'm still trying to figure out if this time is random or not, doc do not specify this., Before this any notification sent is lost and I think ATV ignore them. Why is not connecting to the accessory as soon it advertise is the mystery to solve, need more reding into the docs and take more logs, maybe is a iOS bug like the one introduced in iOS 10.2 Regarding accessories disconnecting and becoming unresponsive after long time, so far I Have not seen the issue, I have the Led example running rock solid since 20 days. BTW do you speak Spanish? |
|
I agree with the HAP specs: It's necessary that iDevice contact with accessory before this can receive accessory events, and time to do this is variable (from few seconds to some minutes, in my case). This issue could be fixed with a function that verify if the accessory central (ATV, or iPad in some cases) is connected. This is possible because debug info shows it. But actual framework doesn't have that function... yet. The losing connection issue, in my case, is gone away since I use a mDNS packet size of 1500 bytes, with the flag -DMDNS_RESPONDER_REPLY_SIZE=1500. I have dozens of HomeKit devices (38 devices), and queries are forming with every HomeKit device adding its answer to mDNS question. With a low size, I saw a lot of dropped mDSN packets. And yes, I'm from Spain and I speak Spanish. |
|
Great to hear you issue was fixed with the latest packet increase, I found a work around for me for the missing notification. I observed that if the power cycle time is less than the time the bonjour record is removed the ATV will not connect, but if I disconnect, and monitor with dns-sd -B when the record is removed and wait a bit more, as soon I reconnect the device and the device advertise and start the http server the ATV establish de HAP session and everything work fine. I will run a couple of days check and try to find if any documentation is available specifying this timeout. For now with a delay of about 2 min, before the homekit_server_init the POW notifications works flawless |
|
I have added a method to get the connected clients to HomeKit framework, and I have implemented it with POW system. Now, every 3,5 seconds, during first 15 minutes, it checks number of connected clients, and if there is any variation, a POW is sent. Please, try it and comment. Many thanks. |
|
I have made some modifications in the POW system. I think that now works well. Check it. |
|
Hi!, latest version is a bit more reliable but in my case is still failing 60% of the time, I think the problem could be an unsung between the notification sent and the connection not yet established Here a log, the power ON event is sent before the Pair Verify Step 2, also there is an overflow because it shows 255 client connections.
Is by design that the notification is resend with each connected client? I'm still testing in average so far it takes 7 min to get a connection and the automation take place. Also found that the TTL of the bonjour record is 60 secs |
|
Hello, Overflow is caused due to a bad implementation by myself. "Closing client connection" produces a -1 connected client. And I think that just now it is fixed. By design, every 10 seconds function checks if there are more connected clients and, if the number of connected clients is different compared with anterior check, it sends the POW notification. And yes, TTL appears to be 60 secs here:
|
|
I have set TTL to 120 according to https://tools.ietf.org/html/rfc6762#page-33 |
|
Good idea I will test this new one, previous version during my full night test expired many times and got renewed, other home kit devices I have in my network did´t show that behaviour, I have also a LED test compiled with no changes, running in a WEMO D1, and his record did´t expired. What do you think about sending a goodbye packet of a flush cache on power up?, Maybe this should clean the record on the connected clients and should allow a faster renewal and notification delivery. |
|
Sending a goodbye packet is a great idea, but I have tried the goodbye packet using a TTL=0 before announces mDNS (with a pause between them), and it doesn't work. iDevices ignore it and there isn't any change. But I have found a little error in the HomeKit framework. In _hap._tcp Bonjour TXT Record Keys table there is a "c#" key, and official documentation describes it as:
Well, HomeKit framework here uses a constant =1, but I have changed it to a random number, forcing to iDevices to query the (fake) new version of the accessory, and my AppleTV always connects very quickly. I know that using a random number here is not accomplished with official specifications, but I want an accessory that works. I don't care about official specifications, it's not a MFi project. Please, update git and test it... and comment ;) |
|
Look at what I found! https://developer.apple.com/library/content/qa/qa1310/_index.html This is exactly what we are facing!. BTW your latest build improved the responsiveness a lot, now I´m getting a re-registrarion almost immediately or only few minutes later the power cycle the only issue I´m facing is multiple notifications and automation triggers, when the automation is just turn off light its not a big problem but if you are trying to automate something else then you may face issues. Also I see that these notification are async with the registering of devices, some times are triggered in the middle of a new registering, don´t know if this may affect functionalities Anyway these were just my initial tests I need to test more deeply and look at the logs. |
|
I have uploaded more improvements just now... |
|
Great, I wil test tonight! I was reading https://tools.ietf.org/html/rfc6762#section-10.2, wondering why if the Flush-Flag bit is set to 1 as I think reading the code is set, its not triggering and apparent cache flush and reconnect from the ATV |
|
WOW! the latest changes made it work flawless, now with each reboot the behavior is the same and the notification is delivered and the automation triggered. I will stress test it but from what I have seen in the Wireshark logs Bonjour traffic is working lot lot closer to specs, specially the Exponential Back-off. Great work! |
|
Worked quite well during my initial tests, but overnight it stops working, I power cycled the device in the morning and failed, power cycled a second time and failed again (both times the notification was sent but ignored) , after the third PowerCycle it worked. Will let it stay on for the full day and see what happen when I'm back at night. |
|
Please, check the latest commit to see if accessory stops working. |
|
Hi, I tested the yesterday release, the reliability is not better, actually I think is worst than few days ago, but I will test today release with enhancement in the mDNS Task and look a bit deeper to try to figure out whats happening. What keeps me thinking is why with previous releases after reflash the unit all seems to work ok, but after few hours (7h or +) if I power cycle the device the notification is lost. I will test with all my other iDevices turned off and with just the ATV on. |
|
Made some code changes (removed de -1 TTL and added repeated goodbye packets) , and now looks like every time I power cycle the device the record is removed from the cache then re-added and the connection from clients is established, but for some reason ATV not always connect during the first 20 secs. I changed the POW delay from 20 to 40 to see if I can see any improvement. I need more tests because the failure normally happen after 5+ hours |
|
How many goodbye packets do you send (and much delay between them) before normal mDNS announcement? |
|
I sent 3-4 goodbye TTL=0 with a 1500 ms delay between, then one with TTL=120 and then the normal fallback with the defined TTL. I´m sure 3-4 packets is exaggerated it was just a quick test, I will try to finetune the number during the weekend. |
|
Maybe 3 goddbye packets is enough. I’m going to implement a function in HomeKit framework to notify events only to new clients, and not to all connected clients. So, when a new client is conncted, only it recibes the notification, and not the other connected devices, so there is not duplicated automations (thing that happenned in some previous version, as you know). |
|
Will not after a power cycle all the devices considered as new?, I will try with a longer delay before the notification, after a power outage the ATV may take longer to boot than the Sonoff and the notification will be lost anyway. |
|
I have added functions to manage delayed events. |
|
After some tests, I have improved the way to send delayed events, and now they are managed by a different task in a better way. Please, try the latest commit and comment. |
|
Tested but I'm unable to have any conclusion, some times automations are triggered some times not, some times I receive multiple notifications, I have been unable to find a pattern. Testing automations I made an experiment nothing to do with POW, I setup one sonoff to control another, when one is ON/OFF the other will turn ON/OFF, found that that this simple automation was not working 100% of the times, on first connection worked then stop working for a while then work one time and need to wait a while to have it work again. Made same test with HUE and works 100% of times. Tested with the framework examples and got same results. |
|
With the framework examples works 100% of time? I have test a simple automation between 2 Sonoff too, and works 100% of the times for me. The differences between framework from examples and mine are mDNS announcements, delayed events, and mDNS buffer (I use 1500 bytes, but framework from examples uses 512). If you want, you can change it in mine modifying Makefile file of device to build: Delayed events are not involved in devices without POW, and mDNS announcement system shouldn't influence. |
|
Examples from framework have the same behaviour, is not that they don´t work they cannot be triggered multiple time in short period of time. for example:
I also tested the sonoff as master and HUE as Slave and the result is the same Using a HUE lamp as master and the Sonoff and Led example as slave, works as expected Using a HUE lamp as master and another HUE lamp as Slave works as expected Wondering if its something only happening to me or something common. |
|
I have tested it, but I have not any issue. Even I can switch very fast (as fast as I can do with my finger) Sonoff in Home App and slave Sonoff works fine without any fail. Can you see what value do you have in DTIM in your 2.4ghz wifi configurarion of your access point? Try to set it to 1. And if your ATV is connected via wifi 5ghz, change its DTIM to 1 too. (Maybe you need to restart your AP after chages). |
|
My AP is a airport not too much options to play around, my ATV is connected thru Ethernet to the Airport as the HUE bridge, but you gave me a clue, devices that fail uses the 2.4 Ghz network maybe the problem is there. But in case of an automation, is the ATV that should be sending the On/Off commands, I´m not using the physical button on the Sonoff, if a packet get lost because a device was sleeping this should not be the behaviour, in the case I have the Sonoff controlling the HUE the HUE should turn On unless the ATV only send the command to the HUE after receiving confirmation from the Sonoff that he accepted his command. |
|
Maybe do you have another 2.4ghz AP to test with Sonoffs? You can connect it to your wired network so it can be connected with your ATV for automations. |
|
After a complete flash erase, erase de accessory from HomeApp, flash the latest release and pair the accessory again, automations and remote access works as expected. PoW notifications still have some issues, but thats another story. |
|
Great! Now we can continue with POW issues. I have been testing a KooGeek outlet (certified accesory) that has a push button too, so I can configure automations linked to its button, and AppleTV some times takes about 10-15 minutes to contact it. And in other way, goodbye packets seems have no effect. Actually, POW event is sent to all connected devices before initial delay, and then a POW event is sent only to new connected devices during some minutes. But some times you can get 2 notifications or 2 triggered automations. I thnk that if an AppleTV is connected and notified (automation is triggered) and then an iPhone is connected and notified, this iDevice tells AppleTV the event (without accessory tells it to AppleTV) and the automation is triggered again. |
|
Yeah goodbye packets even if part of bonjour and being used in the simulator don't seems to make a difference. What I have observed is that after a reflash, the sonoff gets connections from most of the clients at my home and the POW notification works, then if I disconnect and connect power this not always happen. Maybe the key is not on the bonjour announce but on the socket expiration. My problem is I need to log in a point were I can see my ATV connections to accesories and to do that I need to change a bit my setup. Another idea I had yesterday but have not yet tested, the ci= field in the mDNS TXT record indicates accessories category we are using Switch and I was wondering if change it to sensor or security system will change the reconnections behavior of the ATV. |
|
Delayed events don't work as expected, and I have removed them. Now, POW system works in a different way:
The idea is to attach fake-switch state to the "turn off lights" automation, and turn it off too. So, this way ensures you that automation will launch once (POW events are sent until fake-switch is off). The problem is that you will get an iDevice notification from the motion sensor (You can disable it) every 30 secs until automation launch (or during first 10 minutes). It is not perfect, but works. The best deal is to check if connected iDevice is an accessory HUB, but for now I don't know how do that. In other hand, I have test changing ci flag to an alarm system, but there is no changes. AppleTV connects when it want (some times quickly, and some times takes 5-15 minutes). |
|
Man you read my mind, I was testing something similar but in HW wiring the relay control to the GPIO14 to detect when the automation was triggered your soft solution sound much better, I feel so stupid now :-). I will test during the weekend |
|
I hope this workaround makes POW usable. I'm testing it with a Sonoff TH. For now, my Ikea lights turn off only once when a power outage occurs, and I can turn on selected lights again, without another turn-off automation launched. |
|
Hi, I think we are facing some memory problems, I'm getting Stack overflow messages each time a Notification is activated and after few tries a fatal exception. To test I removed the old accessory, erased the flash, loaded the new firmware, paired the accessory. I found a couple of strange behaviors we may need to verify:
Short Debug log:
Task stack overflow (high water mark=0 name="Power Outage Wa") 0x4021e275: lwip_poll_should_wake at /home/axiom/esp-homekit-devices/sdk/esp-open-rtos/lwip/lwip/src/api/sockets.c:3773 epc2=0x00000000 0x4020356d: prvIdleTask at /home/axiom/esp-homekit-devices/sdk/esp-open-rtos/FreeRTOS/Source/tasks.c:4662 excvaddr=0x0000003f 0x4021e24a: select_check_waiters at /home/axiom/esp-homekit-devices/sdk/esp-open-rtos/lwip/lwip/src/api/sockets.c:3773 Registers: 0x4021e24a: select_check_waiters at /home/axiom/esp-homekit-devices/sdk/esp-open-rtos/lwip/lwip/src/api/sockets.c:3773 a4 00000000 a5 00000000 a6 3fff6810 a7 00000000 Stack: SP=0x3ffef650 0x4022042d: accept_function at /home/axiom/esp-homekit-devices/sdk/esp-open-rtos/lwip/lwip/src/api/api_msg.c:1373 0x3ffef680: 3fff1968 3fff1984 3fff1b9c 4020e748 0x4020e748: tcp_process at /home/axiom/esp-homekit-devices/sdk/esp-open-rtos/lwip/lwip/src/core/tcp_in.c:930 0x3ffef690: 40204a3c 0000000a 3ffef7b4 00000000 0x40204a3c: prvCopyDataFromQueue at /home/axiom/esp-homekit-devices/sdk/esp-open-rtos/FreeRTOS/Source/queue.c:2332 0x3ffef6a0: 1401000a 0c01000a 3fff196a 40204f24 0x40204f24: xQueueReceive at /home/axiom/esp-homekit-devices/sdk/esp-open-rtos/FreeRTOS/Source/queue.c:2332 0x3ffef6b0: 00000000 00004600 0002bc64 00000014 0x4020c659: ip4_input at /home/axiom/esp-homekit-devices/sdk/esp-open-rtos/lwip/lwip/src/core/ipv4/ip4.c:701 Free Heap: 27940 ets Jan 8 2013,rst cause:1, boot mode:(3,6) load 0x40100000, len 2292, room 16 0x40100000: _stext at ??:? tail 4 rBoot v1.4.0 - richardaburton@gmail.com Booting rom 0. ESP-Open-SDK ver: 0.0.1 compiled @ Mar 9 2018 10:23:37
Full log from Re-flash after erase until crash by WDT, note how the name in the stack warning change indicating a memory corruption: Task stack overflow (high water mark=0 name="Power Outage7W") |
|
Task stack overflow fixed. I have increased stack to 256. |
|
I'm not working on this, and due to the delay of AppleTV to connect with devices on boot, I'm going to forget this for now. |
I have been playing around with the SONOFF-Basic-wPCA notification, as far I understand the feature is intended to notify the user a power outage has happened, great idea BTW wish HUE as something like this.
The problem I'm facing is the notification only display on the iPhone if during the Power Cut Alarm: INIT OFF the HomeApp is opened, if not notification are never received and automation not triggered
Step to reproduce:
if the HomeApp is open while the full power cycle process notification works without problem
Is this the expected behavior?, all my devices are in the same network, AppleTV is setup as Hub.
I think this has something to do with the need to establish a connection with the accessory and iPhone/ATV before notification may work but is not this need losing the point to have a PCA notification? Maybe a delay and a verification that at least a client is connected is needed to hold the change notification or a bigger INIT Delay.
The text was updated successfully, but these errors were encountered: