Getting multiples HTTP 401 when polling camera.get_battery_level #31
Comments
|
@viswa-swami To reproduce that on HA, you can checkout the branch https://github.com/tchellomello/home-assistant/tree/arlo_battery |
|
@tchellomello I will pull that and try it out. I think we can try these 2 options that I have mentioned below first. If that doesn't work out, then we may have to add the HTTPAdapter methodology. Option-1: Earlier when I added the service to enable/disable motion detection, we didn't have this capability so we used a local variable in HASS to set a flag for motion_enabled status. For that above mentioned setup to work, a minor change was required in pyarlo to not subscribe before posting the query, if already subscribed. I will push that change in a day or so. My old HDD failed in my home system and had to replace/reinstall everything !!!!!! Once the subscribe is added after the login function in components/arlo.py, that should optimize to some extent. Option-2: |
|
@tchellomello I took the branch that you indicated, and merged those sensor changes into my code-base... But I am not seeing those Battery sensors getting created at all !!! Is any other change missed in that branch that you gave ? |
|
@tchellomello Ignore my previous comment. It was my bad... I didnt add battery to my configuration file and thats why it didn't show up when i tried !!!! Let me try and get back to you |
|
@tchellomello Ok, I got the error that you were talking about in your comment. I will try to debug that further tomorrow and let you know if i find anything.... |
|
@tchellomello I am getting spurious results... not sure why... sometimes I am getting that error that you reported.. while if i restart HASS I am not getting that error... !!!!!!!!! you noticing the same behavior ? |
|
@viswa-swami I have not noticed it. I'll try to play with that later today |
|
@tchellomello I was trying few experiments with this setup. I was also working on adding some of those parameters are attributes/properties for the camera like brightness, signal strength etc. From the perspective of HASS, I think it might be a good idea to create a separate thread (or a async_task) whichever would be appropriate in the components/arlo.py which will subscribe and get all those properties from the base station under regular intervals, say every 5 minutes or 30 seconds for example. It will populate the properties in a variable. The sensors and camera components can just get the value from the respective variable whenever required. Without this, with multiple sensors, and/or these properties for each camera like brightness, etc, it is slowing down the HASS since each request takes some time to get response from the base station. I will try to implement this as a dirty code first and try it out. Meanwhile let me know what you think. I think this will serve as better optimization from the HASS point of view to incorporate the Arlo sensors and properties. |
|
@viswa-swami yes, +1 to the async idea. Yes that is a good idea and will offer a good optimization. I'm running some tests here |
|
@tchellomello This async idea works really well. I have made some optimizations in the pyarlo library code. I will post for PR later tomorrow. This optimization reduced the time consumed to subscribe, and fetch each property from 5 seconds to 0.7 seconds. In addition to that, with the async thread in home assistant, I have added the battery sensors as well as displaying the battery & motion_detection status in the camera attributes/properties. All those are working well without errors as well as not slowing down HASS. I will try to share that code later this weekend for you to try, if you would be interested. |
|
Yes please share then I can test as well. working for me then we can post it for the HASS version. Very nice to hear that!! Good job @viswa-swami ! |
|
@tchellomello Below is the Home Assistant branch that contains the code for arlo async sensors and attributes. This is based on latset 'dev' branch as of today. The mainly modified files are components/arlo.py, components/camera/arlo.py and components/sensor/arlo.py for the respective functions. Please note that, you would need the latest pyarlo including the PR #34 that I have posted a short while ago. First install that latest version of pyarlo including PR #34 and then try these Home Assistant code-set. You will be able to see sensor.battery_<camera_name> as well as sensor.signal_<camera_name>. If you enable the motion detection using camera service (camera.enable_motion_detection), the 'motion_detection' attribute for the camera entity will mention as 'true'. This was available in 0.48.1 HASS itself, but now the status is live from the arlo base-station itself. I am also planning to add the 'mirror', 'flip' and 'sensitivity' as attributes listed for each camera entity. I will keep adding that in the background while you try this out. Let me know how it goes once you have tried it. |
|
Sorry, forgot to paste the link for the Home Assistant branch: https://github.com/viswa-swami/home-assistant/tree/newly_added_async_arlo_sensors_properties |
|
@viswa-swami pyarlo 0.0.6 published on PyPi. I tested your HA branch and I also updated it to make use of the new version published on PyPi too. I got the sensors displayed on the webUI, however it took 90 seconds to the battery and signal sensors populated.
When starting HASS, I got some messages like below: 2017-07-20 02:12:59 WARNING (MainThread) [homeassistant.components.sensor] Setup of platform arlo is taking over 10 seconds.
2017-07-20 02:13:06 WARNING (SyncWorker_7) [requests.packages.urllib3.connectionpool] Connection pool is full, discarding connection: arlo.netgear.com
2017-07-20 02:13:22 WARNING (SyncWorker_13) [requests.packages.urllib3.connectionpool] Connection pool is full, discarding connection: arlo.netgear.comThe message is still displayed but more sporadic now. Also I with the scan_internval set to 10 seconds I hit the problem below: Exception in thread Thread-10:
Traceback (most recent call last):
File "/usr/lib64/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib64/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/mdemello/.virtualenvs/ha-py36/lib/python3.6/site-packages/pyarlo/base_station.py", line 45, in thread_function
for event in (self.__sseclient).events():
File "/home/mdemello/.virtualenvs/ha-py36/lib/python3.6/site-packages/sseclient/__init__.py", line 58, in events
for chunk in self._read():
File "/home/mdemello/.virtualenvs/ha-py36/lib/python3.6/site-packages/sseclient/__init__.py", line 48, in _read
for chunk in self._event_source:
TypeError: 'NoneType' object is not iterableMaybe to address this error we can check if (self.__sseclient).events() with a try condition. What do you think? |
|
Interesting that after running for some time, some battery sensors zeroed out.
After some minutes, it got re-populated. |
|
@tchellomello After we register the timer in the async_setup code we have to call the function once manually to update the data. After that, every SCAN_INTERVAL it will update the properties automatically. I haven't done that yet in the branch. Otherwise for the first time, it will take some time to get the sensor value. If you restart HASS you will see that the sensor gets reading almost immediately because there is some value first available in DB. Also we cannot reduce to below 15 seconds because, in the update properties function where we are doing this sequence, we are subscribing, then populating around 4 properties and then unsubscribing, which could potentially take more than 10 seconds. If you set the timer to 10 seconds, you might potentially end up calling an instance of the function when previous instance is executing causing such issues. To avoid that, I am going to change the timer such that it will call the update properties function and then when it is done, it will reset the timer for (current time + 10 seconds). That way we can try to increase the polling frequency and yet populate the parameters properly. I will update the branch code probably this weekend and send the code across for you to verify. BTW, I have never seen the urllib3 error message in my HASS with this new code. Not sure if you are seeing it because of reducing the scan interval. Let me implement these things that I mentioned above and we will give that a try. Most probably that should take care of these issues I believe. |
|
The PR #52 should address this problem. |
I was working on extending a sensor for Home Assistant to get battery level for the cameras, then I hit the problem below when pulling the cameras to get the battery level:
If I leave it running, due to multiple requests being executed on the Thread(), eventually all sensors will get updated
I believe the issue might be happening for any requests inside of the Thread().
2017-07-07 01:12:31 WARNING (Thread-3) [r2017-07-07 01:12:28 WARNING (SyncWorker_8) [requests.packages.urllib3.connectionpool] Connection pool is full, discarding connection: arlo.netgear.com
equests.packages.urllib3.connectionpool] Connection pool is full, discarding connection: arlo.netgear.com
I believe we'll need to work with HTTPAdapter to increase the pool. There is a good article about it at https://laike9m.com/blog/requests-secret-pool_connections-and-pool_maxsize,89/
But besides extending the pool, what else should we do to avoid the 401 requests? Maybe should we modify the Thread() code to make sure to just request its one component?
@viswa-swami any ideas?
The text was updated successfully, but these errors were encountered: