SmartThings integration does not report contact sensor updates: HTTP_UNPROCESSABLE_ENTITY

[ocalvo]

Home Assistant release with the issue:
0.94.3

Last working Home Assistant release (if known):
Unknown

Operating environment (Hass.io/Docker/Windows/etc.):
Hasbian

Component/platform:
SmartThings

Description of problem:
Door contact sensors from smartthings are not reporting events

Traceback (if applicable):

Additional information:

Try to redo the smartthings integration, ended up in the same situation
Lights can be activated from HASS, and Lights also report event state change to HASS

[andrewsayre]

Can you share the following:

1. The make / model of the device
2. The Device Type Handler (Type) as shown in the SmartThings IDE. If it's custom, please provide a link to the source code. You can find this by clicking Locations, select your location, click List Devices, then click into the device, and locate the "Type" label.
3. Screenshot of the events from the SmartThings IDE showing the value changing for the sensor (click List Events while viewing the device).

[ocalvo]

1. NYCE NCZ-3010-1-HA Door Hinge Sensor
2. SmartThings type: "NYCE Open/Closed Sensor"
3. Screenshoot:
   

[andrewsayre]

OK that all looks good.

1. Can you describe more about the issue you're seeing? I.e. what's not updating? Does it get the initial state correctly? Do other entities update?
2. Turn on debugging for SmartThings - it will log all push updates received. Is it receiving updates for the contact sensor? Are there any errors?
3. How are you exposing HA to the internet?

[ocalvo]

1. I can see that the initial state is recorded correctly when HASS starts up. But after that no more events update.
2. I have turned this on, but I don't see any event coming from the door contact sensors.
3. I have a proper SSL certificate for my domain.

Notes:

• Other sensors seem to be working fine, If I turn a light in SmartThings, I can see the device state change in HASS
• Also, the states for the contact sensor device seem to be inconsistent:
  For "Closed", the dev state is reported as "off"
  For "Open", the dev state is reported as "on"
• The battery sensor (and temperature sensor of the SmartThings multisensor) ARE reporting events in HASS, just not the open/close events are NOT reported.

[andrewsayre]

I have turned [debugging] on, but I don't see any event coming from the door contact sensors.

That would be the cause of it not updating -- either the events are not being sent, or they are somewhere getting blocked before HA receives them. I'll test with a mock device later tonight to confirm if it's a broader ST issue for that capability (contactSensor), or just an issue for you.

Also, the states for the contact sensor device seem to be inconsistent

That matches what is in the docs. Are you saying that it should be mapped the other way? (Closed -> On)

[ocalvo]

I seems that SmartThings maps to open/close events while HASS maps to ON/OFF.
I am saying that there seem to be some mapping lost in the pipeline from SmartThings (open/close) to HASS (on/off)

[andrewsayre]

Because it's a binary sensor (which is only on or off). What is lost in the mapping?

[ocalvo]

When I inspect the binary sensor in lovelace, I see states like "open"/"close" depending on the state of the device when HASS started.
Then, if I inspect the dev_state, I see "on"/"off", is that expected?

[ocalvo]

Do you know where the code of the smartthings app is?
I would like to modify the app to add more logging to it.

[andrewsayre]

I confirmed that the events are coming through fine for me and my contact sensors are updating as expected. I highly suspect your issue is that the messages are not reaching HA / getting blocked. I would invest your time/energy in confirming you're even receiving the HTTP messages through your router or reverse proxy. It has happened to others and it was always an issue with their network configuration.

When I inspect the binary sensor in lovelace, I see states like "open"/"close" depending on the state of the device when HASS started.
Then, if I inspect the dev_state, I see "on"/"off", is that expected?

Yes. The entity type is binary_sensor which has a state value of on or off. For contact sensor, we use the device class opening, which maps the labels to open and closed in the UI. If you override the device class to something else, you'll get different labels.

Do you know where the code of the smartthings app is?
I would like to modify the app to add more logging to it.

There's already logging of push messages when in debug mode. The code for the integration is here: https://github.com/home-assistant/home-assistant/tree/dev/homeassistant/components/smartthings and it uses two libraries pysmartapp and pysmartthings.

[ocalvo]

Why am I able to see live events from motion sensors, battery sensors and Accelerator sensors but not contact sensors?

Could a network issue explain such behavior?

[ocalvo]

When I open my main door, I can see 'accelerationSensor' and 'powerMeter' events in halog:

Jun 18 21:42:06 hassbian hass[553]: 2019-06-18 21:42:06 DEBUG (MainThread) [homeassistant.components.smartthings] Push update received: {'location_id': 'f18f6a32-c896-464f-acb3-72e9787af412', 'device_id': '806d2a98-e8cd-439d-a68f-0493c9b8c341', 'component_id': 'main', 'capability': 'accelerationSensor', 'attribute': 'acceleration', 'value': 'inactive', 'data': None}
Jun 18 21:42:06 hassbian hass[553]: 2019-06-18 21:42:06 DEBUG (MainThread) [homeassistant.components.smartthings] Push update received: {'location_id': 'f18f6a32-c896-464f-acb3-72e9787af412', 'device_id': 'efcff12d-0499-4c6e-9e4d-cc48ffe0eaf2', 'component_id': 'main', 'capability': 'powerMeter', 'attribute': 'power', 'value': 51.5, 'data': None}

But I dont see the contactSensor events

[ocalvo]

I confirmed that the events are coming through fine for me and my contact sensors are updating as expected. I highly suspect your issue is that the messages are not reaching HA / getting blocked. I would invest your time/energy in confirming you're even receiving the HTTP messages through your router or reverse proxy. It has happened to others and it was always an issue with their network configuration.

When I inspect the binary sensor in lovelace, I see states like "open"/"close" depending on the state of the device when HASS started.
Then, if I inspect the dev_state, I see "on"/"off", is that expected?

Yes. The entity type is binary_sensor which has a state value of on or off. For contact sensor, we use the device class opening, which maps the labels to open and closed in the UI. If you override the device class to something else, you'll get different labels.

Do you know where the code of the smartthings app is?
I would like to modify the app to add more logging to it.

There's already logging of push messages when in debug mode. The code for the integration is here: https://github.com/home-assistant/home-assistant/tree/dev/homeassistant/components/smartthings and it uses two libraries pysmartapp and pysmartthings.

Is there a groovy file that I could read?

[ocalvo]

Found the problem, my hass fails to create a subcription for the contactSensor:

Jun 18 23:34:55 hassbian hass[3675]: 2019-06-18 23:34:55 ERROR (MainThread) [homeassistant.components.smartthings.smartapp] Failed to create subscription for 'contactSensor' under app '2a667a4c-303b-4f12-b50b-add571e21612'
Jun 18 23:34:55 hassbian hass[3675]: Traceback (most recent call last):
Jun 18 23:34:55 hassbian hass[3675]:   File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/smartthings/smartapp.py", line 282, in create_subscription
Jun 18 23:34:55 hassbian hass[3675]:     await api.create_subscription(sub)
Jun 18 23:34:55 hassbian hass[3675]:   File "/srv/homeassistant/lib/python3.7/site-packages/pysmartthings/smartthings.py", line 181, in create_subscription
Jun 18 23:34:55 hassbian hass[3675]:     subscription.installed_app_id, subscription.to_data())
Jun 18 23:34:55 hassbian hass[3675]:   File "/srv/homeassistant/lib/python3.7/site-packages/pysmartthings/api.py", line 275, in create_subscription
Jun 18 23:34:55 hassbian hass[3675]:     data)
Jun 18 23:34:55 hassbian hass[3675]:   File "/srv/homeassistant/lib/python3.7/site-packages/pysmartthings/api.py", line 387, in post
Jun 18 23:34:55 hassbian hass[3675]:     data=data)
Jun 18 23:34:55 hassbian hass[3675]:   File "/srv/homeassistant/lib/python3.7/site-packages/pysmartthings/api.py", line 365, in request
Jun 18 23:34:55 hassbian hass[3675]:     data=data)
Jun 18 23:34:55 hassbian hass[3675]: pysmartthings.errors.APIResponseError: 422, message='Unprocessable Entity'

[andrewsayre]

Well that's definitely the cause of it not receiving updates. This error is coming back from the SmartThings cloud API. Every time you restart HA, it will attempt to recreate this missing subscription -- does it continue to log this error after restarting?

[ocalvo]

Yes. It logs the same error on every restart

[andrewsayre]

Hmm... That's unexpected here, there's no reason it should be failing with 422. I'll work on a patch that logs the body of the error response to see if that yields any more detail.

[andrewsayre]

I've opened #24659 which will log the full error body response when this fails. This will go out into the next beta and release with 0.95.0. While it won't fix the issue, it should give us the details to take the next steps. I'll let you know when the beta is released with this update.

[ocalvo]

Thanks for much, this will help.
Do I need to run pip3 install --pre --upgrade homeassistant to get this update?
Where is the release schedule published?

[ocalvo]

Unfurtunately, after removing the SmartThings integration and adding it again this issue went away
So, I am going to close this issue, however I still don't understand why this happened in the first place.

[ocalvo]

I am seeing this in my log:

2019-07-28 20:45:23 ERROR (MainThread) [homeassistant.components.smartthings.smartapp] Failed to create subscription for 'battery' under app '2862d576-ebdf-4ee9-a0a7-6162197fd038': Unprocessable Entity (422): {"requestId": "244113F6-C526-447B-A1FC-FA219D680EBC", "error": {"code": "LimitError", "message": "subscription limit exceeded", "details": []}}

Is it related?

[cjh0817]

how to solved it?
I have same error about Failed to create subscription

Logger: homeassistant.components.smartthings.smartapp
Source: components/smartthings/smartapp.py:361
Integration: SmartThings (documentation, issues)
First occurred: 01:38:28 (1 occurrences)
Last logged: 01:38:28

Some device attributes may not receive push updates and there may be subscription creation failures under app '9eaeb8a8-e39f-4a82-8fdb-2a345abc0c84' because 44 subscriptions are required but there is a limit of 40 per app

Logger: homeassistant.components.smartthings.smartapp
Source: components/smartthings/smartapp.py:328
Integration: SmartThings (documentation, issues)
First occurred: 01:38:28 (2 occurrences)
Last logged: 01:38:28

Failed to create subscription for 'mediaPlayback' under app '9eaeb8a8-e39f-4a82-8fdb-2a345abc0c84': Unprocessable Entity (422): {"requestId": "9D9A5F7A-3D4D-4109-8558-6CD176238530", "error": {"code": "LimitError", "message": "subscription limit exceeded", "details": []}}
Failed to create subscription for 'waterSensor' under app '9eaeb8a8-e39f-4a82-8fdb-2a345abc0c84': Unprocessable Entity (422): {"requestId": "9DA4476F-A816-4E3F-9155-B17A80AA7A7B", "error": {"code": "LimitError", "message": "subscription limit exceeded", "details": []}}