Automation for android notification with attachment

[apaperclip]

Hi Blake,

I had three observations I wanted to share.

HA 117.6 in docker
Frigate custom component
Frigate 8.0 beta integration in a container (not the addin)

1. Following your automation example I updated my notification automation to use the image path from frigate derived from the event info in the mqtt message. But following my existing automation that uses the /api/camera_proxy/camera.entity approach, I kept the path relative to my HA's external domain. image: /api/frigate/notifications/{{trigger.payload_json["after"]["id"]}}.jpg?format=android. I found this did not work like the previous camera proxy path. Once I put https://my.domain:port:/ in front like your example, images started working again . I thought this was odd and was chatting with dshokouhi on the android discord channel. The android app should be prepending the base url whenever it receives just the path. Looking at the android app logs I see that when supplied with just the URI, the app is url encoding the ? to %3F and something in the flow does not like it and was 404'ing the request. This encoding should be fine, but I wanted to see where the issue was. Looking at my traefik logs, it passes the request on through and gets a 404 back from something.

I also bypassed traefik and went http direct to 8123 of home assistant had the same behavior, ruling out traefik.

I hit the frigate enpoint events/id/snapshot.jpg taking the event id from the .jpg image above that the app was using and it did not work if I used %3F, I had to use the ?.

I don't know if this is coming from the HA api reverse proxy, or something inside frigate. What logging can I turn on and where (integration? docker container?) to find where this an issue?

I think I could get around this by using /api/camera_proxy/camera.front_person that the integration creates. But I wanted to know if there was there a reason you went with pulling the image from frigate itself vs using the snapshot from the snapshot camera entity.

2. Another item that came up while I was discussing with dshokoui was the HA docs at https://developers.home-assistant.io/docs/api/rest dont have an endpoint for // like we have here with /frigate/notifications. Is this approach HA supported? I can hit the /api/frigate/notifications image endpoint without authenticating to HA (used incognito mode from a browser). Whereas the api/camera_proxy endpoint requires auth and its handled by the app.

3. In testing of item 1 I removed the android query parameter and found that the image was larger in my notification shade on my Pixel 3a/android v11 and filled the width of the notification. I don't think it was stretched or anything. This could make items 1 and 2 moot for me, but if others want the android format, they may have issues with combination of scenarios.

Sorry this became a lot longer than expected. If you want this info shared as issues or elsewhere please let me know.

[blakeblackshear]

1. If the frigate endpoint is returning a 404 when you use %3f, I suspect it is coming from frigate. I guess flask isn't treating the ? the same when it is url encoded, and that's what I would expect. That is the point of url encoding after all. You can ramp up the flask logs in the logger section of your config with the name werkzeug. The problem with the camera entity is there are no guarantees the image will be for the tracked object id in frigate. That camera just shows the last image for the camera generally.
2. Is adding /api/frigate supported? I never really thought to ask. There are many other integrations that register HomeAssistantViews, many of which do not have authentication enabled. Images must be available publicly for notifications, which is why I created a separate API with a small surface area.
3. Android wants images to have a 2:1 aspect ratio and iOS wants a 1:1. Android doesn't stretch it, but it cuts off the top and bottom. I prefer to see the entire image.

[apaperclip]

Thanks for the quick reply!

1. I'm surprised by that, urlencoding is such a ubiquitous thing, its not like flask is some esoteric webserver/framework. I did this to make sure I caught the logs

logger:
  default: debug
  logs:
     werkzeug: debug

and I see info level and debug level logs for werkzeug. When the %3F is in the URI I do not get a log entry for the request at all.
Great point on the association of image to event.

2/3 Thanks for the reply and information.

[blakeblackshear]

If its not in the logs at all, then my guess is that the request isn't getting to flask. Flask actually sits behind nginx, so you could also look at the nginx logs /var/nginx/access.log.

[apaperclip]

I see traffic and good calls in access.log, but when I add the query parameter I do not see the request in access or error.

I'll look at HA debugs next and anything else you may want me to check out.

[apaperclip]

I think it may be in the integration or whatever HA uses to route the request to you. It looks like the integration rewrites the GET to be /events//snapshot.jpg. Doing some packet captures between the various hops, when the question mark is urlencoded I do not see the request from HA to Frigate on port 5000 for the jpg.

I tried to turn on debug logging for ha core and the custom component through ha's logger and I get logs but not one that shows me the web calls between things inside HA

[blakeblackshear]

HomeAssistant probably isn't matching the request to my registered view then: https://github.com/blakeblackshear/frigate-hass-integration/blob/master/custom_components/frigate/views.py#L155

[apaperclip]

Agreed, I turned on debugging for the homeassistant.http component hoping to get something.

The best I can get right now is when there is no ? I get this log message
2020-12-15 15:05:07 DEBUG (MainThread) [homeassistant.components.http.view] Serving /api/frigate/notifications/1607975160.248261-j7dhww.jpg to x.x.x.x (auth: True)

When there is a question mark I do not get an equivalent log message.

I've pulled on string about as far as I can go. It is not a show stopper for me and if it ever impacts more people we have a starting place. As a last ditch effort I'll ask on discord and see if there is way to get more logs out of HA. You can close/archive this if you like. Thanks for taking the time to look and reply.