Use access_token instead api_password

[awarecan]

It is an epic includes all files need to remove api_password, change to using access_token.

Separate issue/PR may create for each change.

• !! auth.py, only remove it after all others done.
• !! auth_providers/legacy_api_password.py, only remove it after all others done.
• components/api.py, external REST API
• components/camera/proxy.py, both image and stream proxy
• components/camera/push.py, Replace api_password in Camera.Push #16339
• components/device_tracker/geofency.py, should decouple from HA's api_password
• components/device_tracker/gpslogger.py,
• components/emulated_hue/__init__.py Decouple emulated hue from http server Decouple emulated hue from http server #15530
• components/frontend/__init__.py
• components/hassio/handler.py, Auth + Hass.io: create system user #15220
• !! components/http/__init__.py, only remove it after all others done.
• !! components/http/auth.py, only remove it after all others done.
• !! components/http/trusted_networks.py, only remove it after all others done.
• components/mqtt/server.py, embedded mqtt server default password using api_password MQTT embedded broker has to set its own password #15929
• components/rss_feed_template.py,
• !! components/sensor/sigfox.py, not our api_password
• components/websocket_api.py, websocket API
• components/zeroconf.py, reuqires_api_password params
• config.py, the default config template Remove commented out API password from default config #16147
• remote.py, API class, shall be retire, see comment part 1 Remove remote.API from core.Config #15951 removed

All tests are not listed here.

[balloob]

I suggest we drop the API class in Remote and the related methods. Home Assistant is moving away from Rest to WebSocket calls, and we should not maintain these methods. It's also sync instead of async.

[awarecan]

Will that break lots of users?

[awarecan]

We probably need two rounds change. The first round, add access_token, keep api_password, and make sure access_token will be used if auth.active. Make a big announcement, switch auth.active from opt-in to opt-out. Then make the second round, remove api_password.

[gerard33]

I guess Geofency should be added to the list as well.
https://www.home-assistant.io/components/device_tracker.geofency/
components/device_tracker/geofency.py

[awarecan]

We may have to implement accept access token in query string (RFC 6750 2.3) to support web hook usage.

I will suggest to add an optional config in auth components to explicitly enable this support, since it is not recommend validation method

[balloob]

I don't think that I want to support that. Integrations can allow users to configure a token if they want to do auth via url

[awarecan]

I see.

@gerard33 So the Geofency will need change its view as follow example.

class GeofencyView(HomeAssistantView):
    """View to handle Geofency requests."""

    url = URL
    name = 'api:geofency'
    requires_auth = False

    async def post(self, request):
        if not your_custom_validation(request):
            return web.Response(status=401)

EDIT: So, for any 3rd party want to use HA native auth system, they have to official support OAuth 2 protocol, otherwise, the component need implement itself auth system.

To keep backward compatibility, it can still accept api_password in query url or basic auth header. However, the actually validation logic need be written by individual component, HA will eventually remove current api_password configuration and validation function from http component, do not rely on it.

[balloob]

When adding something to the url, it should not be a HASS access token, instead the integration should allow the user to specify some sort of auth in the config.

We don't have permissions yet, access tokens are very powerful, never expiring access tokens even more (as that is what a user would want to use). URLs are getting logged in proxies or other systems left and right. There is a very good reason auth should not be send via a url. So if an integration like Geofency wants to use auth in the url, it should define their own string, as that would only grant access to that one endpoint.

[awarecan]

Another issue for 3rd party use access token if they don't native support OAuth2 is that they cannot pass IndieAuth, since we need a redirect url match the client id. This will be a deadend.

I will edit my previous comment

[pvizeli]

It's very dificult to use websockets with curl in bash scripts. I like also to minimize the REST API support but complete remove in flavor of websockets are heavy.

A developer can write a short python script but a sysadmin ends with call curl.

There are also other integraton like nodered they run with eventstream and Rest API. I think a rest API and mqtt are a minimun support for IoT device. That will be also used for esphome.

But I also think we should remove the remote class.

[jedi7]

Hi, the warning "Please change to use bearer token access" is comming also from appdaemon calls.
I'm also for to keep the rest api (ex: for simple router scripts to track devices)

Regarding appdaemon, how is auth solved for "system" 3rd party services? Like appdaemon?
Thanks

[balloob]

Just like all other apps: migrate to the new auth.

[balloob]

Did a quick grep through the code. These are the components that register HTTP views and probably need to be updated to work with access tokens:

• alexa (manual setup)
• binary_sensor/mystrom.py
• camera/push.py (Replace api_password in Camera.Push #16339)
• device_tracker/automatic.py
• device_tracker/geofency.py
• device_tracker/gpslogger.py
• device_tracker/locative.py
• device_tracker/meraki.py
• device_tracker/owntracks_http.py
• dialogflow.py
• dominos.py
• doorbird.py
• foursquare.py
• google_assistant (manual setup)
• konnected.py
• mailgun.py
• prometheus.py
• rachio.py
• rss_feed_template.py
• sensor/torque.py
• spaceapi.py
• switch/netio.py
• twilio.py

Might be less as I have not checked if all these HTTP views require authentication.

[DavidFW1960]

My Google Assistant (manual setup) used the original gactions works fine still without legacy_api enabled.
I also think you might have missed IFTTT in the list?

[frog32]

binary_sensor.http is also missing in this list

[awarecan]

IFTTT can be categorized with REST API.

Maybe we shall retire sensor.http and binary_sesnor.http?
EDIT: clearly the doc is outdated, those should be removed from components. They are part of REST API

[dgomes]

I'm responsible for camera.push

So currently I have:

POST /api/camera_push/camera.<entity_id>

This endpoint must be secured, and I relied so far in the rest API.

Since most calls are handled using curl as external call to a script, it makes no sense to move this to the websocket API.

Should I implement #15376 (comment) proposal ?

[balloob]

@dgomes yeah. I would suggest something like this:

# configuration.yaml entry
camera:
  - platform: push
    token: sdlkjadslkjadskjlad
    name: MotionEye Outdoor
    buffer: 3
    timeout: 5

And then accept pushes without authentication on the url

POST /api/camera_push/camera.<entity_id>?token=<token>

This is exactly how the Media Player thumbnail endpoint works:

https://github.com/home-assistant/home-assistant/blob/f20a3313b09129b725fec0ac4064bafe940db233/homeassistant/components/media_player/__init__.py#L995-L1014

[dgomes]

Tks @balloob !

Are we OK replicating this solution in all other components relying on api_password ?

Can't we create a special user with limited access to specific API's? (RBAC -> implement user based ACL's) I think this would also pacify the discussion on the last release blog post.

In the meanwhile I'll prepare this change to the camera.push component.

[balloob]

we'll get permissions eventually, not today though 😉

Yeah we're ok with replicating this logic. It's not a lot of logic so no need to create an abstraction for it.

[rohankapoorcom]

Okay, that makes sense. I guess the correct approach for those components then is continuing to use a password from config just not the one from the HTTP component and then disabling auth on the view?

I disagree though regarding responses from webhooks. Some endpoints (Twilio in particular comes to mind) require a webhook to be acknowledged with a certain response (in twilio's case their specific XML format) so that they know you received the data.

[balloob]

Those components should start using long lived access tokens yes.

[balloob]

I Apologize, I did more research and saw the PR by @rohankapoorcom and changed my mind. Webhooks should be able to return responses, but we should make sure we only use them for webhooks.

[rohankapoorcom]

I was thinking about moving the Meraki device tracker platform (first to a split component/platform) and then to use the webhook component instead of the current API password approach. Unfortunately I found this in the Meraki docs:

Upon the first connection, the Meraki cloud will perform a single HTTP GET; the server must return the organization-specific validator string as a response, which will verify the organization's identity as the Cisco Meraki customer. The Meraki cloud will then begin performing JSON posts.

Since we don't allow the webhook component to respond to GET requests, what do you recommend for moving this component off of the API password? The simplest approach I can think of is to switch to an unauthenticated webview and perform our own authentication (via a custom API password) for this platform.

[awarecan]

After #21884, only api_password left are in either test code or legacy_api_password auth provider or http and websocket_api related codes. This issue can be closed.

[RyuzakiKK]

@awarecan What about components that still requires http.api_password to work, like alexa?
I hope api_password will not get removed until all HA components switched away from it.

[awarecan]

I believe I have already checked all usage place, there should be no more dependency on http.api_password.

Could you please point to me where alexa need http.api_password

[RyuzakiKK]

For Alexa the wiki guide says that the api_password is still required as an endpoint https://www.home-assistant.io/components/alexa/#requirements and apparently we can't yet use a long lived access token for example.

[awarecan]

You can still use api_password in URL as long as you have legacy_api_password auth provider configured.

BTW, use api_password in alexa endpoint is not a good design, we should change it by verify the Amazon signature.

[balloob]

It should use a webhook.

[awarecan]

Actually, just like google assistant, alexa also can leverage our oauth2 authentication. I can look into it, https://developer.amazon.com/docs/account-linking/account-linking-for-sh-and-other.html

[balloob]

Oh there you go!

[jerrychong25]

Based on the long conversation of this issue, it is confirmed that api_password will be dropped completely soon. A migration definitely is unavoidable.

Just curious, what is available method for this migration?

AFAIK, currently only have long live access token with curl command method.

[awarecan]

@jerrychong25 #15376 (comment)

[RyuzakiKK]

@awarecan any news for the oauth2 authentication in the alexa component?

Sorry for being probably off topic, but as far as I know there isn't an open issue about that.

[awarecan]

Document has been updated. https://www.home-assistant.io/components/alexa.smart_home/

[filikun]

Looking at the Meraki integration it seems to still need the api_password? https://www.home-assistant.io/integrations/meraki/

[Josti79]

Looking at the Meraki integration it seems to still need the api_password? https://www.home-assistant.io/integrations/meraki/

Has anyone had any luck to fix the Meraki component?

[123dev]

Looking at the Meraki integration it seems to still need the api_password? https://www.home-assistant.io/integrations/meraki/

Has anyone had any luck to fix the Meraki component?

And considering that Meraki Location and Scanning settings does not allow to set the HTTPs headers, the integration doesn't work with Long Lived Access Tokens
Setting up nginx reverse proxy to append the required headers based on the URI query api_password value seems to have worked around the issue for now.

[awarecan]

Meraki need to change to use webhook implementation, @filikun @Josti79 @123dev you can look other device tracker platform, such as gpslogger or locative.

Lots of device tracker platform have changed to use webhook.