Neato/Vorwerk Integration fails with "Invalid Credentials" #39378

grantclem · 2020-08-29T08:35:46Z

The old Vorwerk app is being phased for a new 2FA based app (MyKobold) from Vorwerk. The Neato/Vorwerk integration is failing with "Invalid Credentials", I assume because of this new requirement.

Environment

Home Assistant Core release with the issue:
Last working Home Assistant Core release (if known):
Operating environment (OS/Container/Supervised/Core):
Integration causing this issue: Neato
Link to integration documentation on our website: https://www.home-assistant.io/integrations/neato/

Problem-relevant `configuration.yaml`

Traceback/Error logs

2020-08-29 14:30:43 ERROR (SyncWorker_19) [homeassistant.components.neato] Invalid credentials

Additional information

The text was updated successfully, but these errors were encountered:

probot-home-assistant · 2020-08-29T09:11:03Z

neato documentation
neato source
_{^{(message by IssueLinks)}}

dshokouhi · 2020-08-29T15:09:45Z

If they added 2fa then we need the parent library fixed and that goes beyond my understanding. Do you get a text message with the code or something after logging in?

dshokouhi · 2020-08-29T15:11:46Z

Can you try to interact with the parent library as outlined here https://github.com/stianaske/pybotvac then open a issue there with your results.

grantclem · 2020-08-29T15:13:46Z

Hi, yes via email a code is sent

grantclem · 2020-08-29T15:17:21Z

Can you try to interact with the parent library as outlined here https://github.com/stianaske/pybotvac then open a issue there with your results.

This I would assume only be related to Vorwerk and would not be a Neato issue?

Santobert · 2020-08-29T15:17:49Z

Try to use token-based authentication. I know that neato supports oauth, which is token based ~~and also supported by pybotvac~~. If that works, we could consider using oauth in HA as well. The downside is that we have to drop file-based configuration in favor of configuration flow.

malkie-hass · 2020-09-15T17:32:15Z

I have the same issue that I moved to the new App and can now not use my Vorwerk anymore with Hass.
Is there anything I can do to help?

9kon · 2020-09-25T13:02:33Z

Im also very happy to help in any way possible, as i moved mine to the new app, before checking HA compatibility.

Santobert · 2020-10-10T08:31:35Z

I have done some research. Neato supports an OAuth flow, which means we can work around this problem. Currently pybotvac only supports stateful authentication. This requires the application to log into the account using the user's credentials. Obviously this does not work with 2-factor authentication. OAuth is not really stateless, but works token-based. As a result, the user has to enable the application once via 2FA so that the application can request access tokens. These tokens can be used, for example by hass, to authenticate requests against the Neato API.

Unfortunately as far as I know, pybotvac does not yet support token-based authentication. Please correct me if I am wrong. The best and maybe only solution to achieve this would be for HA to manage the OAuth flow. This is necessary because HA must redirect the user to Neatos login page and HA is able to store all access and refresh tokens. Pybotvac has to accept valid access tokens for each request. Therefore every public function must accept an optional parameter access token.

Additionally, we have to adapt neatos config_flow. We should drop support of yaml configuration. Furthermore we have to implement the OAuth flow there.

Any opinions? I am not an expert, so please correct me if I said something wrong.

Documentation:

dshokouhi · 2020-10-10T23:20:01Z

@Santobert dropping support of yaml sounds good to achieve this. I think also using the built-in HA methods are the right way to go since that is why they are there. If you need to make changes to pybotvac then I would be hesitant since I have not seen activity there in some time. I submitted a PR earlier to allow us to clear the current alert but have not heard back yet.

Santobert · 2020-10-11T17:21:13Z

Tbh, I'm not sure I have time for this. Any help is welcome! Furthermore, I don't have any Vorwerk bots here, so I can't verify if OAuth can solve this problem.

We definitely need to make changes to pybotvac. Unfortunately, I have another unmerged PR since October 2019. @dshokouhi do you mind writing an E-Mail to @stianaske ?

Our next steps should be:

Verify that the Vorwerk API follows the documentation here and accepts OAuth tokens
Change pybotvac so that it accepts access tokens with every request
Bump pybotvac and change config_flow.py according to the new OAuth Flow

stianaske · 2020-10-12T17:17:25Z

@Santobert Sorry about the unmerged PR. I'll look into that one right away.

I would also be happy to accept a PR where we change to using OAuth if you guys need that to support additional robots.

Santobert · 2020-10-18T11:08:16Z

stianaske/pybotvac#59

Can someone with a Kobold please test if this branch works? You just have to install pybotvac and run sample.py with valid client_id, client_secret and redirect_uri. If you have any questions, feel free to comment this PR.

The corresponding changes to HA are prepared here:
https://github.com/Santobert/home-assistant/tree/neato_add_oauth

Santobert · 2020-10-20T15:48:48Z

Any updates?
I definitely need someone with a Vorwerk Kobold to test stianaske/pybotvac#59 before I can implement these changes in Home Assistant. @malkie-hass @9kon @grantclem

The changes can be tested as follows

Clone Santobert/pybotvac, checkout the branch add_oauth and run setup.py (https://stackoverflow.com/a/41537134)
Visit https://developers.neatorobotics.com/, create a new application and receive client_id and client_secret
Edit sample/sample.py and insert client_id and client_secret
Run sample.py
Enjoy 🎉

malkie-hass · 2020-10-20T16:44:29Z

Hey,
I would try it on the weekend but would need some help. I have no idea how and where to install this.
Using hass on a nuc clone with the premade image snot on docker or something.
No idea if and how I can install this there via pip.
For testing I would rather setup a rpi4 I have as a spare.
Any chance you can maybe give a helping hand. Maybe a remote session or better explanation?

Thanks
Malkie

Santobert · 2020-10-20T17:02:05Z

Any chance you can maybe give a helping hand. Maybe a remote session or better explanation?

Thanks for your answer. You can find me on Discord as Santobert#6222. Don't edit your existing setup. This is not necessary.

malkie-hass · 2020-10-20T17:35:38Z

Sent a friend request on discord.

9kon · 2020-10-20T18:18:28Z

Thanks alot for your work Santobert. I was just trying to log into https://developers.neatorobotics.com/ but cant.

In the new kobold app you dont actually get to set a user password, you put in your email and then you are send a code that you use to log in. I tried loggin out and then asking for a new code to log in, but that code didnt work on there either.

Santobert · 2020-10-31T08:06:24Z

The current situation is as follows:

As far as we now, there is are no credentials for the new kobold accounts. The app gets authenticated via a code that is sent by mail. I don't know the exact process behind this.
Therefore there is no option to obtain OAuth credentials (client_id and client_secret) as well. We can't log into a Webservice to create them and Neatos client_id and client_secret do not work with Vorwerk.
Even if vorwerk still uses beehive/nucleo servers (which can be used with pybotvac), there is no known way to authenticate against them.
I wrote an email to vorwerk support a week ago, but have not received an answer yet.

Tbh, I do not know how to proceed here without further knowledge or documentation.

9kon · 2020-11-02T19:52:51Z

I have been playing around with fiddler to see how the app authenticates and get the credentials.

The app makes a POST call to https://mykobold.eu.auth0.com/passwordless/start with a body that contains {"client_id":"xxxxxx","email":"xx@xx.xx","connection":"email","send":"code"}

This will make the server send the user a code to the email used.

When that code is entered the app makes another POST to https://mykobold.eu.auth0.com/oauth/token with body
{"source":"vorwerk_auth0_international","audience":"https:\/\/mykobold.eu.auth0.com\/userinfo","grant_type":"http:\/\/auth0.com\/oauth\/grant-type\/passwordless\/otp","scope":"openid email profile read:current_user","client_id":"xxx","otp":"CODE RECIVED IN MAIL","country_code":"XX","locale":"xx","username":"xx@xx.xx","prompt":"login","realm":"email","platform":"ios"}

This will return a token that is used for.

https://beehive.ksecosys.com/
and
https://nucleo.ksecosys.com:4443/vendors/vorwerk/robots/

Santobert · 2020-11-02T21:20:25Z

@9kon Thank you. It is good to know what is happening there. The described procedure is more or less an OAuth flow. In fact it seems to be an OpenID Connect flow that extends OAuth2. Even though I don't recognize the passwordless thing.

OAuth: https://darutk.medium.com/the-simplest-guide-to-oauth-2-0-8c71bd9a15bb
OpenID Connect: https://darutk.medium.com/diagrams-of-all-the-openid-connect-flows-6968e3990660

The app makes a POST call to https://mykobold.eu.auth0.com/passwordless/start with a body that contains {"client_id":"xxxxxx","email":"xx@xx.xx","connection":"email","send":"code"}

Unfortunately, we still need a client_id. Usually you can generate a client_id (and optionally a client_secret, which is recommended) via a developer website or similar. I could not find an official way to get one.

malkie-hass · 2020-11-02T21:30:13Z

@9kon could you read the client is clear text when the app sends the data?
If that would be the case can't we use that one? Of course everybody has to get his own.
Sorry, not good at this but willing to help :)
I'm more the network engineer

Santobert · 2020-11-02T22:01:17Z

https://github.com/nicoh88/node-kobold

The issue seems to be solved for the js library. I'll take a look at their solution soon.

@9kon have you captured the same client_id, that is mentioned there? Can you please find differences and similarities between the flow you captured and the one described in the repository?

Santobert · 2020-11-03T17:03:07Z

It's kind of hard for me to work on it because I don't have a Vorwerk bot. If someone else wants to help, don't hesitate.

Things to know:

I couldn't find any python lib that supports the passwordless flow out of the box
The flow is documented here: https://auth0.com/docs/connections/passwordless/guides/email-otp
OIDC Discovery Endpoint: https://mykobold.eu.auth0.com/.well-known/openid-configuration
For the moment I hope that everything else works as usual

Things to do:

Extend the existing PR with the passwordless flow Add OAuth2 stianaske/pybotvac#59 (finished and successfully tested)
Find a way to combine the different flows of neato and vorwerk (maybe finished)
Find a way to outsource token fetching to HA without breaking pybotvac for non-HA users https://developers.home-assistant.io/docs/api_lib_auth/#sync-example-1

Santobert · 2020-11-21T14:30:11Z

Update: The current implementation of pybotvac works so far for Neato and Vorwerk. Furthermore the old authentication via username and password is still possible next to modern OAuth2 (neato) and Passwordless Auth (vorwerk) mechanics. All these features will be available within on of the next releases of pybotvac.

I can't keep working on this issue because I don't know enough about the Auth-Flows of HA and I don't have a Vorwerk robot to do tests. But I offer every possible help for someone who wants to jump in. Since pybotvac works so far with both vendors, it's only the HA integration that is missing.

I've talked to Balloob on Discord. We came to the conclusion that it would be easier to separate Neato and Vorwerk in the future. As a result, Vorwerk would get its own integration to support its own config_flow. A good starting point for a developer would be this documentation: https://developers.home-assistant.io/docs/creating_component_index
Furthermore, the Neato integration will no longer support Vorwerk robots in the future (since this is broken anyways). This is necessary so that we can use the new OAuth2 mechanics for Neato robots.

cobirnm · 2020-12-05T12:46:00Z

Can't we just make a copy of neato component and have it work with Vorwerk? Are so many differences in order to make to component?

Santobert · 2020-12-05T13:10:34Z

Can't we just make a copy of neato component and have it work with Vorwerk? Are so many differences in order to make to component?

In fact, that's the plan. Besides the different auth mechanisms there are no differences. Therefore the config flow and parts of the initialisation are different. The platforms itself (vacuum, sensor, switch, ...) are the same.

cobirnm · 2020-12-05T15:05:31Z

I have a Vorwerk vacuum! Unfortunately don’t know how to code! It’s sad to loose the integration since I have several automations running on it...😤

cobirnm · 2020-12-07T19:56:04Z

So anyone can help vorwerk users or are we just be left without being to use the vacum in home assistant?

malkie-hass · 2020-12-08T18:54:19Z

Hope it is okay to link this here but I think a lot of people might find this here:

https://community.home-assistant.io/t/creation-of-vorwerk-kobold-component/253166

Upvote this for new Vorwerk Integration.

Santobert · 2020-12-16T22:41:33Z

Let's close this issue since #44031 is merged

frenck added the integration: neato label Aug 29, 2020

probot-home-assistant bot assigned dshokouhi and Santobert Aug 29, 2020

dshokouhi added the problem in dependency label Aug 29, 2020

dshokouhi mentioned this issue Aug 29, 2020

Add alert for vorwerk users on the neato integration home-assistant/alerts.home-assistant.io#195

Merged

Santobert mentioned this issue Dec 7, 2020

Add OAuth to Neato #44031

Merged

21 tasks

Santobert closed this as completed Dec 16, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Neato/Vorwerk Integration fails with "Invalid Credentials" #39378

Neato/Vorwerk Integration fails with "Invalid Credentials" #39378

grantclem commented Aug 29, 2020

probot-home-assistant bot commented Aug 29, 2020

dshokouhi commented Aug 29, 2020

dshokouhi commented Aug 29, 2020

grantclem commented Aug 29, 2020

grantclem commented Aug 29, 2020

Santobert commented Aug 29, 2020 •

edited

malkie-hass commented Sep 15, 2020

9kon commented Sep 25, 2020

Santobert commented Oct 10, 2020 •

edited

dshokouhi commented Oct 10, 2020

Santobert commented Oct 11, 2020 •

edited

stianaske commented Oct 12, 2020

Santobert commented Oct 18, 2020 •

edited

Santobert commented Oct 20, 2020

malkie-hass commented Oct 20, 2020

Santobert commented Oct 20, 2020

malkie-hass commented Oct 20, 2020

9kon commented Oct 20, 2020

Santobert commented Oct 31, 2020 •

edited

9kon commented Nov 2, 2020

Santobert commented Nov 2, 2020 •

edited

malkie-hass commented Nov 2, 2020

Santobert commented Nov 2, 2020 •

edited

Santobert commented Nov 3, 2020 •

edited

Santobert commented Nov 21, 2020

cobirnm commented Dec 5, 2020

Santobert commented Dec 5, 2020 •

edited

cobirnm commented Dec 5, 2020

cobirnm commented Dec 7, 2020

malkie-hass commented Dec 8, 2020

Santobert commented Dec 16, 2020

Neato/Vorwerk Integration fails with "Invalid Credentials" #39378

Neato/Vorwerk Integration fails with "Invalid Credentials" #39378

Comments

grantclem commented Aug 29, 2020

Environment

Problem-relevant configuration.yaml

Traceback/Error logs

Additional information

probot-home-assistant bot commented Aug 29, 2020

dshokouhi commented Aug 29, 2020

dshokouhi commented Aug 29, 2020

grantclem commented Aug 29, 2020

grantclem commented Aug 29, 2020

Santobert commented Aug 29, 2020 • edited

malkie-hass commented Sep 15, 2020

9kon commented Sep 25, 2020

Santobert commented Oct 10, 2020 • edited

dshokouhi commented Oct 10, 2020

Santobert commented Oct 11, 2020 • edited

stianaske commented Oct 12, 2020

Santobert commented Oct 18, 2020 • edited

Santobert commented Oct 20, 2020

malkie-hass commented Oct 20, 2020

Santobert commented Oct 20, 2020

malkie-hass commented Oct 20, 2020

9kon commented Oct 20, 2020

Santobert commented Oct 31, 2020 • edited

9kon commented Nov 2, 2020

Santobert commented Nov 2, 2020 • edited

malkie-hass commented Nov 2, 2020

Santobert commented Nov 2, 2020 • edited

Santobert commented Nov 3, 2020 • edited

Santobert commented Nov 21, 2020

cobirnm commented Dec 5, 2020

Santobert commented Dec 5, 2020 • edited

cobirnm commented Dec 5, 2020

cobirnm commented Dec 7, 2020

malkie-hass commented Dec 8, 2020

Santobert commented Dec 16, 2020

Problem-relevant `configuration.yaml`

Santobert commented Aug 29, 2020 •

edited

Santobert commented Oct 10, 2020 •

edited

Santobert commented Oct 11, 2020 •

edited

Santobert commented Oct 18, 2020 •

edited

Santobert commented Oct 31, 2020 •

edited

Santobert commented Nov 2, 2020 •

edited

Santobert commented Nov 2, 2020 •

edited

Santobert commented Nov 3, 2020 •

edited

Santobert commented Dec 5, 2020 •

edited