Netatmo API Security Update - is the node ready?

[AleksCee]

Have you see the announcement from Netatmo for the API use? Is this node already use this method?

Dear Netatmo developer,
To improve the security of our products, we inform you that the Client Credentials grant type method will be completely removed. It will no longer be possible to authenticate with the username and password of the user. The effective date of this update is October 2022.

From this date, the OAuth2 authorization code flow must be followed for authentication.

You can find details on this method on our website:
https://dev.netatmo.com/apidocumentation/oauth#authorization-code

[altery]

Netatmo talks about removing "Client Credentials Grant", but I suspect they actually mean "Resource Owner Password Credentials Grant". I assume this plugin uses the latter authentication flow, because the user password must be entered in the node configuration and there is no user-interaction for token exchange. So I guess this plugin will cease to work.

[martiko70]

Is there a chance that the Node will be updated to support the new OAuth2 authorization?

[fmarzocca]

@ssadams11 This package uses the "grant type method= password", that will be completely removed.
The new grant type = authorization_code requires interaction from the user, who has to accept the connection in order to retrieve the access_token, and this will be a mess with NodeRed..

[Dunnenj]

As none of the contributors climbed in I'm afraid that the Node will be dead after September 30th.

Looks like we should build something else.

---

John

[jostrasser]

Hi all,
I hope we get this new auth method working...

I get in touch with the Netatmo Dev Support to request some more informations and details about the changes.
I am only a user of this plugin but I am happy to assist to find a solution for it.

There is also another plugin which is having the same situation and facing the same limitations of Node-Red:
guidone/node-red-contrib-netatmo-dashboard#19 (comment)

Here the feedback from the Netatmo Developer Support:

Q:
Hi, "Client credentials grant type" is deprecated what I can read and will be disabled in October 22.
What is the default lifetime of a token?
Is there a way to get an OAuth Authentication with a refresh token which never expires?
How should we authenticate with the API on the server side?
Example: Node-Red cannot store refresh tokens. If a token expires then a manual user interaction is required.

A:
What you can do is setting up a server on your device hosting the code and access it via your smartphone or any display you would have. Your mobile phone has to be on the same network than the device hosting your code. 
The device should then redirect the call directly to https://api.netatmo/com/oauth2/authorize?client_id=''&redirect_uri=''&scope='' and setting the redirect uri as the locale ip of your device 192.168.xx. It does not need to be accessible from the outside but only on your local network. 
On the mobile phone, you'd then be redirected on Netatmo front and it will ask you whether you want to gave access to the API app. When you'll click on 'Yes, I accept', Netatmo servers will send the code to Netatmo frontend which will redirect it to the locale ip from the redirect uri. In the end https://192.168.xx/?code='' will be received by the device hosting your development and will then be able to get the pair of tokens (access tokens& refresh token). 
Note that if you already have a valid refresh and access token, you do not need to redo the access.

Q:
Thanks for the informations. Can you confirm the following: When a local node-red server plugin is authenticated one time (as example via a web frontend auth) , there is no refresh of the token update needed anymore?
So the auth is valid „for ever“? As mentioned, it is not possible to store it somewhere, users has to redeploy the config every time manually. Is there a way to manually create a key pair which never expires via dev.netatmo.com ?
All the users of the plugin are only importing data from their own stations.
Thanks for your help.

A:
At the moment you do not need to recreate a new token, you can refresh it ad ifinitum once you have a valid token.
This is however also prone to change in the future, also to improve the security of the devices.
At the moment I can't confirm if the refresh token, the access token or both will have to be changed every three hours as we are still considering the best option.  
Thank you for your understanding.

I hope this can help somebody.

Thanks JO

[altery]

@ssadams11 This package uses the "grant type method= password", that will be completely removed.
The new grant type = authorization_code requires interaction from the user, who has to accept the connection in order to retrieve the access_token, and this will be a mess with NodeRed..

You can issue an access and refresh token from the netatmo developer portal for your application. If the plugin allowed to input those tokens (instead of user credentials), API access would be possible without user interaction. I think the change would be fairly straightforward to implement.

[fmarzocca]

You can issue an access and refresh tolen from the netatmo developer portal for your application. If the plugin allowed to input those tokens (instead of user credentials), API access would be possible without user interaction. I think the change would be fairly straightforward to implement.

I agree. But this implies to store the refresh_token somewhere and to refresh it when needed. AFAIK Node Red cannot store it.
In addition, even if we fix the possibility to save the refresh token, this method is not guaranteed to work forever, as Netatmo said that "this is prone to change in the future"

[altery]

Interesting, where did you find that information? Refreshing access tokens with refresh tokens is a standard oauth2 process. It is also described in the developer documentation: https://dev.netatmo.com/apidocumentation/oauth#refreshing-a-token

As long as the 'grant' of the application is not invalidated by the user, refreshing should work indefinitely.

[fmarzocca]

Yes, you can refresh indefinitely, but this requires you to store the refresh token each time somewhere.
Concerning Netatmo possibly changing this method in the future, I was referring to the answer to @jostrasser in this thread.

[altery]

Thank you, I missed that note in the post of @jostrasser . I can't imagine that rotating refresh tokens every three hours is feasible for any application...

Anyway, I guess it would be possible to store the (renewed) refresh token in persistent context, though this would require additional configuration. This won't solve the problem that refresh tokens might expire without using them, for example if the flow is stopped, of course.

[altery]

Oh, I just noticed, that the node uses an external lib to access the netatmo api: https://github.com/floetenbaer/netatmo
The library doesn't seem to be in active development anymore. Maybe it would be simpler to replace the library with, for example, https://www.npmjs.com/package/netatmo-nodejs-api ? The latter supports fetching access tokens with a refresh token.

[jostrasser]

Oh, I just noticed, that the node uses an external lib to access the netatmo api: https://github.com/floetenbaer/netatmo
The library doesn't seem to be in active development anymore. Maybe it would be simpler to replace the library with, for example, https://www.npmjs.com/package/netatmo-nodejs-api ? The latter supports fetching access tokens with a refresh token.

Hi @altery
this is a great idea...!

About the information "Netatmo possibly changing this method in the future":
I got this from the Developer Support but right now it is not fact. They are only thinking about it what I can read in my conversation with them. But they have to find a suitable option because as you mentioned, it isn´t feasible for the most of applications. So for now it should be good to only switch the auth-method.

[jostrasser]

Hi all,
latest info from the Netatmo Support:

Our developpement team are going to allow the creation of the refresh and access token directly from the dev.netatmo.com account owning the application. This means that you will only need to manually generate it once, then you will simply be able to refresh it and save the new ones after refresh.

At the moment the token validity is 3 hours.

Don´t know if this is solving the "saving issue of keys in Node-Red"

[fmarzocca]

Don´t know if this is solving the "saving issue of keys in Node-Red"

No. If you read the message from Netatmo Support: "you will simply be able to refresh it and save the new ones after refresh"

[AleksCee]

I have taken a look at the code, the node uses the Netatmo.js packed which uses:

var form = {
    client_id: client_id,
    client_secret: client_secret,
    username: username,
    password: password,
    scope: scope,
    grant_type: 'password',
  };

source https://github.com/floetenbaer/netatmo/blob/481f0652c60d39ea74c29a0e680bded658a98516/netatmo.js#L98

as Post Parameter, so I think this must change to the new grant_type and exchange the username/password with the manuell generated token in the dev.netatmo.com config from the app.
Problem: this is not only impacting this module but also this: http://github.com/floetenbaer/netatmo
The changes above must be done in the Netatmo.js and in this project the gui must extend to take the token from Netatmo.
The refresh is already in the Netatmo.JS project included, only the first „login“ must be changed to the new grand_type and token, I think.

---

Ok I have try this - not so easy at all :-( seams to do much more. :-(

[AleksCee]

OK, another test. I have done a request like in the FAQ - I get a CODE after confirm the access. With this code I can do one request like this:

var form = {
    grant_type: 'refresh_token',
    refresh_token: refresh_token,
    client_id: client_id,
    client_secret: client_secret,
  };

now it's only possible to make requests with token/refresh token - but the netatmo.js is using the same auth-request at every request - so the code at the next call is not valid anymore. So the netatmo.js must do this on other way - and use only token/refreshtoken instant of a full auth-request.
Another problem is the code - after 3h of not refresh the token it must use a new grant-request with manual user interactivity.

So netatmo.js has to improved for this before this node can work again. :-(

[fmarzocca]

the refresh_token received must be saved somewhere, for next request.

[AleksCee]

Not only the refresh also the normal token - but when you stop the requests for more then 3h you need a new grant-code also.

Problem in this node is, every input-trigger creates a new netatmo.js an call AUTH and not reuse the toke/refreshtoken.

[fmarzocca]

Not only the refresh also the normal token - but when you stop the requests for more then 3h you need a new grant-code also.

Absolutely NOT!
As long as you save your code, you can indefinitely make refresh requests, without the need of an auth_code.

[AleksCee]

In my tests I can use the CODE from the grant-request only one time. The token and refresh-token are reusable. We are talking about the same "code"?
And is anyone on the way to update this node to work after end of this month?

[fmarzocca]

The refresh_token returned from you refresh request. You should have a look at Netatmo developer's documentation. - Sent from my mobile phone. Sorry for typos - Il Sab 3 Set 2022, 17:41 Alexander Palm ***@***.***> ha scritto:

…

In my tests I can use the CODE from the grant-request only one time. The token and refresh-token are reusable. We are talking about the same "code"? And is anyone on the way to update this node to work after end of this month? — Reply to this email directly, view it on GitHub <#27 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACBCQ3RB6WAQGXHKX7C3T43V4NWQXANCNFSM55SE5Y2Q> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[AleksCee]

@fmarzocca yes, the refresh_token can reuse but I talk about the CODE from the Documentation your revert:

Step 3 - The user is redirected to your application

The user is now redirected to the callback URL defined in your application settings or to the redirect_uri provided in the request. If they authorized the application they will be redirected with additional parameters :

[YOUR_REDIRECT_URI]?state=[YOUR_STATE_VALUE]code=[NETATMO_GENERATED_CODE]

This works only one time.

[fmarzocca]

Yes. You don't need to call it more. - Sent from my mobile phone. Sorry for typos - Il Sab 3 Set 2022, 17:55 Alexander Palm ***@***.***> ha scritto:

…

@fmarzocca <https://github.com/fmarzocca> yes, the refresh_token can reuse but I talk about the CODE from the Documentation your revert: Step 3 - The user is redirected to your application The user is now redirected to the callback URL defined in your application settings or to the redirect_uri provided in the request. If they authorized the application they will be redirected with additional parameters : [YOUR_REDIRECT_URI]?state=[YOUR_STATE_VALUE] *code=[NETATMO_GENERATED_CODE]* This works only one time. — Reply to this email directly, view it on GitHub <#27 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACBCQ3QWT2IN53EZ6ZLI6W3V4NYIPANCNFSM55SE5Y2Q> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[altery]

@AleksCee with all due respect, your contributions to this thread are not very constructive for resolving this issue. Please familiarize yourself with the concepts in OAuth2 (e.g. https://oauth.net/2/grant-types/authorization-code/ ). I think the problem domain is clear, and as long as netatmo does not expire refresh-tokens, we're all set.

[jostrasser]

I am also a bit worried that the plugin will stop working shortly.

No cotributor has joined the discussion yet. It would be good to get a feedback when this problem will be addressed. Otherwise, many users will have to think of an alternative quickly. In my case there are many dependencies to the entire home automation.

[AleksCee]

@altery sorry for this - but I had see the node from this guy https://github.com/csuermann/node-red-contrib-virtual-smart-home and he has implemented the whole oauth2 process in node-red - with grant and code requests. So this was about my testing and documentation. But I understand and will be quit for now. Hope someone fix this helpful node.

[altery]

No worries, just don't want to use this thread to discuss the oauth2.0 standard ;)

Anyway, I quickly forked and patched both https://github.com/floetenbaer/netatmo and https://github.com/ssadams11/node-red-contrib-netatmo to see how that goes. Good news is, the change is quite straight forward and seems to work so far:

The bad news is, the token that you can generate from the netatmo developer portal (https://dev.netatmo.com/apps/) is an access token. The portal does not display the refresh token. This makes it cumbersome for users to get a refresh token. One way to obtain a refresh token without implementing a custom backend is by using postman, which supports oauth2.0 auth flows (by implementing a custom protocol handler):

If i got time, i'll push the fork to github, so that at least we got a (dirty) option in case netatmo really disables password grant in october. I also placed an inquiry to netatmo developer support, if they could postpone the timeline for this change.

[AleksCee]

For this problem (Token) I have make a Flow in 2 Steps - first generate the URL for the Auth-Request and show it in debug-output. After copy this URL in the Browser an send it, the second has a http-in which received the code and post it to the "toke-get-request". Only the client_id and client_secret must set in the Injection node.

Should I post the export or is the Postman way more easy?

Here it is, if anyone like to use this direct in nodered without other tools.

[
    {
        "id": "4c7abcf5eaf7e6ac",
        "type": "tab",
        "label": "Netatmo get Token",
        "disabled": false,
        "info": "",
        "env": []
    },
    {
        "id": "522dca94a0793db9",
        "type": "inject",
        "z": "4c7abcf5eaf7e6ac",
        "name": "getNewToken",
        "props": [
            {
                "p": "payload.client_id",
                "v": "",
                "vt": "str"
            },
            {
                "p": "payload.client_secret",
                "v": "",
                "vt": "str"
            },
            {
                "p": "payload.scope",
                "v": "read_station read_thermostat write_thermostat read_camera write_camera access_camera read_presence access_presence read_smokedetector read_homecoach",
                "vt": "str"
            },
            {
                "p": "payload.redirect_uri",
                "v": "http://localhost:1880/netatmoCallback",
                "vt": "str"
            },
            {
                "p": "payload.state",
                "v": "localstate",
                "vt": "str"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "topic": "",
        "x": 130,
        "y": 80,
        "wires": [
            [
                "255e1a5e9dc7e6ad"
            ]
        ]
    },
    {
        "id": "d40f33c144cda375",
        "type": "http in",
        "z": "4c7abcf5eaf7e6ac",
        "name": "Callback",
        "url": "/netatmoCallback",
        "method": "get",
        "upload": false,
        "swaggerDoc": "",
        "x": 100,
        "y": 160,
        "wires": [
            [
                "0ee8f75ae622e73e",
                "511f0a32dc2b9dea"
            ]
        ]
    },
    {
        "id": "255e1a5e9dc7e6ad",
        "type": "change",
        "z": "4c7abcf5eaf7e6ac",
        "name": "Copy",
        "rules": [
            {
                "t": "set",
                "p": "netatmo",
                "pt": "flow",
                "to": "payload",
                "tot": "msg"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 310,
        "y": 80,
        "wires": [
            [
                "a31f79ee97f9d351"
            ]
        ]
    },
    {
        "id": "9689aa8e1e876c79",
        "type": "debug",
        "z": "4c7abcf5eaf7e6ac",
        "name": "link",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "payload",
        "targetType": "msg",
        "statusVal": "",
        "statusType": "auto",
        "x": 650,
        "y": 80,
        "wires": []
    },
    {
        "id": "65561978886a6650",
        "type": "debug",
        "z": "4c7abcf5eaf7e6ac",
        "name": "Token",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "payload",
        "targetType": "msg",
        "statusVal": "",
        "statusType": "auto",
        "x": 990,
        "y": 160,
        "wires": []
    },
    {
        "id": "a31f79ee97f9d351",
        "type": "function",
        "z": "4c7abcf5eaf7e6ac",
        "name": "make link",
        "func": "msg = {\n    payload: 'https://api.netatmo.com/oauth2/authorize?'\n    +'client_id='+msg.payload.client_id\n    +'&redirect_uri='+msg.payload.redirect_uri\n    +'&scope='+msg.payload.scope\n    +'&state='+msg.payload.state\n};\n\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 480,
        "y": 80,
        "wires": [
            [
                "9689aa8e1e876c79"
            ]
        ]
    },
    {
        "id": "0ee8f75ae622e73e",
        "type": "http response",
        "z": "4c7abcf5eaf7e6ac",
        "name": "",
        "statusCode": "200",
        "headers": {},
        "x": 100,
        "y": 220,
        "wires": []
    },
    {
        "id": "511f0a32dc2b9dea",
        "type": "change",
        "z": "4c7abcf5eaf7e6ac",
        "name": "copy code",
        "rules": [
            {
                "t": "set",
                "p": "netatmo.code",
                "pt": "flow",
                "to": "payload.code",
                "tot": "msg"
            },
            {
                "t": "set",
                "p": "payload",
                "pt": "msg",
                "to": "netatmo",
                "tot": "flow"
            },
            {
                "t": "set",
                "p": "payload.grant_type",
                "pt": "msg",
                "to": "authorization_code",
                "tot": "str"
            },
            {
                "t": "delete",
                "p": "payload.state",
                "pt": "msg"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 330,
        "y": 160,
        "wires": [
            [
                "ed818eaf2166a832"
            ]
        ]
    },
    {
        "id": "fd7cc7b27e91b653",
        "type": "http request",
        "z": "4c7abcf5eaf7e6ac",
        "name": "Get Token",
        "method": "POST",
        "ret": "obj",
        "paytoqs": "ignore",
        "url": "https://api.netatmo.com/oauth2/token",
        "tls": "",
        "persist": false,
        "proxy": "",
        "insecureHTTPParser": false,
        "authType": "",
        "senderr": false,
        "headers": [],
        "x": 770,
        "y": 160,
        "wires": [
            [
                "65561978886a6650"
            ]
        ]
    },
    {
        "id": "ed818eaf2166a832",
        "type": "function",
        "z": "4c7abcf5eaf7e6ac",
        "name": "make request",
        "func": "\nmsg = {\n    headers: {\n        'content-type': 'application/x-www-form-urlencoded',\n        'charset': 'UTF-8'\n    },\n    payload: {\n        'grant_type': 'authorization_code',\n        'client_id': msg.payload.client_id,\n        'client_secret': msg.payload.client_secret,\n        'code': msg.payload.code,\n        'redirect_uri': msg.payload.redirect_uri,\n        'scope': msg.payload.scope\n    }\n}\n\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 540,
        "y": 160,
        "wires": [
            [
                "fd7cc7b27e91b653"
            ]
        ]
    }
]

[fmarzocca]

Should I post the export or is the Postman way more easy?

Your flow is way more easy than Postman, and much more consistent. You could save the refresh_token in a global context.

[jostrasser]

Hi @altery

Do you have any chance to publish your patched version? Or do you think https://github.com/ssadams11/node-red-contrib-netatmo will be patched until Netatmo is disabling the old auth method?

October is near... :/

Thanks!

[altery]

Unfortunately, I didn't get around to continue working on it. I could publish a fork of the netatmo library, but the forked library and additional changes to this node would have to be integrated by a contributor of this node (else, all existing flows would have to be recreated with the patched node, which I don't think is feasible). Any contributor active, @floetenbaer , @osos , @ssadams11 , @sadmin91 , @rainerCH that would be willing to do that?

[JesusRosaB]

No te preocupes, simplemente no quiero usar este hilo para discutir el estándar oauth2.0 ;)

De todos modos, rápidamente bifurqué y parcheé tanto https://github.com/floetenbaer/netatmo como https://github.com/ssadams11/node-red-contrib-netatmo para ver cómo va eso. La buena noticia es que el cambio es bastante sencillo y parece funcionar hasta ahora:

La mala noticia es que el token que puedes generar desde el portal de desarrolladores de netatmo (https://dev.netatmo.com/apps/) es un token de acceso. El portal no muestra el token de actualización. Esto hace que sea engorroso para los usuarios obtener un token de actualización. Una forma de obtener un token de actualización sin implementar un backend personalizado es utilizando postman, que admite flujos de autenticación oauth2.0 (al implementar un controlador de protocolo personalizado):

Si tengo tiempo, empujaré el bifurcación a github, para que al menos tengamos una opción (sucia) en caso de que netatmo realmente desactive la concesión de contraseña en octubre. También hice una consulta al soporte para desarrolladores de netatmo, si podían posponer el plazo para este cambio.

Do you upload this version to use the refresh token in the netatmo node?

[jostrasser]

Hi all,
all the workarounds to get the refresh token are no longer needed.

https://dev.netatmo.com is now delivering both tokens.

[JesusRosaB]

Hi all, all the workarounds to get the refresh token are no longer needed.

https://dev.netatmo.com is now delivering both tokens.

But, how many time is this token valid? And, the node is not prepared for input the refresh token and auto refresh it every x hours, isn't it?

[jostrasser]

Hi all, all the workarounds to get the refresh token are no longer needed.

https://dev.netatmo.com is now delivering both tokens.

But, how many time is this token valid? And, the node is not prepared for input the refresh token and auto refresh it every x hours, isn't it?

Correct, the access token must be refreshed periodically by the plugin and the plugin must be updated but no contributor is joining us.

[jostrasser]

Hi all,
a quick update from my side.
As we know, Netatmo didn't shutdown the old auth method yet.
I was wondering so I requested a quick update from them.

I only want to inform about the latest information from Netatmo.

[jostrasser]

A bad sign for this plugin:

Authentication update

Dear Netatmo developer,
As of today, when you refresh an Access Token using the associated endpoint https://api.netatmo.com/oauth2/token, Netatmo servers respond with a couple of tokens : an Access Token and a Refresh Token.

If the previous Access Token is still valid, the newly returned access token is identical but its expiration time is extended for 3 hours.

In any case, the refresh token is not renewed.

Starting from the 17/04/2023, this behavior will change to to be compliant with the recommendations of the RFC of the OAuth2 Authorization Framework (section 10.4) and improving the security of the data of our users.

When refreshing tokens, Access Token and Refresh Token will be automatically renewed and former tokens invalidated.

 

What does it means for me ?

If you were already updating the tokens provided when refreshing your tokens, this change will not impact you.

If you do not update the refresh token when refreshing your Access Token, your users will be disconnected after 3 hours as the former tokens will become invalidated.

To fix it, you need to update the tokens as soon as you get the newly generated ones.

 

Sincerely,

Legrand - Netatmo - Bticino

[S474N]

Will be fix for this? :(