Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OpenWeatherMap failed to retrieve data, onecall version 3.0 not supported (for new OWM accounts) #5336

Closed
gsagsa99 opened this issue Sep 11, 2022 · 37 comments
Closed

OpenWeatherMap failed to retrieve data, onecall version 3.0 not supported (for new OWM accounts) #5336

gsagsa99 opened this issue Sep 11, 2022 · 37 comments
Assignees
@kiddigital
Labels
enhancement

Comments

@gsagsa99
Copy link

Domoticz Version: 2022.1 (build 14477)
Platform : Windows 10

OpenWeatherMap is not supporting version 2.5 of the 'onecall' API.

I specified the city ID = 6455259 (Paris). The first call (version 2.5) worked fine . The pair Lat/Lon is retrieved.
This pair is used to build version 2.5 of 'onecall' API. This is failing.

Below the log

2022-09-10 11:08:24.727 Debug: Meteo_Local: Got location parameter 6455259
2022-09-10 11:08:24.727 Debug: Meteo_Local: Starting with setting 0, 0, 0, 0
2022-09-10 11:08:24.727 Status: Meteo_Local: Using specified location (Station ID 6455259)!
2022-09-10 11:08:24.728 Debug: Meteo_Local: Get data from https://api.openweathermap.org/data/2.5/weather?id=6455259&appid=f4bc1306c8c0bda6307d0dc8941437a4&units=metric&lang=fr
2022-09-10 11:08:24.903 Status: Meteo_Local: City (6455259)-> Lat/Long = 48.8565,2.3524
2022-09-10 11:08:24.904 Status: Meteo_Local: Started
2022-09-10 11:08:27.907 Debug: Meteo_Local: Get data from https://api.openweathermap.org/data/2.5/onecall?lat=48.8565&lon=2.3524&exclude=minutely&appid=f4bc1306c8c0bda6307d0dc8941437a4&units=metric&lang=fr
2022-09-10 11:08:28.005 Debug: HTTP 401: Unauthorized
2022-09-10 11:08:28.007 Error: Meteo_Local: Error getting http data!

I checked both using a browser : the first one (weather) is OK and the second one fails.

Instead using the version 3.0 of 'onecall' is OK. Below the working URL (just replace 2.5 with 3.0)

https://api.openweathermap.org/data/3.0/onecall?lat=48.8565&lon=2.3524&exclude=minutely&appid=f4bc1306c8c0bda6307d0dc8941437a4&units=metric&lang=fr

Regards
gsagsa99
@kiddigital
Copy link
Contributor

@gsagsa99 , thx for reporting. I will take a look asap...

Did not get/see any deprecation message/e-mail coming by. Also no message on their website as far as I can see.

From 2.5 to 3.0 might be (assuming semantic versioning) a backward compatibility breaking change... hopefully not too much has changed. Would be great if we can find some documentation/changelog revealing the API changes.

@kiddigital kiddigital self-assigned this Sep 11, 2022
@gsagsa99
Copy link
Author

gsagsa99 commented Sep 11, 2022 •
edited

Thanks @kiddigital
Apparently the 2.5 is deprecated for new account . I created my account and got my key one week ago
Found this on the Net
openhab/openhab-addons#13314
openhab/openhab-addons#13374

@kiddigital
Copy link
Contributor

Ah.. ok... thx for this info.. Was just looking at my existing OWM hardware module in Domoticz and that one is working fine.

Also when using my existing API key on a test Domoticz environment with a newly setup OWM module, it works as expected.

So it is most likely as described in the OpenHab issue, newly/recently created OWM API keys only work with the 3.0. I will try to see what happens if we just replace the 2.5 in 3.0... maybe it just works...

@kiddigital
Copy link
Contributor

Did some quick testing... not conclusive yet...

If I switch to use the OneCall 3.0 endpoint, I get a 401: Unauthorized message. I am using the (old) API key that does work when calling the old (2.5) endpoint.

To test I generated a new API key, but that still gives me the 401. But this is likely caused by the API key not being active yet (it give 401 first at the 2.5 weather calls, but these calls do work now. So just have to be patient I think).

At the moment, we abuse the weather data call to do the conversion between Location(s) and its Latitude/Longitude. Maybe it is better to switch to the dedicated GeoCoding API for that purpose.

@gsagsa99
Copy link
Author

gsagsa99 commented Sep 12, 2022 via email

@gsagsa99
Copy link
Author

gsagsa99 commented Sep 12, 2022 via email

@kiddigital
Copy link
Contributor

Hm... can you try to get an API key for the 'free' plan? And check if that one works?

This 'base' plan for the OneCall API seems to be new and previously the OneCall API (before 3? Like the 2.5?) was included as one of the API's available in the free plan (next to the weather, geoCoding, etc. API's).

@gsagsa99
Copy link
Author

gsagsa99 commented Sep 13, 2022 via email

@kiddigital
Copy link
Contributor

Thx again @gsagsa99 , could you try to add the free plan to the account with NO subscription? (Or use a 3rd account) and let me know what that does?

@gsagsa99
Copy link
Author

Hi @kiddigital ,
I am not sure I understand what you are asking.

I have two accounts both are free plans. One has a subscription , the other NO
I did the tests for both.

I am not sure what you are asking ?
Regards

@kiddigital
Copy link
Contributor

Hello @gsagsa99 , maybe we mean the same but call it different.

Currently I have the following billing plans
image

And the following Services
image

When looking at the API keys tab, I have 2 API keys. My old one, generated two year ago or so. And a new one generated a few days ago. But unfortunately, it is not clear which API key works for which API (and version).

I was not sure if your second account (without the OneCall subscription) has which billing plans and Services. Hence my question.

To get a better understanding, I have reached out to their support desk. Let's see what they can tell us.

Sorry that for your situation it does not work yet. If I would change the module now (switch to use the 3.0 OneCall), it will break for the current users.

@gsagsa99
Copy link
Author

gsagsa99 commented Sep 14, 2022 •
edited

Hi @kiddigital
Based on my understanding:

  1. your old key will still work with the version 2.5 call. This is because your key is old (two years)
  2. your new key , I am not sure. Just try manually with a browser to get the answer

I my case , I have two accounts I just created a few days ago

  1. First account with a Billing plan (called Base Plan), one key
    Version 2.5 does not work , version 3.0 works
  2. Second account no billing plan , one key
    None is working

Thanks

@kiddigital
Copy link
Contributor

I my case , I have two accounts I just created a few days ago

  1. First account with a Billing plan (called Base Plan), one key

    Version 2.5 does not work , version 3.0 works

  2. Second account no billing plan , one key

    None is working

Is it possible to have your 2nd account to subscribe to the 'free' plan? (See my screenshot above).

It seems logical that if your 2nd account isn't subscribed to any thing at all, the API key will not provide access to any data.

@gsagsa99
Copy link
Author

Hi @kiddigital

The free plan you are talking is a professional one and is the default. I think there is nothing to subscribe.

I used my account (no subscription) and I could call the following API

https://api.openweathermap.org/data/2.5/forecast?lat=50.7001&lon=4.988987&appid=364d01e717bf215a28cc072aed7234cd&units=metric&lang=fr

This works fine but I don't know if the json output is the same .

It is not the 'onecall' but for professional it is 'forecast'.

It is a version 2.5. The 3.0 does not exist

regards

@kiddigital
Copy link
Contributor

Ok, clear. We do mean the same indeed.

So now we have a decision to make for the future (but I will also await input from OWM if/once they respond to my question):

  • In the 'old' situation: the free professional subscription (= default subscription) gave access to many API's including the OneCall API
  • Now in the 'new' situation: the same free professional subscription still gives access to many API's, but NOT anymore to the OneCall API. The OneCall API now has its own subscription

The current Domoticz OWM modules leverages 2 API's, weather (but for the 'wrong' reason, it is used to retrieve Lon/Lat from a name. Which should now be done using the Geo API!) and OneCall. These 2 API's now need different (or both?) subscriptions.

We should make a choice which (or both) subscription to use for the OWM module.

Looking at what we do/need and how OWM positions its API's, it feels that the OneCall API offers the best 'value for money' and a lot of useful information. It needs just 1 call (hence it being called OneCall :) ) to retrieve all relevant information so the free limit of 1000 per day seems more than adequate (every 10 mins = 6 per hour X 24 hours = 144 per day!).

The Professional subscription on the other hand, gives much more flexibility and some more options like the Geo API, WeatherMaps, WeatherTriggers but also Widgets that could be leveraged in the UI, etc. It would make it possible to create a much more advanced Weather module in Domoticz.

Or maybe we need to split things into 2 independent OWM modules. 1 'simple' one using OneCall API (and its subscription) and an 'advanced' module leveraging the Professional subscription (and levels).

Let's continue this discussion on the Domoticz Slack channel(s) and/or forum.

@kiddigital kiddigital changed the title OpenWeatherMap failed to retrieve data, onecall version 2.5 not supported anymore OpenWeatherMap failed to retrieve data, onecall version 2.5 not supported anymore (for new OWM accounts) Sep 14, 2022
@gsagsa99
Copy link
Author

gsagsa99 commented Sep 15, 2022 via email

@kiddigital
Copy link
Contributor

Thanks

What is next ?. As new customer (and I guess I will not be alone), I am
blocked with this issue.

regards

Yes, unfortunately you are.

But I think you agree changing the existing module now, will cause that existing users functionality becomes broken. That is not an option.

We need to decide a way forward. And find people/time to build it.

Let's continue this discussion on the Domoticz Slack channel(s) and/or
forum.

@gsagsa99
Copy link
Author

gsagsa99 commented Sep 16, 2022 •
edited

Fine with me

As I am really new in this, what do you mean
"
Let's continue this discussion on the Domoticz Slack channel(s) and/or forum "

Which channel . Shall I open an item in the forum ? Which one ?

Sorry but this is the first time for me .
regards

@gsagsa99
Copy link
Author

Hi @kiddigital

I don't know if it is feasible and I don't know the amount of work if would be to a get a 'dedicate' version with version 3.0 I could use ?

Which code need to be recompiled ?

Regards

@gsagsa99
Copy link
Author

Hi

a better solution would be to have the version as a parameter , default being 2.5. Feasible ?

regards

@kiddigital
Copy link
Contributor

First have to test if OneCall 2.5 (actually OWM calls it 1.0) has the same result structure as 3.0.

If those are the same, than creating a 3.0 'version' is relatively easy.

If people have time and ability to test, that would be great. I will do the same when time allows.

@gsagsa99 , could you attach a full output of a call so maybe I can do the same call against 2.5 and run a diff?

@gsagsa99
Copy link
Author

Hello @kiddigital

Please find attached, the query, the response as well as the formatted json.

I you need to change the query (additional params) let me know,

When a "modified version" is available, I am ready to test it even if I am not expert :-)

Full.zip

@gsagsa99
Copy link
Author

gsagsa99 commented Sep 25, 2022 •
edited

hello @kiddigital

I don't know what is going on at OWM site but the good new is that the oncall API 2.5 is WORKING since 23 sep 2022.

As you can see in the following log , during the same day

  1. Morning the call was failing (around 8:22)
  2. In the afternoon the same call was OK (arounf 17:51)

2022-09-23 08:22:53.794 Debug: Meteo_Local: Get data from https://api.openweathermap.org/data/2.5/onecall?lat=48.8565&lon=2.3524&exclude=minutely&appid=f4bc1306c8c0bda6307d0dc8941437a4&units=metric&lang=fr
2022-09-23 08:22:53.905 Debug: HTTP 401: Unauthorized
2022-09-23 08:22:53.905 Error: Meteo_Local: Error getting http data!

2022-09-23 17:51:59.306 Debug: Meteo_Local: Get data from https://api.openweathermap.org/data/2.5/onecall?lat=48.8565&lon=2.3524&exclude=minutely&appid=f4bc1306c8c0bda6307d0dc8941437a4&units=metric&lang=fr
2022-09-23 17:51:59.580 Meteo_Local: General/Text (Weather Description)
2022-09-23 17:51:59.587 Meteo_Local: Temp + Humidity + Baro (TempHumBaro)
2022-09-23 17:51:59.592 Meteo_Local: Temp (Feel Temperature)
2022-09-23 17:51:59.597 Meteo_Local: Wind (Wind)
2022-09-23 17:51:59.601 Meteo_Local: General/Visibility (Visibility)
2022-09-23 17:51:59.601 Meteo_Local: General/Percentage (Clouds %)
2022-09-23 17:51:59.601 Meteo_Local: Rain (Precipitation)
2022-09-23 17:51:59.601 Meteo_Local: Lighting 2 (Is it raining/snowing)

of course the setup did not change in Domoticz. I did not check at the OWM yet to see if something happened: Rollback to previous API ?

The API key I am using in my setup is 'Base Plan'

If I use the API key free (no plan) , it is still failed.

In summary

  1. New account with Base plan is OK,
    https://api.openweathermap.org/data/2.5/onecall?lat=50.7001&lon=4.988987&appid=f4bc1306c8c0bda6307d0dc8941437a4&units=metric&lang=fr ---> OK

  2. New account no Base plan is FAILING
    https://api.openweathermap.org/data/2.5/onecall?lat=50.7001&lon=4.988987&appid=364d01e717bf215a28cc072aed7234cd&units=metric&lang=fr ---> FAILED

This is good new as I am blocked anymore

@gsagsa99
Copy link
Author

gsagsa99 commented Oct 11, 2022 via email •
edited by gizmocuz

@Zaphod-dev
Copy link

Hi, I can confirm that I was unable to display the OWM forecast using domoticz after creating a brand new "free" API key (with the same "Error getting http data" error). After subscribing to the "Base Plan" (and crossing fingers I never reach the 2k calls) I immediatly could display the forecast in domoticz (errors also stopped after a while).
Indeed if the plugin could be made to cater for old & new OWM API keys (without the need for paid plans) it would be such a relief !
Kind regards

@gizmocuz
Copy link
Contributor

gizmocuz commented Nov 22, 2022 •
edited

@gsagsa99 , maybe create a channel OpenWeatherMap ?

@gizmocuz
Copy link
Contributor

As you can see in the code, the API is called every 300 seconds = every 5 minutes
that is 288 calls a day, is 9000 calls a month
I think you should be safe with the free plan?

@DarkPlayerr DarkPlayerr mentioned this issue Dec 23, 2022
Open
@lmarzen lmarzen mentioned this issue Jan 19, 2023
Closed
@pipiche38
Copy link
Contributor

I have a similar issue, via Domoticz I'm unable to get it working and have :

Mar 23 09:07:44 pi3 domoticz[7582]: 2023-03-23 09:07:44.800 Status: OpenWeatherMap: Worker stopped...
Mar 23 09:07:44 pi3 domoticz[7582]: 2023-03-23 09:07:44.802 Status: OpenWeatherMap: Using specified location (Lon lon=5.8333, Lat lat=45.0833)!
Mar 23 09:07:44 pi3 domoticz[7582]: 2023-03-23 09:07:44.802 Status: OpenWeatherMap: Started
Mar 23 09:07:48 pi3 domoticz[7582]: 2023-03-23 09:07:48.056 Error: OpenWeatherMap: Error getting http data!

calling the API with the URL + the right Key works
http://api.openweathermap.org/data/2.5/weather?q=London,uk&APPID=

@pipiche38
Copy link
Contributor

Is there any chance to get this addressed ?

Same problem with Onecall v3., old keys still working.
https://www.domoticz.com/forum/viewtopic.php?p=301207&hilit=openweathermap#p301207

@kiddigital
Copy link
Contributor

See the discussion at the beginnen of this issue.

We can not update the existing module as it will break functionality for users with an older API key.

So someone might have to develop a second OWM module that uses the new OneCall API that could be used by users that have the new API key and subscription.

Once I have to time, I might give it a go but happy if others beat me to it 😁

@gizmocuz
Copy link
Contributor

Why not add a option on the hardware setup page to use V1 or V2, oe maybe it is possible to detect it?
This way we keep the code in the right source file

@patrickschaerer
Copy link

patrickschaerer commented Aug 26, 2023 •
edited

Hi, I can confirm that I was unable to display the OWM forecast using domoticz after creating a brand new "free" API key (with the same "Error getting http data" error). After subscribing to the "Base Plan" (and crossing fingers I never reach the 2k calls) I immediatly could display the forecast in domoticz (errors also stopped after a while). Indeed if the plugin could be made to cater for old & new OWM API keys (without the need for paid plans) it would be such a relief ! Kind regards

this worked for me.
Thanks

@blakeNaccarato blakeNaccarato mentioned this issue Oct 16, 2023
Open
@jcolladosp jcolladosp mentioned this issue Oct 20, 2023
Merged
@nuckle nuckle mentioned this issue Jan 7, 2024
Open
This was referenced Jan 18, 2024
Closed
Closed
@pipiche38
Copy link
Contributor

Why not add a option on the hardware setup page to use V1 or V2, oe maybe it is possible to detect it? This way we keep the code in the right source file

Alternative could be:

e5e3ee5894b15cee562c06cc9735686. -> interpreted as a 2.5 API
V2.5:e5e3ee5894b15cee562c06cc9735686 -> interpreted as 2.5 API
V3.0:e5e3ee5894b15cee562c06cc9735686 -> interpreted as 3.0 API

@pipiche38 pipiche38 mentioned this issue Apr 6, 2024
Closed
@pipiche38
Copy link
Contributor

@gizmocuz see #6053 I did my best to fix this. Hope that is convenient for everyone

@gizmocuz
Copy link
Contributor

gizmocuz commented Apr 6, 2024

@pipiche38 , thanks for this!!! Aldo this works, it might be better to make a dropdown to select the API and put it in one of the free 'mode' fields (0=2.5=default, 1=3.0)
Is it possible to change it like this?

@pipiche38
Copy link
Contributor

then better to find someone who can handle the web part.
I did my best here, sorry

@gizmocuz gizmocuz changed the title OpenWeatherMap failed to retrieve data, onecall version 2.5 not supported anymore (for new OWM accounts) OpenWeatherMap failed to retrieve data, onecall version 3.0 not supported (for new OWM accounts) Apr 7, 2024
@gizmocuz
Copy link
Contributor

gizmocuz commented Apr 7, 2024

OpenWeatherMap API 3.0 is now supported in beta 15932.
You can specify the API version to use in the web interface.
For existing setup this is set to 2.5, but for new setup this is set to 3.0 by default

@gizmocuz gizmocuz closed this as completed Apr 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kiddigital kiddigital

Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@gizmocuz @pipiche38 @kiddigital @Zaphod-dev @patrickschaerer @gsagsa99