None of the indicators you requested could be downloaded.

[msgoussi]

Hi...
This code grabs data correctly
WDI::WDI(country = "all", indicator = c("GE.EST", "RQ.EST"),
start = 2000, end = 2020)

However when i change the start year, i receive an error.
WDI::WDI(country = "all", indicator = c("GE.EST", "RQ.EST"),
start = 1990, end = 2020)

None of the indicators your requested could be downloaded. Please verify the arguments of the WDI() function. You can also type a URL of this form in your browser to check if the World Bank web API is currently serving the indicator(s) of interest: https://api.worldbank.org/v2/country/all/indicator/GE.EST?format=json&date=1960:2020&per_page=32500&page=1

Please note that the data before 1996 is not available (https://databank.worldbank.org/reports.aspx?source=1181&series=GE.EST)

Please Advise

[vincentarelbundock]

Clicking on the link shows that the World Bank does not serve the data in question. This either means (a) the indicator code you entered is not valid, or (b) the World Bank does not currently (or ever) supply the given code over the web API.

Unfortunately, this is not something that can be solved by the R package.

[msgoussi]

1. https://api.worldbank.org/v2/country/all/indicator/GE.EST?format=json&per_page=32500&page=1 : works fine

2. https://api.worldbank.org/v2/country/all/indicator/GE.EST?format=json&date=1996:2020&per_page=32500&page=1 : works fine although the is no data in 2019 and 2020

3. https://api.worldbank.org/v2/country/all/indicator/GE.EST?format=json&date=1995:2020&per_page=32500&page=1 : DO NOT work since there is no data in 1995

Can you explain the behavior of API in 2 and 3 !!!!!

[msgoussi]

I sent an email to WB, and they replied back:
"GE.EST comes from the Worldwide Governance Indicators database, which currently has a date range of 1996-2018. This error occurs when you request a date range that precedes the earliest date in the database. This is a quirk of the API that dates before the earliest available year cause errors while later years do not."

So what is the possibility of editing WDI::WDI, so start and end parameters can accept NUL?????

[vincentarelbundock]

WDI(indicator="GE.EST", start=1996)

[vincentarelbundock]

FYI, I just sent this support request to the WB:

Hi,

I maintain a library for the R statistical language to download World Bank data through the API: https://cran.r-project.org/web/packages/WDI/index.html

Until recently, it seemed possible to download any data series by specifying an arbitrary early date (e.g., 1950). The API would then return data only for the dates where they are available (e.g., post-1996).

There seems to have been a recent change to the API, which no longer returns any data if the supplied start date is antecedent to the available start date.

This poses major problems for my package, because it is not immediately obvious which series are available for which date.

Ideally, this would be handled server-side, and the API would automatically set the start date of the series to the first available date if the supplied date is too early. Is that possible?

Alternatively, is there an easy way for me to infer the correct start date of any arbitrary series?

If I can't find a solution to this problem, I will need to discontinue my package, because I am currently getting too many support requests that I can't do anything about.

Thanks for your help!

[msgoussi]

Thanks a lot for your support, you do not need to discontinue your package. Just need to mention this problem in the help dictionary of your function. it's WB's problem not your code.

[vincentarelbundock]

Thanks.

Just trying to blackmail the WB into doing something ;)

[msgoussi]

:)

[vincentarelbundock]

WB response:

Hi Vincent,

I've reported this to our devops team. Unfortunately, the v1 endpoints aren't available anymore so we can't do any before/after testing to confirm the change in behavior. But everyone agrees this is awkward, so they're looking into fixing it as a bug, i.e. make it work as you ideally suggested. I'll keep the ticket open and inform you how things develop. Sorry for the inconvenience.

[msgoussi]

WB response
Hi - following up again. We've looked at this issue more closely, and it appears that this may be fairly new behavior for the API resulting from the upgrade of endpoints from v1 to v2. We're looking to see if we can treat it like a bug and get it fixed so that queries with ranges before the available dates would still return data. I'll keep the ticket open and let you know how it shakes out.

Blackmailing works fine

[etiennebacher]

Hi, I just ran WDI::WDI(country = "all", indicator = "GE.EST", start = 1990, end = 2020) and it correctly returned the data from 1996 to 2019, so I guess the API issue has been fixed by the World Bank.