Recovered Data Issue

[GunjanChugh]

Recovered data is not available for 24/03/2020.

[ahmadsyamim]

{
      "date": "2020-3-24",
      "confirmed": 123,
      "deaths": 5
    }

[nicolaschenet]

Coming from CSSEGISandData/COVID-19.

It’s due to the deprecations on the data files in the time series folder. They provided new files for deaths and confirmed cases but nothing for recovered.

However, the updated data is available in the daily reports folder. We should switch the data source to these daily reports to stay accurate, but this is a massive change!

[GunjanChugh]

Are you planning to do the required changes? If yes, what is the estimated time?

[nicolaschenet]

Let's @pomber respond to this one :)

[nicolaschenet]

A little bit more info: according to https://github.com/CSSEGISandData/COVID-19/blob/master/csse_covid_19_data/csse_covid_19_daily_reports/03-24-2020.csv, here are the numbers for France for example:

Confirmed	Deaths	Recovered	Active
22304	1100	3281	17923

---

The figures for confirmed and deaths are a bit different than the ones located in

const FILENAME_CONFIRMED = "time_series_covid19_confirmed_global.csv";
const FILENAME_DEATHS = "time_series_covid19_deaths_global.csv";

which I find a tad weird though...

[chandan-reddy-k]

Yes, @pomber my charts are also broken due to this. Can you please look into this? 😅

[RemDelaporteMathurin]

Mine are as well ! 😃

[bauefi]

Same here 😃

[agaktr]

yeah, made a fallback in my code but please inform us when its fixed! you can as a fallback just put the previous day recovered in the variable so we do not have a problem!!

[GunjanChugh]

@agaktr I adopted for same fallback mechanism. The code application will not break by this but the data will be outdated.

[chandan-reddy-k]

@agaktr I did the same thing but if the country has more recovered cases today then the previous day, it won't be updated and will lead to false calculations.

[pcmnac]

did the same as @agaktr and the others.

reference for those using js
https://github.com/dsplay/covid19-ds-panel/blob/master/src/util/series.js#L40

[RemDelaporteMathurin]

Hi @pomber, can this issue be solved by any mean ?

[pomber]

As far as I understand the changes from JHU CSSE, they have stopped publishing the recovered cases (at least for some countries). That's why you'll see nulls in the recovered property in the JSON.

I don't think using the last known number of recovered cases as a fallback instead of null is a good idea (for this repo, in your app do as you wish). I know nulls can break some apps that weren't expecting it, but showing false numbers without knowing is usually worse.

However, the updated data is available in the daily reports folder. We should switch the data source to these daily reports to stay accurate, but this is a massive change!

I'm not sure why they (JHU CSSE) are still posting some recovered numbers in the daily reports and not updating the time-series, but they still haven't done some of the changes they announced (adding testing_global.csv, adding the country ISO code). Also, they said "No recovered cases will be reported in the daily reports".

And if you ask me, they are doing an outstanding job collecting the data, but they are not handling the breaking changes very well.

[RemDelaporteMathurin]

I agree that the "last known number" isn't a good solution at all and breaks the calculations. So if I understood correctly, we shall wait for changes from JHU CSSE ?

[pomber]

we shall wait for changes from JHU CSSE ?

Yes. Meanwhile, make sure your apps support null recovered cases.

You can also read the whole thread or open issues for clarifications on the JHU repo.

[chandan-reddy-k]

@pomber is there a way to get active cases from the source ?

[ExpDev07]

@chandan-reddy-k confirmed - recovered - deaths should give active, but now that recoveries are not available, not really :/.

[chandan-reddy-k]

@ExpDev07 Yup that’s how I was doing but what I meant is CSSE is giving out daily reports and it has both active and recovered counts. So is there a way we can parse that file and get the values ?

[ExpDev07]

@chandan-reddy-k youd have to build a scraper that takes the daily reports and puts them in a queryable database. It’s a heavy task and not something you wanna do when just starting the app. This is why the timeline files were so great, you just make 3 HTTP requests, parse the CSV and you’re done. No scraping needed, it’s something you can do on the run (given you cache the results).

[Mikeaat]

JHU CSSE added a new File for recovered in the time_series.
Now these 3 are available:
time_series_covid19_confirmed_global.csv
time_series_covid19_deaths_global.csv
time_series_covid19_recovered_global.csv

[ExpDev07]

There’s still some issues with the recovery file. I have no idea what they’re even doing at this point. Contradicting announcements, format changes every day almost...

[awaisahmed119]

All recovered data is null now.

[nicolaschenet]

@awaisahmed119 A fix is on the go

[pomber]

All recovered data is null now.

See #74

Should be fixed now.

[azrindipu]

API gives null value in "recovered" field at date "2020-3-25" for all countries.

[pomber]

API gives null value in "recovered" field at date "2020-3-25" for all countries.

Yep, that's because there isn't data reported for that day in the source csv.

[chandan-reddy-k]

@pomber @ExpDev07 looks like the source has reported recovered cases today, including the previous days.