Skip to content
This repository has been archived by the owner on Mar 10, 2023. It is now read-only.

Created a JSON-based API. #8

Closed
ExpDev07 opened this issue Feb 10, 2020 · 57 comments
Closed

Created a JSON-based API. #8

ExpDev07 opened this issue Feb 10, 2020 · 57 comments
Labels
user creation User created capabilities

Comments

@ExpDev07
Copy link

ExpDev07 commented Feb 10, 2020

Hello and thank you so much for maintaining this! I have created an API that reads your data and returns it in a way that's more friendly to use in programs. It also supports history. It's still a W.I.P and I would love contributions! It is open-sourced here: https://github.com/ExpDev07/coronavirus-tracker-api. Feel free to use it in your projects!

It is very fast due to caching.

The current endpoints are (more will be added):
https://coronavirus-tracker-api.herokuapp.com/confirmed
https://coronavirus-tracker-api.herokuapp.com/deaths
https://coronavirus-tracker-api.herokuapp.com/recovered

For all of them combined:
https://coronavirus-tracker-api.herokuapp.com/all

@brent-field
Copy link

This is super helpful and works swimmingly. Thanks!

@altezza04
Copy link

Thank you! I start to use it on my site.

@ExpDev07
Copy link
Author

Glad to hear you're using it! @altezza04 . Feel free to share your site :)! I'd love to see it.

@bjarkimg
Copy link

WIP - Power BI
http://bit.ly/38qeMbz

@bjarkimg
Copy link

@ExpDev07 possible to return the date fields unpivoted ?
Can do this in power bi just a bit of hassle.

grate job btw :)

@3verse
Copy link

3verse commented Feb 12, 2020

WIP - Power BI
http://bit.ly/38qeMbz
hi @bjarkimg - nice viz! I created my own on Tableau public (https://tabsoft.co/2OKTU72) but this was before the dataset was moved to github, I'll have work on the datasource again to update this now

@bjarkimg
Copy link

@3verse nice viz :), yea did the google sheet first.. hope this will be stable.

@altezza04
Copy link

Glad to hear you're using it! @altezza04 . Feel free to share your site :)! I'd love to see it.

http://chinavirus.live/
I use it on the world ranking part only, it seem the data is not updated often , but anyway, thanks for the effort!

@bjarkimg
Copy link

FYI added Deaths and Recovered, with Measure select option.
http://bit.ly/38qeMbz

@bjarkimg
Copy link

@ExpDev07 possible to return the date fields unpivoted ?
Can do this in power bi just a bit of hassle.

grate job btw :)

@ExpDev07 Ignore this request, solved in Power BI, auto update should work now. thanks for the API!

@ralyodio
Copy link

Looks to be broken now.

@ExpDev07
Copy link
Author

@chovy it broke when they re-formatted the repo, I've fixed it to point to the correct paths now and it's working again.

@ralyodio
Copy link

awesome. thank you sir.

@bjarkimg
Copy link

FYI
Added Forecast on Page 3
http://bit.ly/38qeMbz

image

@wobsoriano
Copy link

wobsoriano commented Feb 17, 2020

I made a website too with latest tweets, timeline and charts regarding the virus https://the2019ncov.com

Screen Shot 2020-02-18 at 7 51 40 AM

@leopoul
Copy link

leopoul commented Feb 24, 2020

Screenshot 2020-02-24 at 21 25 59

I parse your API endpoint, massage it and throw it in an influxDB. At the same time I build a unique list of locations to feed the Grafana WorldMap plugin served via Flask. Here is a first draft of the Grafana dashboard. I plan to work a bit more around my code and share dashboard, parser and Flask server code.

@iceweasel1
Copy link

@leopoul Grafana and InfluxDB, I love it. Nice work!

@smartcuc
Copy link

Cool, how did you get the data into influxDB?

@ralyodio
Copy link

I noticed a map that has quanrtined data. Can we add that?

@ExpDev07
Copy link
Author

@chovy ill see what I can do!:)

@leopoul
Copy link

leopoul commented Feb 27, 2020

Code is here: https://github.com/leopoul/covid19_grafana . It's a quick and dirty release so feel free to contribute. I transform the data to get it into Influx and extract the unique locations to make it work with the Grafana plugin. I wanted to have something quickly out so there is lots of room for improvement.
grafana_1
grafana_2

@elmortero
Copy link

Great work! Is there a possibility to filter by country in the request?

@ExpDev07
Copy link
Author

Please head over to https://github.com/ExpDev07/coronavirus-tracker-api for any questions/discussions regarding the API. :)!

@Laeyoung
Copy link

Laeyoung commented Mar 4, 2020

Oh nice work!

I did similar thing : )!
https://github.com/Laeyoung/COVID-19-API

@mauro-gobbi
Copy link

Hello, this API is never up to date and not in sync with the daily reports

@ExpDev07
Copy link
Author

ExpDev07 commented Mar 4, 2020

Hello @mauro-gobbi. It caches the daily reports for 1 hourr. So at most, it takes 1 hour for it to show the changes after an update. This is to allow for faster results and avoid having to make calls to GitHub per request. If you fork the API, you can change the cache duration to 30 minutes, or completely remove it if you wish. It's as simply as just changing a variable. Feel also free to open an issue or pull request if you believe it should be lowered. I haven't had any complaints besides yours, so most seem OK with the delay.

@mauro-gobbi
Copy link

mauro-gobbi commented Mar 6, 2020

Hello @mauro-gobbi. It caches the daily reports for 6 hours. So at most, it takes 6 hours for it to show the changes after an update. This is to allow for faster results and avoid having to make calls to GitHub per request. If you fork the API, you can change the cache duration to an hour, 30 minutes, or completely remove it if you wish. It's as simply as just changing a variable. Feel also free to open an issue or pull request if you believe it should be lowered. I haven't had any complaints besides yours, so most seem OK with the delay.
EDIT: It’s cached for 1 hour.

Hi, thanks a lot for your reply and thanks for making this API available. 😊
Sorry but I do not understand, even if you refresh the data every hour, and regardless caching results, isn't the data source (the .csv in "csse_covid_19_time_series") updated only once a day from J. Hopkins?
It's probably my fault but I do not actually understand how to get fresh data more frequently than one time per day.

@ExpDev07
Copy link
Author

ExpDev07 commented Mar 6, 2020

isn't the data source (the .csv in "csse_covid_19_time_series") updated only once a day from J. Hopkins

Yes, you are correct in that. So whenever JSU updates their time series, it takes a maximum of 1 hour and a minimum of 0 seconds for it to reflect in the API (which is why the data may differ). If you're talking about live updates, that's not really a thing. Most sources will give you daily updates.

@mauro-gobbi
Copy link

mauro-gobbi commented Mar 6, 2020

isn't the data source (the .csv in "csse_covid_19_time_series") updated only once a day from J. Hopkins

Yes, you are correct in that. So whenever JSU updates their time series, it takes a maximum of 1 hour and a minimum of 0 seconds for it to reflect in the API (which is why the data may differ). If you're talking about live updates, that's not really a thing. Most sources will give you daily updates.

Ok great, fair enough. Live updates are not possible for sure, I hope they will start to upload a new .csv report at least every 12h. Thanks again 👍

@ExpDevv07 also, it's quite weird, that this branch https://github.com/CSSEGISandData/COVID-19/tree/web-data mentions a crawler to worldometers.info and bnonews.com.

I thought those two websites used to get data from the JHU dashboard, not the opposite (they are fully synchronized indeed).

@Omaroid
Copy link

Omaroid commented Mar 7, 2020

Hello everyone,

I maid an android app calling @ExpDev07 Rest API, check the repo. https://github.com/Omaroid/Covid-19-Android

@CSSEGISandData CSSEGISandData added the user creation User created capabilities label Mar 9, 2020
@MohammedAlrozzi
Copy link

@ExpDev07 thanks for the great work. could you please explain on how to get confirmed cases for example for one single country. how the GET url will look like. Thanks a lot.

@Jeff-Lewis
Copy link

Jeff-Lewis commented Mar 13, 2020

Bjarki @bjarkimg Can you share your Power BI pbix file please? Or is it possible to make the PBI clonable/save as on app.powerbi.com?
I'm wanting to restore County level data and add a Logistics curve chart #637

@ralyodio
Copy link

api missing data again.

@roenw
Copy link

roenw commented Mar 17, 2020

@ExpDev07 Your API works great. I have one hosted locally since I’m developing my own coronavirus dashboard.

Sorry if off topic but typically how often do the numbers presented update? Thank you.

@DevakiTrishani
Copy link

@bjarkimg : I'm new to Power BI, and would like to use the data in this Repo. The problem is I don't know how to connect said data in Power BI.
Please can you help. Sorry for asking on this thread, I have no other means to ask you.
btw, your Dashboards look impressive.
Thank you.

@ExpDev07
Copy link
Author

@roenw Updates whenever this repo updates (with a delay of max an hour for cache to reload) :).

@roenw
Copy link

roenw commented Mar 21, 2020

Thanks @ExpDev07!

@Aurel-PBI
Copy link

Aurel-PBI commented Mar 21, 2020

Hi, bjarkimg

Could you share your pbix file. I would like to learn how you do this amazing dashboards
Thanks again

@Aurel-PBI
Copy link

@ExpDev07 possible to return the date fields unpivoted ?
Can do this in power bi just a bit of hassle.
grate job btw :)

@ExpDev07 Ignore this request, solved in Power BI, auto update should work now. thanks for the API!

Hi bjarkimg
How did u unpivoted the date with PowerBi with data Json AND with an auto update.
I'm realy interested to know.
Thanks again

@DevakiTrishani
Copy link

@ExpDev07 Thank You!

@itsgorain
Copy link

Are individual US states data no longer coming through in recoveries and deaths?

@ExpDev07
Copy link
Author

@itsgorain JHU has dropped tracking of recoveries completely, for confirmed and deaths, they have decided to do U.S.A as a whole instead of individual states. These changes obviously are reflected in the API.

@ajmishra1989
Copy link

ajmishra1989 commented Mar 25, 2020

I'm a beginer here and wanted to store how we can store this data in Google cloud storage first (Using the API). Would be helpful if anyone can help me with the exact steps.

Also, any resources for data related to user sentiments like shopping behavior, spending etc.

Much thanks in advance.

@ExpDev07
Copy link
Author

@ajmishra1989 I'm guessing you'd have to write a script to periodically retrieve the data you want and push it to Google Cloud Storage.

@ajmishra1989
Copy link

@ajmishra1989 I'm guessing you'd have to write a script to periodically retrieve the data you want and push it to Google Cloud Storage.

OK, Thanks. Any possible workaround my second question? I'm looking for place to fetch data for user sentiments around Covid-19.

Thanks.

@ExpDev07
Copy link
Author

I believe there’s some scrapers out there that offers a news feed etc, google around “Covid19 news scraper github” @ajmishra1989

@wobsoriano
Copy link

Good bye recovered data.

@Omaroid
Copy link

Omaroid commented Mar 27, 2020

Good bye recovered data.

They added the recovered data !

@mahdighazali
Copy link

mahdighazali commented Mar 29, 2020

@ExpDev07 possible to return the date fields unpivoted ?
Can do this in power bi just a bit of hassle.
grate job btw :)

@ExpDev07 Ignore this request, solved in Power BI, auto update should work now. thanks for the API!

Hi bjarkimg
How did u unpivoted the date with PowerBi with data Json AND with an auto update.
I'm realy interested to know.
Thanks again

Hi,
Basically:
Move the data columns (before expand) to be the last one,
Create a variable to get all the column names, ( Cols = Table.ColumnNames(#"Reordered Columns"),)

Create a variable all the columns (by excluding the one who are not part of the unpivot) ColsUnPivot = List.Skip(Cols,9),

Unpivot the columns:
Unpivot = Table.Unpivot( #"Reordered Columns",ColsUnPivot,"Attribute","Value"),

Source = Json.Document(Web.Contents("https://coronavirus-tracker-api.herokuapp.com/all")),
#"Converted to Table" = Record.ToTable(Source),
#"Expanded Value" = Table.ExpandRecordColumn(#"Converted to Table", "Value", {"last_updated", "latest", "locations", "source"}, {"Value.last_updated", "Value.latest", "Value.locations", "Value.source"}),
#"Expanded Value.locations" = Table.ExpandListColumn(#"Expanded Value", "Value.locations"),
#"Expanded Value.locations1" = Table.ExpandRecordColumn(#"Expanded Value.locations", "Value.locations", {"coordinates", "country", "country_code", "history", "latest", "province"}, {"Value.locations.coordinates", "Value.locations.country", "Value.locations.country_code", "Value.locations.history", "Value.locations.latest", "Value.locations.province"}),
#"Expanded Value.locations.coordinates" = Table.ExpandRecordColumn(#"Expanded Value.locations1", "Value.locations.coordinates", {"lat", "long"}, {"Value.locations.coordinates.lat", "Value.locations.coordinates.long"}),
#"Expanded Value.locations.history" = Table.ExpandRecordColumn(#"Expanded Value.locations.coordinates", "Value.locations.history", {"1/22/20", "1/23/20", "1/24/20", "1/25/20", "1/26/20", "1/27/20", "1/28/20", "1/29/20", "1/30/20", "1/31/20", "2/1/20", "2/10/20", "2/11/20", "2/12/20", "2/13/20", "2/14/20", "2/15/20", "2/16/20", "2/17/20", "2/18/20", "2/19/20", "2/2/20", "2/20/20", "2/21/20", "2/22/20", "2/23/20", "2/24/20", "2/25/20", "2/26/20", "2/27/20", "2/28/20", "2/29/20", "2/3/20", "2/4/20", "2/5/20", "2/6/20", "2/7/20", "2/8/20", "2/9/20", "3/1/20", "3/10/20", "3/11/20", "3/12/20", "3/13/20", "3/14/20", "3/15/20", "3/16/20", "3/17/20", "3/18/20", "3/19/20", "3/2/20", "3/20/20", "3/21/20", "3/22/20", "3/23/20", "3/24/20", "3/25/20", "3/26/20", "3/27/20", "3/3/20", "3/4/20", "3/5/20", "3/6/20", "3/7/20", "3/8/20", "3/9/20"}, {"Value.locations.history.1/22/20", "Value.locations.history.1/23/20", "Value.locations.history.1/24/20", "Value.locations.history.1/25/20", "Value.locations.history.1/26/20", "Value.locations.history.1/27/20", "Value.locations.history.1/28/20", "Value.locations.history.1/29/20", "Value.locations.history.1/30/20", "Value.locations.history.1/31/20", "Value.locations.history.2/1/20", "Value.locations.history.2/10/20", "Value.locations.history.2/11/20", "Value.locations.history.2/12/20", "Value.locations.history.2/13/20", "Value.locations.history.2/14/20", "Value.locations.history.2/15/20", "Value.locations.history.2/16/20", "Value.locations.history.2/17/20", "Value.locations.history.2/18/20", "Value.locations.history.2/19/20", "Value.locations.history.2/2/20", "Value.locations.history.2/20/20", "Value.locations.history.2/21/20", "Value.locations.history.2/22/20", "Value.locations.history.2/23/20", "Value.locations.history.2/24/20", "Value.locations.history.2/25/20", "Value.locations.history.2/26/20", "Value.locations.history.2/27/20", "Value.locations.history.2/28/20", "Value.locations.history.2/29/20", "Value.locations.history.2/3/20", "Value.locations.history.2/4/20", "Value.locations.history.2/5/20", "Value.locations.history.2/6/20", "Value.locations.history.2/7/20", "Value.locations.history.2/8/20", "Value.locations.history.2/9/20", "Value.locations.history.3/1/20", "Value.locations.history.3/10/20", "Value.locations.history.3/11/20", "Value.locations.history.3/12/20", "Value.locations.history.3/13/20", "Value.locations.history.3/14/20", "Value.locations.history.3/15/20", "Value.locations.history.3/16/20", "Value.locations.history.3/17/20", "Value.locations.history.3/18/20", "Value.locations.history.3/19/20", "Value.locations.history.3/2/20", "Value.locations.history.3/20/20", "Value.locations.history.3/21/20", "Value.locations.history.3/22/20", "Value.locations.history.3/23/20", "Value.locations.history.3/24/20", "Value.locations.history.3/25/20", "Value.locations.history.3/26/20", "Value.locations.history.3/27/20", "Value.locations.history.3/3/20", "Value.locations.history.3/4/20", "Value.locations.history.3/5/20", "Value.locations.history.3/6/20", "Value.locations.history.3/7/20", "Value.locations.history.3/8/20", "Value.locations.history.3/9/20"}),
#"Removed Columns" = Table.RemoveColumns(#"Expanded Value.locations.history",{"Value.source"}),
#"Reordered Columns" = Table.ReorderColumns(#"Removed Columns",{"Name", "Value.last_updated", "Value.latest", "Value.locations.latest", "Value.locations.province", "Value.locations.coordinates.lat", "Value.locations.coordinates.long", "Value.locations.country", "Value.locations.country_code", "Value.locations.history.1/22/20", "Value.locations.history.1/23/20", "Value.locations.history.1/24/20", "Value.locations.history.1/25/20", "Value.locations.history.1/26/20", "Value.locations.history.1/27/20", "Value.locations.history.1/28/20", "Value.locations.history.1/29/20", "Value.locations.history.1/30/20", "Value.locations.history.1/31/20", "Value.locations.history.2/1/20", "Value.locations.history.2/10/20", "Value.locations.history.2/11/20", "Value.locations.history.2/12/20", "Value.locations.history.2/13/20", "Value.locations.history.2/14/20", "Value.locations.history.2/15/20", "Value.locations.history.2/16/20", "Value.locations.history.2/17/20", "Value.locations.history.2/18/20", "Value.locations.history.2/19/20", "Value.locations.history.2/2/20", "Value.locations.history.2/20/20", "Value.locations.history.2/21/20", "Value.locations.history.2/22/20", "Value.locations.history.2/23/20", "Value.locations.history.2/24/20", "Value.locations.history.2/25/20", "Value.locations.history.2/26/20", "Value.locations.history.2/27/20", "Value.locations.history.2/28/20", "Value.locations.history.2/29/20", "Value.locations.history.2/3/20", "Value.locations.history.2/4/20", "Value.locations.history.2/5/20", "Value.locations.history.2/6/20", "Value.locations.history.2/7/20", "Value.locations.history.2/8/20", "Value.locations.history.2/9/20", "Value.locations.history.3/1/20", "Value.locations.history.3/10/20", "Value.locations.history.3/11/20", "Value.locations.history.3/12/20", "Value.locations.history.3/13/20", "Value.locations.history.3/14/20", "Value.locations.history.3/15/20", "Value.locations.history.3/16/20", "Value.locations.history.3/17/20", "Value.locations.history.3/18/20", "Value.locations.history.3/19/20", "Value.locations.history.3/2/20", "Value.locations.history.3/20/20", "Value.locations.history.3/21/20", "Value.locations.history.3/22/20", "Value.locations.history.3/23/20", "Value.locations.history.3/24/20", "Value.locations.history.3/25/20", "Value.locations.history.3/26/20", "Value.locations.history.3/27/20", "Value.locations.history.3/3/20", "Value.locations.history.3/4/20", "Value.locations.history.3/5/20", "Value.locations.history.3/6/20", "Value.locations.history.3/7/20", "Value.locations.history.3/8/20", "Value.locations.history.3/9/20"}),
Cols = Table.ColumnNames(#"Reordered Columns"),
ColsUnPivot = List.Skip(Cols,9),
Unpivot = Table.Unpivot( #"Reordered Columns",ColsUnPivot,"Attribute","Value"),

@mahdighazali
Copy link

Hello and thank you so much for maintaining this! I have created an API that reads your data and returns it in a way that's more friendly to use in programs. It also supports history. It's still a W.I.P and I would love contributions! It is open-sourced here: https://github.com/ExpDev07/coronavirus-tracker-api. Feel free to use it in your projects!

It is very fast due to caching.

The current endpoints are (more will be added):
https://coronavirus-tracker-api.herokuapp.com/confirmed
https://coronavirus-tracker-api.herokuapp.com/deaths
https://coronavirus-tracker-api.herokuapp.com/recovered

For all of them combined:
https://coronavirus-tracker-api.herokuapp.com/all

Hi @ExpDev07 ,
Thank you for the API. It's really helpfull...Please if you can provide me when you refresh the data...It's 2020-03-29 and the last refresh of the data is 2020-03-27 :)
Thank you very much for your help :)

@rolson21
Copy link

Hey All, I am trying to set up a Power BI map for US Counties, but literally have no clue what I am doing. Can anyone give me a quick tutorial to connect this? I was able to get a Shape Map with US Counties, but how to connect the confirmed coronavirus cases to it??

@joefrey
Copy link

joefrey commented Apr 6, 2020

@ExpDev07 Is there any way to update the json files same as the worldometer. I checked the json https://coronavirus-tracker-api.herokuapp.com/confirmed but it's a bit delayed compared to worldometer.

@RafiqCR7
Copy link

RafiqCR7 commented Apr 8, 2020

@ExpDev07 appreciate this but it doesn't look like there is a report available for the 8th yet, however the api is reporting the last updated date on the figures provided as the 8th? Surely it should be reporting these with the last updated date set to the 7th as this is when the figures are from?

@orestis-z
Copy link

The endpoint https://coronavirus-tracker-api.herokuapp.com/all seems to be down at the moment (Status 503 Service Unavailable)
image

@CSSEGISandData
Copy link
Owner

Hello! We are also providing JSON query of the feature layer, such as this one. Thanks!

@orestis-z
Copy link

Is the API still maintained?

@wobsoriano
Copy link

Is the API still maintained?

Nope, looks dead. I am using a csv based daily data https://github.com/bumbeishvili/covid19-daily-data

Using it here https://github.com/wobsoriano/covid3d

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
user creation User created capabilities
Projects
None yet
Development

No branches or pull requests