Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flatten The Curve Web App #911

Open
carlaiau opened this issue Mar 17, 2020 · 33 comments
Open

Flatten The Curve Web App #911

carlaiau opened this issue Mar 17, 2020 · 33 comments

Comments

@carlaiau
Copy link

@carlaiau carlaiau commented Mar 17, 2020

I'm currently working on a web app that is built on top of this data. The ideal user of the app is a non statistically minded person who doesn't fully appreciate how quickly an outbreak can escalate, and has no intuition of non linear growth. "Why is everyone been hysterical, there are only 8 cases" is the common sentence I hear in New Zealand and now there are 12 cases.

App: https://flattenthecurve.co.nz/
Repo: https://github.com/carlaiau/flatten-the-curve

Please let me know any feedback regarding this development and the repo is definitely open to PRs. Frontend is built using GatsbyJS. The app itself can definitely do with some work, I have been hacking it together while working on UI prototyping. Data Processing should be pulling directly from the github hosted CSVs but have needed to double check a few things up until this stage.

Thanks in advance, all feedback appreciated!

Please let me know if I need to include further data accreditation to this site

@Bost
Copy link

@Bost Bost commented Mar 17, 2020

The ideal user of the app is a non statistically minded person

There are too many numbers for such a person. I guess "too many numbers"-feeling sets in every time he/she is presented with 4 or more numbers, and I don't think we can do much about it. We can try to alleviate using graphs and pictures but...

Every tool, technology & way of explanation has it's limits.

@carlaiau
Copy link
Author

@carlaiau carlaiau commented Mar 17, 2020

@Bost in saying that, we could show only one table row for the results instead of 'total' and 'per million' depending on what the user has selected for the sort criteria.

I.E If they're looking for confirmed cases total, we hide the millions row, and vice versa.

@rjblake
Copy link

@rjblake rjblake commented Mar 17, 2020

@carlaiau - Your App does not include 'The Netherlands' (Netherlands, or Nederland) for some reason. Definitely has ~1,400 confirmed cases and population over 3 million

@JiPiBi
Copy link

@JiPiBi JiPiBi commented Mar 17, 2020

As France is not in your app, I supposed the situation improved, I will call President Macron to stop total confinment that began today :-)

Not sure in fact with these curves ......
index

@JiPiBi
Copy link

@JiPiBi JiPiBi commented Mar 17, 2020

I read a bit your code , perhaps the issue for France is that you don't consider it as a group of Provinces like UK , and that you capture a small territory value that is low at the contrary of mainland

@carlaiau
Copy link
Author

@carlaiau carlaiau commented Mar 17, 2020

Thanks for picking those two up @JiPiBi @rjblake I have made my multiple rows per country detection more dynamic and future proofed, Denmark and France are now in the results and live on the site

@TakeItAndRun
Copy link

@TakeItAndRun TakeItAndRun commented Mar 17, 2020

"We want to show when these countries were at a similar level to New Zealand and how their situation has progressed since then.

Each countries progression can be used as a potential forecast for New Zealand's future."

No it cant. It really depends on the number and speed of tests done, the quality of the case tracking and of testing of tracked cases and the support for people who need to go into isolation.
Taiwan, Singapore and South Korea doing a better job than others.
South Korea can track about 80% of infections.

Other countries (like mine: Germany) with a doubling time of about 3 days are not doing such a good job.
see: https://github.com/valeriupredoi/COVID-19_LINEAR

I would suggest to find the doubling time of your country and find an other country with the same doubling time to compare it to.

@carlaiau
Copy link
Author

@carlaiau carlaiau commented Mar 17, 2020

@TakeItAndRun You are correct, the fact that you can compare against lots of historical presents you different potential outcomes. Do you have an alternative proposed wording that could be used instead of the sentence?

Each countries progression can be used as a potential forecast for New Zealand's future.

@onepalone
Copy link

@onepalone onepalone commented Mar 17, 2020

nice, i see a lot of potential in your app.... a bit of simplification and maybe more graphics than tables
But good piece of work.

@carlaiau
Copy link
Author

@carlaiau carlaiau commented Mar 17, 2020

@onepalone Thanks! Love to hear suggestions regarding simplification, and for the graphs I was always intending on making it graphical, but concerned with how this will function on mobile as we have limited horizontal viewport

@carlaiau
Copy link
Author

@carlaiau carlaiau commented Mar 17, 2020

@TakeItAndRun I have restructured the header of the page to highlight some of the issues you were concerned with. If you have some links to further information I am happy to also place these here

@klartext
Copy link

@klartext klartext commented Mar 17, 2020

I like to have these tables with numbers, carefully selected.
Much easier to find out the values than in the JHU-Dashboard.

I don't understand the progression of X based on Y.
Intuitively I would await progression of the country based of the numbers for that country.
But the country selected on top of the page has no progression-Button.
So it's a bit confusing.
Also different dates are used in the higher ranked countries below.
So an eplxanation on your math/method would be fine.

@TakeItAndRun
Copy link

@TakeItAndRun TakeItAndRun commented Mar 18, 2020

@TakeItAndRun You are correct, the fact that you can compare against lots of historical presents you different potential outcomes. Do you have an alternative proposed wording that could be used instead of the sentence?

Each countries progression can be used as a potential forecast for New Zealand's future.

Dear carlaiau

I believe your method is inherently flawed.
You should plot only the recent numbers on a log plot for the country in question.
In all cases, except China and South Korea (as they seem to have the spread under control), you will find a more or less flat line.
Do an exponential regression to find the doubling time.
Use this number for a precise forecast.
If the numbers turn out less than your forecast, then containment measures that were started the mean incubation period earlier are effective.
This was done for some European countries by:

https://github.com/valeriupredoi/COVID-19_LINEAR
(I believe you can even use his code to generate plots for other countries)
Here his graph for Germany:

We are starting a (partial) lock-down in Germany, today. So I would expect a deviation from the strait line in about 5 days.

He even plotted the numbers for China, were you can see a change in the doubling time 4 days after the start of the lock-down:
valeriupredoi/COVID-19_LINEAR#2

@TakeItAndRun
Copy link

@TakeItAndRun TakeItAndRun commented Mar 18, 2020

I did a quick plot for New Zealand (eyeballing in a strait line to fit the last data pints)

Corona Cases for New Zealand

The doubling time is about 2.5 days.
With New Zealand, you don't have much of a statistic, yet.
You also need to know if your country is doing enough tests.

@MerliMomo
Copy link

@MerliMomo MerliMomo commented Mar 18, 2020

Great job so far. It would be great to see the infection rate combined with the population density.

@JimBudde
Copy link

@JimBudde JimBudde commented Mar 18, 2020

My Power BI app already shows country and US state new case volume trends over time. I think that is what you're talking about. You can flip thru various reports at the bottom of the page. It defaults to US states, but you can find country analysis in the earlier pages. I occasionally post observations on my LinkedIn page, https://www.linkedin.com/in/jim-budde-249931/

https://app.powerbi.com/view?r=eyJrIjoiZjM1ODk3NjMtNDJhNi00ZmY1LTg3MjItMjgzYzlhOGEyNzgyIiwidCI6IjRhYTg0YzZkLWU0N2QtNGYwOC05ZDgwLTZiOGY1MGE4ZDdkNCIsImMiOjZ9

@TakeItAndRun
Copy link

@TakeItAndRun TakeItAndRun commented Mar 18, 2020

You also need to have a closer look at the contakt tracing for New Zealand.
If there is travel between Australia and New Zealand, than many cases are imports from Australia.

@carlaiau
Copy link
Author

@carlaiau carlaiau commented Mar 18, 2020

I have added graphs to the site which help communicate the ideas quite a lot better.

There is definitely room for incorporating a solid approach as you're describing @TakeItAndRun

We could then quantify how each country's containment measures are working, as well as project based on doubling-rate. After this is incorporated, we can use the historical comparisons between other countries as just that, comparisons.

Thanks for getting back to me and sharing links of the regression repo. I will spend some time on it later this week. Your feedback has been really helpful 🚀

@valeriupredoi
Copy link

@valeriupredoi valeriupredoi commented Mar 18, 2020

Have a look at this and check out the links within to gitHub repo and data - I am updating these plots daily (and data as well). Also, let me know if you'd like me to run more stats, I am not a virologist 😁 Good luck with your efforts!! 🍺

@carlaiau
Copy link
Author

@carlaiau carlaiau commented Mar 21, 2020

Thanks to everyone's feedback the application has evolved quite substantially from what it was initally.

https://flattenthecurve.co.nz/

Brining in cumulative graphs were a game changer. I am still yet to bring the cumulative components into each countries view but that is the plan. As well as having per region comparisons within each country, if the country infection has spread enough.

@monfortt
Copy link

@monfortt monfortt commented Mar 21, 2020

@tmcklin
Copy link

@tmcklin tmcklin commented Mar 22, 2020

@valeriupredoi
Copy link

@valeriupredoi valeriupredoi commented Mar 22, 2020

@carlaiau - great site, mate! Very easy to see what's going on. One thing: the UK reported deaths is wrong for a few points: I keep the record in a daily updated file here since the way the government agency releases this information is very tricky - they post it on their official site in the daily indicators file but that gets wiped off every day and recreated with the new daily data. Very sneaky if you ask me. Cheers 🍺

@valeriupredoi
Copy link

@valeriupredoi valeriupredoi commented Mar 22, 2020

can cross check against my daily published data here (I ditched the github pages site coz I am bad at html haha) but there you can find the cleaned up data from Johns Hopkins and the official data from the UK - the Johns Hopkins data suffers here and there (repeated time stamps, some weird numbers for UK etc)

@onepalone
Copy link

@onepalone onepalone commented Mar 23, 2020

Carl, i can see how much your app has involved., really well done mate ..
I really like it, you are good on shaping things into simple graphs - people can easily digest it
well done again

@JiPiBi
Copy link

@JiPiBi JiPiBi commented Mar 23, 2020

@carlaiau
Now that France is in your curves , it's almost perfect :)

I made JupyterNotebooks that gave the same visualisations but not so easy to use for everyone ....

Just a remark , the date for France are always wrong because CSSE and worldometers take the official values for France + overseas territories as France and they keep also territories in their data set .
It doesnt change the trends but values are wrong ( around 1% that is not so great)

After saying that , it's sad that so many of us are looking at these desesperating increasing values , without any possible helping action , apart to warn that almost all the countries are on the same trend than Italy (some are even worse like Spain and perhaps UK ) .....

@valeriupredoi
Copy link

@valeriupredoi valeriupredoi commented Mar 23, 2020

After saying that , it's sad that so many of us are looking at these desesperating increasing values , without any possible helping action , apart to warn that almost all the countries are on the same trend than Italy (some are even worse like Spain and perhaps UK )

Very true! You can volunteer to help here https://crowdfightcovid19.org/ (I did, I am not working on this part of my job). BTW the UK is not doing any worse than the rest of Europe, it's just that we should have done something about it sooner, since we had a bit of a delta compared to mainland 😢

@JiPiBi
Copy link

@JiPiBi JiPiBi commented Mar 23, 2020

@valeriupredoi
I joined too this organisation , I don't know if my english or coding level are sufficient , but w'll see..
Thanks

About UK , its not an advice or judgement , it's only on your death curves a sad observation
A week before the end of February , we , in France, had no more hospitalised people for covid , so you were looked at as crazy when you warned about the Chinese curves

image

All countries have issues , here in France there are huge polemics about the avaibility of masks .
On the other hand , for the moment everyone is not convinced that France is following the same trend than Italy in death Evolution . But curves are curves , our values are doubling every 3-4 days .

Alsace is our Lombardy , (as Wuhan area was some weeks ago) same difficulties , saturation of health services , and crual choices to be made ..
After seing military trucks transporting dead people out of Bergamo , I have heard of cuffins that are aligned in the skating rink in Strasbourg because everything is overloaded

@valeriupredoi
Copy link

@valeriupredoi valeriupredoi commented Mar 23, 2020

oh yeah true - I was and still am concerned about high deaths rate but I made two projections: black dotted line keeping the rate from two days ago constant, projected over 10 days, green dotted line - the same but keeping a 0.2 day-1 rate which is more like what we see with quarantined countries https://github.com/valeriupredoi/COVID-19_LINEAR/blob/master/country_plots/COVID-19_LIN_UK_DARK_SIM_UK.png
yesterday's single data point sits nicely on the green line 😃 - it reamins to be seen for the future - London will get hit very badly, unfortunately.
PS - great you joined that, well done! 🍺

@valeriupredoi
Copy link

@valeriupredoi valeriupredoi commented Mar 23, 2020

BTW the black dotted line is absolutely scary but totally possible, so you are quite right in your original post about the UK 👍

@JiPiBi
Copy link

@JiPiBi JiPiBi commented Mar 23, 2020

So many curves are possible with one or two points :-( as you say, wait and see...

@carlaiau
Copy link
Author

@carlaiau carlaiau commented Mar 23, 2020

Thanks everyone for comments and great to see a lot of discussion here.
I'm still working on improving everything so please keep the suggestions coming, or better yet, open issues on the main application repo. 90%+ of the iterative improvements have been base on user feedback.

I have just updated the readme about what I'm planning on doing next.
https://github.com/carlaiau/flatten-the-curve

@carlaiau
Copy link
Author

@carlaiau carlaiau commented Apr 5, 2020

Quite a few changes now, New Zealand, United States, Australia, Canada and China have region based tables and graphing. I have switched the US data to https://covidtracking.com/

I really just need to keep collating different sources of region based data and getting it into the correct structure, then we can start giving lots of users region based views in their countries

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet