Questions, discussions

[ivan-aksamentov]

COVID-19 scenarios

Questions, discussions

---

How to navigate

📚 Please read this first
🐞 If you want to report a bug or to suggest an enhancement
❔ If you have a specific question (algorithm, theory, development)
🖥️ If you want to contribute code and data

---

Post a message below If you are not sure which category your question or discussion belongs to or if you want to:

• ask a general question
• discuss the project in general
• thank people working on the project

In all cases, the text of the message can be styled with Markdown

---

We welcome contributions in any form, as well as suggestions and constructive criticism. We'd be happy to hear about your use-cases and workflow. Don't hesitate!

[rfonseca]

Hi, what data do we need to get to add a country to the tool? I'm thinking of Brazil, but this is a general question.

[ivan-aksamentov]

@rfonseca
Hi Rodrigo, that would be fantastic!

We are currently trying to make data pipeline more convenient for external contributors.

The work is done on branch feat/crowdsource-data-submodule

This include:

• "crowdsource"- allow people to easily add and modify the data
• "submodule" - move intermediate data into a separate repository (it's here: covid19_scenarios_data and add it as a submodule of the main repo
• prepare a guide for data contributors and curators
• add people willing to curate the data for their chosen country as maintainers to the data repository

More info is coming, but you may take a look on the mentioned branch at:

• the /tools directory
• the /data directory, which is a pointer to the covid19_scenarios_data repository

Basically, the python scripts in /tools download the data from all the different sources and transform it into an intermediate format (TSV currently). These intermediate files are committed into the covid19_scenarios_data repository. These are then taken on every build, and consumed by the application.

I created an issue for contributor's guide:

• https://github.com/neherlab/covid19_scenarios/issues/36

Related:

• https://github.com/neherlab/covid19_scenarios/issues/27

[kmid5280]

Hello. I was attempting to explore your application but am having trouble starting it. I followed the installation steps as outlined in the readme. When I run 'yarn dev', I get an error in the console that says 'error Command "dev:nowatch;" not found. Did you mean "dev:nowatch"?' I'm running Windows 10 and installed yarn version 1.22.4 using the downloadable .msi file from the yarn site. Yarn is showing as installed, and typing 'yarn --version' in bash reveals the correct version number. The Windows path variable matches the installation folder. Not sure if this is an issue on my end with the yarn installation itself, or if something is missing in the code. Any tips appreciated!

[ivan-aksamentov]

Hi @kmid5280 ,

All of us currently running Linux and there are few Unix-isms still in the config.

I created issue to support Windows: #38

Meanwhile you may want to take a look at the chain starting from dev command in package.json. It refers to nodemon config file here: config/nodemon/dev.json. and the executes the exec line. Maybe Windows does not like the ; characters in there? If I remember correctly, && parts should be fine.

We would appreciate if you could help!

Also, I heard using WSL should make things easier for Windows devs now.

We also have a docker-compose file in docker/ directory.
You may try to install docker and docker-compose, and then to run from the project root:

docker-compose -f docker/docker-compose.dev.yml up --build

That should run yarn dev on Alpine Linux inside a docker container. Port mapping should allow a browser on Windows to access port 3000 inside the container.

Let me know if any of this helped.

[kmid5280]

@ivan-aksamentov Thanks for the response. I have Docker installed already, so I attempted this method using the above command. It gave me the error 'error An unexpected error occurred: "EIO: i/o error, scandir '/code/node_modules/node-sass/node_modules/glob-parent'".

I thought the same thing you did re: the exec string in the dev.json file, in that it may be a syntax problem. Any suggestions on how to rewrite it? Perhaps there is a way to separate out the 'yarn install' and 'yarn dev:nowatch' sections without using the && operator.

[ivan-aksamentov]

@kmid5280
The docker error is strange. Try to remove node_modules .build and .cache directories and try again.

Without docker, for the time being you can try to run:

yarn install

cross-env NODE_ENV=development BABEL_ENV=development babel-node node_modules/.bin/webpack-dev-server --config config/webpack/webpack.client.babel.ts

Alternatively, try to run all these commands in exec string separately. dev:nowatch is just another script in package.json (it runs the second command above).

[kmid5280]

@ivan-aksamentov Running 'cross-env NODE_ENV=development BABEL_ENV=development babel-node node_modules/.bin/webpack-dev-server --config config/webpack/webpack.client.babel.ts' produces the error ''babel-node' is not recognized as an internal or external command, operable program or batch file.' Adding the babel-node package via 'yarn add babel-node' doesn't change the error.

Running 'yarn dev:nowatch' produces the error 'node_modules.bin\webpack-dev-server:2
basedir=$(dirname "$(echo "$0" | sed -e 's,\,/,g')")
^^^^^^^
SyntaxError: missing ) after argument list'

If I delete the node_modules and .cache folders (I don't have a .build folder) and then run docker-compose -f docker/docker-compose.dev.yml up --build, then it hangs at the message 'Attaching to covid19_scenarios-dev-web' before timing out.

[alfredorevilla]

Hi @ivan-aksamentov, can you provide a couple of samples of Unix-isms that Windows cannot digest as mentioned in #38 ?

Regards.

[ivan-aksamentov]

@alfredorevilla I think it may be these semicolons in the nodemon's exec:

covid19_scenarios/config/nodemon/dev.json

Line 32 in 00a3b2f

"exec": "node -e 'console.clear()'; yarn dev:clean; yarn install && yarn dev:nowatch; true"

Maybe there's something else. But I cannot investigate this right now.
Also, check the error messages @kmid5280 encountered above.

[ribrault]

Hi,
thanks a lot for this web app, very useful indeed.
From what I got from the severity assumptions table, ca. 90% of severe infections occur in people aged over 60. I'd be very interested in seeing what happens in a scenario where this age class (>60), as a population specifically at risk, would be "confined" for their protection, while the remaining would not be. (such a scenario would certainly bring ethical and practical questions, yet so does a scenario with no age-dependent restrictions).
I'm sorry I'm not able to code it myself, and I thank you in advance for your answer.
Kind regards,

[rneher]

@ribrault This is possible. there is a column isolation in the severity table at the bottom left in which you can enter values that specify how much different age groups are separated from the bulk of the population

[heitzk]

@rneher for the column isolation, it is assumed that the age group is isolated from the bulk of the population from before the start of the outbreak, correct? Is it possible to input isolation as a mitigation strategy as part of the model at a later part of the outbreak? Thank you for the clarification and for this helpful tool!

[ribrault]

Indeed. Thanks. I was using a web browser (Safari) where editing of this table was not supported. Adding a warning when parameter editing is not supported by the browser would be helpful.

On the simulations: it looks like confining the population at risk (e.g. > 60 yo) while partly releasing confinement of the rest of the population allows to avoid ICU overflow and to importantly reduce the total number of deaths… do you know whether this strategy has been considered in practice and do you have any opinion on that ?

Also, please tell me which channel is most appropriate to discuss that, if not here…

[darwinbandoy]

My computed reproductive number is 2.5 for the Philippines with temperatures around 28 degree celsius. I don't this will share the seasonality of other diseases. The default seasonality I suggest should be zero.

[DanielZug]

@ribault, I had very similar thoughts and it's great to have this simulation tool available to play with! I'm quite unhappy about the prospect of living / working with the restrictions in place for many months to come and desperately look for an exit strategy. (Of course, in the current situation, we must at first get the outbreak under control before anything else.)
One important question for me is how to ensure that the virus is not transmitted by health care staff that is taking care of the elderly people. Actually one would not only have to isolate the >60 year olds and other persons at risk from the rest of the population, but also their regular contacts, unless the risk of transmissions can be safely reduced.
And then, of course, it's very important to validate the assumptions or parameters of the model with reality before starting a large-scale experiment!
In my view, we need to learn to live with the virus, like any other, we won't be able to eliminate it worldwide. So we need to ban the risk of an epidemic that threatens large parts of our population every few months. It would be interesting to hear what scenarios scientists and polticians are considering for this and on what basis decisions will be made. (Edit: As long as their is no vaccination available.)

[jc92]

Apologies for opening a ticket, just discovered this thread.

I'm trying to find granular data on confirmed cases, but haven't been successful to do so until now. Could you please share how you are getting the numbers from BAG?
Thanks in advance.

[codycotner]

One more question - do the length of stay assumptions of 3 days hospitalized and 14 days in the ICU come from a particular paper? The 3 days hospitalized seems particularly optimistic compared to other models. Thanks

[cbird808]

I have 3 simple questions.

1. How is the start of the Simulation time range being determined for USA-Texas (or any other scenario)?

2. How is the Initial number of cases being determined for USA-Texas (or any other scenario)?

3. If I were to estimate these two parameters for another location, would I have to consider any additional information above an beyond the answers to the first 2 questions?

Using Scenario USA-Texas as an example, the model starts 18 Feb 2020, 2 weeks prior to the first reported case on 03 Mar 2020, with an initial number of cases of 18. If I change the sim to start on 03 Mar with 1 case, the actual values, the result is completely different. I conclude from this that the 'initial number of cases` is actually the initial number of infections in the whole population.

any guidance is much appreciated

[nnoll]

Hey @cbird808. Sorry for the confusion! Both the start of Simulation time range and the Initial number of cases are estimated from the empirically provided data. The algorithm we utilize is as follows:

We optimize the predictions of the model to fit the cumulative case counts and fatalities to minimize "residuals" for early times before mitigation procedures were enacted. We estimate the following 3 parameters for each individual scenario

• R0
• The initial time, defined as 3 weeks before we estimate 20 total infections have occurred in the community
• The estimated underreporting fraction.

So you are correct. Initial cases is better described by initial number of infections in the population, not initial number of reported infections. Hope this is clear.

[cbird808]

got it, thank you for the quick response!

So we've loaded our scenario and case data into a fork of your repo, which I presume means that the values for these parameters that populate the web page are generated from your algorithm. Reading from the graph:

• The starting date chosen is ~9 days prior to 20 infectious and ~4 weeks prior to 20 cumulative cases.

• For USA-Texas, it starts ~7 days before 20 infectious ~3 weeks prior to 20 cumulative cases.

• For Australia, it starts ~10 days priot to 20 infectious and ~3 weeks prior to 20 cumulative cases

this doesn't exactly match up, so I presume the values you gave above are approximate?

[nnoll]

Sorry these are approximate for various reasons (e.g. mitigations are not accounted for during the fitting procedure). And it is important to emphasize as I misspoke earlier that its 20 _cumulative_ infections. All details of our fitting algorithms can be found within data/scripts/model.py.

…

On Mon, Apr 13, 2020 at 3:55 PM cbird808 ***@***.***> wrote: got it, thank you for the quick response! So we've loaded our scenario and case data into a fork of your repo, which I presume means that the values for these parameters that populate the web page are generated from your algorithm. Reading from the graph: - The starting date chosen is ~9 days prior to 20 infectious and ~4 weeks prior to 20 cumulative cases. - For USA-Texas, it starts ~7 days before 20 infectious ~3 weeks prior to 20 cumulative cases. - For Australia, it starts ~10 days priot to 20 infectious and ~3 weeks prior to 20 cumulative cases this doesn't exactly match up, so I presume the values you gave above are approximate? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#18 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHAVMGZOMGZZII6R4QCD323RMOJ5HANCNFSM4LVMO6HQ> .

[cbird808]

once we finish the present task, we are turning our attention to building a command line interface to negate the need for the web interface, enable hpc. Any interest in pulling this when we are done?

[jilladona]

I saw that the mitigation tool was recently updated to specify exact dates of interventions. We ran our estimates again, to arrive at nearly unchanged numbers for total deaths, peak critical, and peak severe.

However, total severe doubled. The % Severe and % Critical also doubled.

The basic assumptions are the same. What happened?

[jilladona]

Yes, we used 60 for two months now and then 40! Thanks for that. :)

Here is the file (using the new tool). It has additional columns at the end but it is basically the exported file.

[nnoll]

I think I see the problem. Are you comparing the sum of the entries within the severe column to our reported total severe cases? If so, you are counting the same patients multiple times! The way to see this is that the number reported in this column is the number of severe cases each day of the simulation. A person be considered a "severe" case for multiple days (the length of which you choose as length of stay in the hospital). So when you sum over entries, you count that person multiple times. Let me know if I was unclear or I misunderstood!

[jilladona]

Thank you! Does that go for ICU too, then?

Still unclear why the change in summary totals differ with the new mitigation tool:
Old
New

/EDIT: Updated links to match periods /

Can you help?

[nnoll]

Hey @jilladona. Sorry for the delay, I was investigating the problem. The difference is due to a regression in the computation of the cumulative hospitalizations on our end. I've fixed the issue and we've hotfixed the website so the numbers should be consistent now. I'm terribly sorry for the confusion and really appreciate your bug report!

The logic I outlined for severe is also true of critical (the current category, not cumulative).

[jilladona]

Thank you very much!!! Really glad to clarify this!

[cbird808]

Plot produced by model.py

Can anybody confirm if I have this right?

• the blue points are the cumulative number of cases
• the orange line is fit to blue points
• green points are fatalities
• red line is fit to green points
• purple line may be projected severe cases?
• brown line is projected infections?

Any guidance, however small, on how you dial in the params in model.py is much appreciated

[nnoll]

@cbird808. Sorry for the confusion. The debug plots that you are looking at there were not intended for usage outside of myself so I never annotated the plot!

You have it mostly correct. I'll list this out here

• Blue points = cumulative # of tested cases
• Orange line = model predicted cumulative # of tested cases
• Green dots = cumulative fatalities
• Red line = predicted cumulative fatalities
• Brown Line = All cumulative infections (tested + non tested)
• Purple Line = cumulative recoveries

Let me know if you have any other questions!

[cbird808]

Seasonal Forcing

let's say Ro=2 and Seasonal forcing = 0.1

Will the peak Ro be 2.2 and the trough be 1.8? or would it be 2.1 and 1.9 respectively?

[nnoll]

The formula, in the absence of mitigation, is R(t) = R0 * (1 + seasonalForcing*cos(...)). So the peak and trough would be 2.2 and 1.8 respectively.

[AxiaTim]

Hi all, thanks for everyone's contribution to this - it's been invaluable to for us building more localised models.

I just wanted to confirm and double check one thing:
For the Severity of ICU overflow - if I want to turn this off (so no effect from overflow), should this factor be 1 or 0?

Thanks,

[rneher]

Thank you for the kind words!

A severity factor 1 turns it off. Alternatively set the number of available beds to a very large number and overflow is never triggered.

[AxiaTim]

Thanks Richard for the confirmation and for the quick reply

[tleonard6]

Hello, this is probably a silly question. I have an R shiny dashboard that I run with IHME and CHIME on the county level for the U.S.. All credit is paid to the original research and references are included. With IHME I'm able to download the data every morning and with CHIME, we just programmed the model into our dashboard. Has anyone discussed adapting the Neher Lab models into R at this point or is there a convenient way to do so? Our current dashboard is below.

https://covid19dash.shinyapps.io/covid19/

Thank you,
Taylor

[rneher]

We have a separate python implementation to fit parameters to empirical data, but I am not aware of an R implementation yet. your dashboard looks great!

[sachadray]

Hi,

Fantastic tool, thanks for sharing it!

Is there an API or a way to export the results of simulations for a set of countries/parameter estimates for research purposes?

Many thanks,
Sacha

[tleonard6]

@sachadray We have a couple of report export buttons that will save csv's straight to your computer on the summary tab. We'll be adding some more options as well. Thank you for checking it out!

[rneher]

@brunorzn has worked on an API, he might be able to provide some details.

[brunorzn]

Yes, we've been exchanging some emails.

[JamiesonL]

Hello! Thanks again for all your work on this.

I'm having some issues modifying the latest version of the code for our specific regions (as a side, I've successfully done this with a version 0). I make some changes to a number of files (listed below), specifying them for regions we are interested in. However, it fails to compile (I've reproduced the model locally via a VM and run it off my localhost). The files I'm making changes to are the following:

state.ts; case_counts.json; country_age_distribution.json; severityData.json; scenarios.json; country_codes.csv; intitialCondition.tsv; fit_parameters.json; populationData.tsv; download_age_dists.py

Is there anything I am missing? Sorry it is a bit of a vague question but just wanted to check if there isn't something obvious that I am missing.

Thanks so much!

[nnoll]

Hey @JamiesonL. My best guess is that we've changed schemas around to account for the range inputs on both R0 and mitigation efficacies. Before, these parameters were specified by a single real number, now they are specified by two. This change directly touches a few of the files you list, namely scenarios.json. If this doesn't answer your question, perhaps you could post a screenshot of the error?

[JamiesonL]

Hi @nnoll. Thanks so much for responding. Managed to fix it and thought I would respond in case anyone else has the same problem. I double checked every line of code and also made certain my code syntax was correct (eg. white space, indentation), and this worked. Since the json and other files are compiled using Python, syntax/structure is crucial (it didn't seem to matter in the version 0). Thanks again!

[schaupets]

Too slow decline of infectious, ICU, Severly after 100% intervention?

I'm trying to simulate the effect of a 100% shutdown intervention, i.e. in a specific local hotspot.
In which time could a breakout be stopped and eliminated by a complete shutdown.

An Intervention of 100% should immediately stop new infections.

The simulation model does this, but has some strange effects in the timeline, which I do not understand:
Example for Germany:
Intervention from 1.June to 30.Sept = 100%

a) The number of infecctious decreases to zero only after 4 Weeks, altough latencies+infectious = 3+3 days . So I would expect a much faster decline after 6 days.
b) the number of ICU and Severly ill decreases even slower and are greater Zero still after 2 Month. There is almost no change compared to 40%+60% Intervention. With an hospital-stay = 3 days and ICU = 14 Days I would expect a decline to zero after about 3 weeks.

In Podcast(38) Chr.Drosten says, that 4 weeks are the time from infection to death.
So in my understanding latest 4 weeks after a 100% intervention all cases should be = 0 !?

Question: What did I miss?

(Hope this is the correct stream, I had this question before in #641 )

[rneher]

We model a 2w residence time in the ICU. But this is not a fixed number, but an exponential distribution. Hence the people in ICU goes down exponentially (straight line on a log plot) with a time constant of 14 days. The number of infectious drops much quicker since the typical time to stop being infectious is just 3 days in our model. One could replace this exponential distribution by a different distribution, but it would require a more complicated model. As such, the number of ICU patients should drop by about 85% in four weeks and another 85% in the next four weeks and so on.

[schaupets]

OK; thankx for the quick answer. This explains the differences to my privat Excel Simulation.

To understand you correctly: So in a real world situation, the numbers would drop quicker than the simulation for a 100% intervention and probably also for a 80-90% intervention?

[cbird808]

Imports per day should be a parameter that can be changed like the mitigations. This parameter is expected to change through time, especially here in the US where states are trying to go back to normal.

fyi, we made a "wrapper" that allows c19scenarios to be run with parameters in the form of distributions. It generates n urls that are fed to the webserver and processes the output into the mean, upper CI bound, and lower CI bound (the same format as the *summary*tsv file). Do you have any interest in using this or adding it into your repo?

[rneher]

regarding imports, I agree. this would be preferable. The constant was just what seemed reasonable in March. Now that cases are low and borders reopen, it would make sense to make this time dependent.

regarding the distributions: we now run a user-specified number of parameter samples. I am not sure how this relates to your parameter distributions. But it sounds interesting.

[ivan-aksamentov]

Hi @cbird808,

Thanks for your suggestions. We are open for discussion and contributions. If you have some concrete plans, feel free to open feature request issues(s), let's talk!

"wrapper" that allows c19scenarios to be run with parameters in the form of distributions.It generates n urls [...]

Folks from Github/Microsoft contributed a little command line interface some time ago in src/algorithms/cli.ts (#689, #725). They do some sort of a bulk processing with it too. Here in the lab we don't have much resources to support it officially, but maybe you will find it useful. In this case feel free to extend/improve. We could also have a programmatic Node API and publish it all to NPM. Associated ticket: #663

[cbird808]

thank you, gentlemen. We'll take a look at cli.ts and see how it compares with our c19s_client.py and get back in touch

[r-s-rai]

Hello,

I am part of a team of three Washington University in St. Louis - Calvin Mitchell, Nathan Springman, and myself - that are looking to help contribute to the covid19_scenarios open-source project this summer. We hope to be of some help to your efforts in perfecting this project. Is there any pressing issue or documentation that you would want us to work on first?

Sincerely,

Rohan Rai

[rneher]

Hi Rohan,

thanks for reaching out. Yes, there are still a few issues that could use help! Is your expertise more on the front-end side or more on the modeling side? I think we could find stuff on both ends but it would be good to coordinate to maximize the impact of your contributions.

best,
richard

[r-s-rai]

Hello Richard, We are more experienced with front-end development, but we are willing to learn about modeling as well. Thank you, Rohan Rai

…

________________________________ From: Richard Neher <notifications@github.com> Sent: Thursday, June 25, 2020 2:57 AM To: neherlab/covid19_scenarios <covid19_scenarios@noreply.github.com> Cc: Rai, Rohan <r.s.rai@wustl.edu>; Comment <comment@noreply.github.com> Subject: Re: [neherlab/covid19_scenarios] Questions, discussions (#18) Hi Rohan, thanks for reaching out. Yes, there are still a few issues that could use help! Is your expertise more on the front-end side or more on the modeling side? I think we could find stuff on both ends but it would be good to coordinate to maximize the impact of your contributions. best, richard — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#18 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AKI6K4CYR63MBMUID33LSADRYL7NRANCNFSM4LVMO6HQ>.

[rneher]

ok, great. there are a number of issues that are labeled good-first-issue are front-end related. Why don't you have a look and see whether anything is interesting? Probably best if you sketch a proposal that we can discuss before starting in earnest.

[r-s-rai]

Hello Richard, Would it be possible to schedule a meeting about how you want the data implemented for issue #744? Here is a link to it: #744. My group and I were thinking about using an Amazon S3 bucket to store the data, but we wanted to coordinate with you guys since you'll be the ones in charge of the S3 bucket and managing it, so we need to discuss about how to set up the bucket and the access privileges for it. Thank you, Rohan Rai

…

________________________________ From: cbird808 <notifications@github.com> Sent: Tuesday, June 30, 2020 1:36 PM To: neherlab/covid19_scenarios <covid19_scenarios@noreply.github.com> Cc: Rai, Rohan <r.s.rai@wustl.edu>; Comment <comment@noreply.github.com> Subject: Re: [neherlab/covid19_scenarios] Questions, discussions (#18) thank you, gentlemen. We'll take a look at cli.ts and see how it compares with our c19s_client.py and get back in touch — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#18 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AKI6K4F5NR64EDY4C72CJM3RZIWDJANCNFSM4LVMO6HQ>.

[rneher]

sorry, I thought I responded to this. happy to schedule a meeting. I'll get in touch via email.