Final epidemic size with susceptibility groups #39

pratikunterwegs · 2022-09-15T16:05:33Z

Implements:

Fixes final_size using Cpp Newton solver #4 by porting code from @BlackEdder (at https://gitlab.com/epidemics-r/code_snippets/-/blob/feature/newton_solver/include/finalsize.hpp) into the package. Final size Newton solver now included in the header finalsize.h.
Fixes Cpp implementation of final size with susceptibility groups #40 by adding a new source file with the Rcpp function final_size_grps_cpp. This calls solve_final_size_by_susceptibility, solve_final_size_newton, and epi_spread from the header.
Adds tests for internal C++ functions called by Rcpp functions.

Warning!
Build sometimes fails because the test for final_size_grps_cpp fails to calculate the correct final size, instead returning implausible values - likely due to an issue with the Newton solver. Tagging @BlackEdder in case this can be fixed.

Review input requested

Should the Newton solver should be made more modular, allowing for testing of intermediate functions?
Suggest more tests for the plausibility of solutions.
Suggest text for the p_susceptibility and susceptibility parameters in function documentation.

tests/testthat/test-finalsize_by_group.R

pratikunterwegs · 2022-09-23T10:09:01Z

We’re still getting errors in the 1. simple, 2. complex, and 3. Polymod data case, with the final sizes returned as either 0.0 or NaN.

This behaviour is not uniform across test runs. It seems a maximum of 7 tests are failing, but sometimes only 5 fail.

The failures are also not uniform across systems: locally, 5-7 tests fail (Mac M2), but only 1 test (the complex data case) fails on GHA. However, all 7 fail on Windows GHA.

Latest build reports are here: https://github.com/epiverse-trace/finalsize/actions/runs/3111847610/jobs/5044606831

Possible reasons:

Contact matrices are not correctly scaled
r0 is not being included - should all final size calls specify the contact matrix argument as: r0 * contact_matrix? Where the contact matrix is assumed to be correctly scaled.

* WIP finalsize_grps_cpp tests: test Newton solver * WIP finalsize_grps_cpp tests: Test internal solve fun * Check finalsize_grps_cpp with r0 = 1.3 (passes) * Check finalsize_grps with r0 = 2.0 (fails) * WIP formatting for R tests * Test Cpp solve_by_susc with r0 = 2.0 (fails) * Return ArrayXd * Rcpp export return ArrayXd * Test Newton solver with r0 = 2.0 (fails)

R implementation of final size of an epidemic with susceptibility groups. * WIP - first draft finalsize_grps in R * WIP - first draft tests for finalsize_grps in R * WIP - missing comma for args * Split off epi_spread and remove Newton solver * Implement iterative solver for susc grps, fixes #44 * Use iterative solver in final_size_grps, fixes #45 * Test for epi_spread, WIP #46, #47 * Add tests for iterative solver, fixes #46 * Test final_size_grps, fixes #47 * Documentation for final_size_grps and related fns, fixes #48 * Add extra test for epi_spread, WIP #46, #47 * Minor refactor of p_susc in epi_spread_data, remove commented code * Housekeeping: No explicit namespacing for testthat * Kronecker con_mat replication, vectors instead of mats, fixes #49 * Vectorise con_mat filling in iterative solver, fixes #51 * Vectorise fn_f in iterative solver, and pi adjust, fixes #52 * Squash stopifnot calls into one, fixes #50 * Pass function args re solver steps, solves #53 * Update tests for epi_spread and final_size_grps * Refactor final size from pi to epi_final_size, fixes #54 * Remove test for Newton solver

Correct struct member in return

Move Rcpp C++ 11 plugin to header Format helper functions WIP move helper funs to new header

Fun for epi spread returns custom struct

Minor whitespace

WIP remove unused params WIP add documentation for finalsize by grp WIP add iterations as fn arg to Newton solver WIP minor edits, eg. to remove atuo return types WIP finalsize_by_grp add checks on args WIP finalsize_by_grp begin documentation

Add tests for internal funs in helper_funs.h Add tests for cpp funs in finalsize.h WIP - add tests: finalsize_grps_cpp with polymod data WIP finalsize_grps_cpp: scale contacts and test for NaN WIP finalsize_grps_tests: check for correct proportions infected WIP - checks for final size = 0 Add more tests for final size cpp code Check that final sizes are not Inf, NaN, or zero Add check for p_susceptibility rowwise sum = 1.0 WIP finalsize_grps_cpp tests: Add check for error on p_susc sum WIP finalsize_grps_cpp tests: do not scale contacts in simple case WIP finalsize_grps_cpp tests: p_susceptibility in tests sums to 1 WIP finalsize_grps_cpp tests: test susc - final size relation WIP finalsize_grps_cpp tests: scale contacts by r0 and scale demo

Procedural Rcpp export Supress permissive export, export new finalsize by grp fun

* WIP finalsize_grps_cpp tests: test Newton solver * WIP finalsize_grps_cpp tests: Test internal solve fun * Check finalsize_grps_cpp with r0 = 1.3 (passes) * Check finalsize_grps with r0 = 2.0 (fails) * WIP formatting for R tests * Test Cpp solve_by_susc with r0 = 2.0 (fails) * Return ArrayXd * Rcpp export return ArrayXd * Test Newton solver with r0 = 2.0 (fails)

…susc_grps

pratikunterwegs · 2022-10-03T09:35:12Z

Closing this PR as (1) we are instead focusing on an R implementation of the Newton solver, and (2) the commit history has been wrecked by rebasing on to main after merging #42 - picked commits are duplicated, adding to an already long commit history. The thing to do here was to squash-merge main into this feature branch.

pratikunterwegs added 27 commits September 15, 2022 17:04

Add epi_spread fun from @BlackEdder snippet

a38c056

Modularise solver fun, move helpers to header

1ad8a73

Move Rcpp C++ 11 plugin to header

0647f47

Format helper functions

949714b

Add fun to scale next gen matrix for basic cpp implementation

84e3440

Struct to hold intermediate data; was 'spreaded_t' in @BlackEdder code

a738b0c

Fun for epi spread returns custom struct

c9a786e

Newton matrix solver from @BlackEdder code

2307b46

Fun wrapping epi_spread and Newton solver

ea6140f

Minor whitespace

1b851d0

WIP final size fun with risk groups

289582a

WIP remove unused params

c68a09e

WIP add documentation for finalsize by grp

76cccbf

final_size_cpp: Use scale_nextgen_mat fun

e1a850f

Add tests from @BlackEdder for final size by grp

7c7fc8f

Procedural Rcpp exports

e9d81ca

Supress permissive export, export new finalsize by grp fun

9e021b3

WIP move helper funs to new header

ba37853

WIP add iterations as fn arg to Newton solver

82929a4

WIP minor edits, eg. to remove atuo return types

70b6a73

Correct struct member in return

93fddc4

WIP finalsize_by_grp add checks on args

43ffa9e

WIP finalsize_by_grp begin documentation

5889401

Add tests for internal funs in helper_funs.h

e8dc2b8

Add tests for cpp funs in finalsize.h

b8654cf

Add checks for finalsize_by_grps incl. error messages

fe1c725

Procedural edits to RcppExports and documentation

1daace3

pratikunterwegs self-assigned this Sep 16, 2022

pratikunterwegs added New feature New feature or request Tests Tests and statistical correctness labels Sep 16, 2022

BlackEdder reviewed Sep 23, 2022

View reviewed changes

tests/testthat/test-finalsize_by_group.R Outdated Show resolved Hide resolved

pratikunterwegs added 4 commits September 23, 2022 10:44

Add check for p_susceptibility rowwise sum = 1.0

5a16c3a

WIP finalsize_grps_cpp tests: do not scale contacts in simple case

ba782bc

WIP finalsize_grps_cpp tests: Add check for error on p_susc sum

46175db

WIP finalsize_grps_cpp tests: p_susceptibility in tests sums to 1

ac82651

pratikunterwegs and others added 3 commits September 23, 2022 12:33

WIP finalsize_grps_cpp tests: test susc - final size relation

b2fe756

WIP finalsize_grps_cpp tests: scale contacts by r0 and scale demo

d121935

pratikunterwegs marked this pull request as draft September 29, 2022 15:11

pratikunterwegs and others added 15 commits October 3, 2022 09:08

Add epi_spread fun from @BlackEdder snippet

ad25f59

Correct struct member in return

Modularise solver fun, move helpers to header

535d2f8

Move Rcpp C++ 11 plugin to header Format helper functions WIP move helper funs to new header

Add fun to scale next gen matrix for basic cpp implementation

5215464

Struct to hold intermediate data; was 'spreaded_t' in @BlackEdder code

712f0b3

Fun for epi spread returns custom struct

Newton matrix solver from @BlackEdder code

f545c40

Fun wrapping epi_spread and Newton solver

7eaadec

Minor whitespace

WIP final size fun with risk groups

7d426a8

WIP remove unused params WIP add documentation for finalsize by grp WIP add iterations as fn arg to Newton solver WIP minor edits, eg. to remove atuo return types WIP finalsize_by_grp add checks on args WIP finalsize_by_grp begin documentation

final_size_cpp: Use scale_nextgen_mat fun

b3a50b6

Add tests from @BlackEdder for final size by grp

f71d5e7

Correct final_size_susc_grps, remove extra epi_spread

268bc98

Procedural Rcpp exports

5468e2e

Procedural Rcpp export Supress permissive export, export new finalsize by grp fun

Merge remote-tracking branch 'origin/feature/susc_grps' into feature/…

e6c3550

…susc_grps

pratikunterwegs closed this Oct 3, 2022

pratikunterwegs deleted the feature/susc_grps branch October 3, 2022 09:50

pratikunterwegs restored the feature/susc_grps branch October 3, 2022 11:08

pratikunterwegs deleted the feature/susc_grps branch October 10, 2022 14:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Final epidemic size with susceptibility groups #39

Final epidemic size with susceptibility groups #39

pratikunterwegs commented Sep 15, 2022 •

edited

pratikunterwegs commented Sep 23, 2022 •

edited

pratikunterwegs commented Oct 3, 2022

Final epidemic size with susceptibility groups #39

Final epidemic size with susceptibility groups #39

Conversation

pratikunterwegs commented Sep 15, 2022 • edited

pratikunterwegs commented Sep 23, 2022 • edited

pratikunterwegs commented Oct 3, 2022

pratikunterwegs commented Sep 15, 2022 •

edited

pratikunterwegs commented Sep 23, 2022 •

edited