Optional profiling

[adrian-lison]

This PR implements optional profiling of nowcasting models. Profiling of stan models as offered by cmdstan is useful for debugging and performance optimization (see https://mc-stan.org/cmdstanr/articles/profiling.html). However, it also comes with a slight overhead, making it suboptimal to have profiling activated by default. Stan does not offer an option to turn profiling on and off at the moment. This PR implements a workaround which allows to keep the profiling statements in the same model that will be used for deployment by adding a corresponding option to enw_model. If profile=FALSE, all profiling statements are removed from the model code before compilation.

Some basic profiling statements are added to epinowcast.stan for illustration.

Further details:

• The implementation assumes that the model is syntactically correct. If the model syntax is incorrect, compilation would fail anyway, but might fail in unforeseen ways when profile=FALSE.
• Nested profile statements are also supported. This is useful for profiling a larger section but having individual profiling for certain subsections of interest.
• Profiling statements in model code that is included from additional files in the include path are currently not supported. That is, they will not be removed even if profile=FALSE. Extending to these include files at a later stage should not be too difficult however.
• A further extension for later would be to allow specific profiling statements to be toggled on/off, e.g. by providing a list of profiling identifiers that should be kept, and the rest will be removed.

[codecov]

Codecov Report

Merging #41 (1870025) into main (20274a8) will not change coverage.
The diff coverage is 0.00%.

❗ Current head 1870025 differs from pull request most recent head 2369829. Consider uploading reports for the commit 2369829 to get more accurate results

@@          Coverage Diff          @@
##            main     #41   +/-   ##
=====================================
  Coverage   0.00%   0.00%           
=====================================
  Files         13      13           
  Lines        641     648    +7     
=====================================
- Misses       641     648    +7     

Impacted Files	Coverage Δ
R/model.R	0.00% <0.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 20274a8...2369829. Read the comment docs.

[adrian-lison]

One more interesting thing to note: Generally, switching from profiling to no profiling requires a recompilation of the stan model. However, once the model has been compiled with profile=FALSE, one can always switch between profiling and no profiling without triggering a recompilation. I have verified that this still has an actual effect on whether the model will be profiled or not.

I lack a deeper understanding of what goes on during profiling under the hood, but it seems that the profiling statements are handled separately from the main stan C code.

If we don't want to confuse users with the recompilation, we could hence change the model instantiation function such that if profile=TRUE, it always calls itself with profile=FALSE first.

[seabbs]

This is really great and will be super useful going forward.

Not surprised that the likelihood is so dominant compute wise (though I would expect that to change with more flexible models where more delay distributions need to be calculated). #40 has reinforced my view that optimisation wise most of run-time in the likelihood (that we can do something about) is linked to the coversion from hazard to probability but you are correct being able to profile within functions would be useful to confirm this. Do you think (and can you in fact do this) it is worth inserting some profiling into the generated quantities to see if optimisation is worth doing with that code. Part of me wants it to remain relatively clear but if that comes at a big speed cost then it isn't worth it.

Left some very minor comments on this PR. There are also some issues that the automated linting is flagging that would be good to correct to keep a consistent style. We should maybe talk about setting up automated PR tools to allow styling, redocing etc using github actions.

[seabbs]

/document

[seabbs]

/style

[adrian-lison]

@seabbs I added profiling statements to the generated quantities section. If you think everything is fine, you can already merge into main, but let's keep this branch still - I still want to add support for included files. I can submit an additional pull request then.

[seabbs]

This looks good to go in main to me. Will merge and can do another PR for in function profiling (which will be very nice to have).