Issue 289: Getting help with stan vignette #405

medewitt · 2023-11-29T16:46:09Z

Description

Getting the discussion on the PR to draft mode for future discussion. This PR closes #289.

This PR includes the initial draft for users getting help with Stan. It provides some extended information on the following:

Getting Stan and dependencies installed (toolchain)
How to manipulate Stan options through the epinowcast functions
Approaches to address common failure modes and error messages thrown by Stan

Checklist

My PR is based on a package issue and I have explicitly linked it.
I have included the target issue or issues in the PR title in the for Issue(s) issue-numbers: PR title
I have read the contribution guidelines.
I have tested my changes locally.
I have added or updated unit tests where necessary.
I have updated the documentation if required.
My code follows the established coding standards.
I have added a news item linked to this PR.
I have reviewed CI checks for this PR and addressed them as far as I am able.

…stan-help

Merge branch 'main' of https://github.com/epinowcast/epinowcast into stan-help

…stan` syntax. (#386) * first draft action that installs cmdstan * use the correct approach to checking for stan changes * add check for cmdstanr compilation and syntax correctness * add skeleton news item * update PR number * use pull request not PR * Automatic readme update [ci skip] * make sure to install epinowcast * missing testthat call * Automatic readme update [ci skip] * remove spurious readme figures * add comments for cmdstan workflow * make the refex match exactly only * try a simpler approach to error catching * fail around more --------- Co-authored-by: GitHub Action <action@github.com>

medewitt · 2023-11-29T16:49:27Z

Tagging @zsusswein, too.

I wanted to get this in front of everyone. Happy for feedback (i.e., take a machete to it). I'm happy to edit/expand/etc after feedback.

I feel like it needs some examples using synthetic data which I can make (e.g., put in too little data and let the model fail, put in multi-modal data and let the posteriors be wide and ESS low).

codecov · 2023-11-29T16:59:56Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (b273412) 96.84% compared to head (41d45c4) 96.84%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #405   +/-   ##
=======================================
  Coverage   96.84%   96.84%           
=======================================
  Files          15       15           
  Lines        1873     1873           
=======================================
  Hits         1814     1814           
  Misses         59       59

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

.github/workflows/check-cmdstan.yaml

seabbs

This is so good!

A few comments mostly discussing my personal quest for validation via control of spacing. I think we could link out to more detail on some of these topics (like adapt delta settings etc) a bit more liberally and we might also want to give some tips for working with cmdstan objects (like tidybayes, posterior etc.).

I think we should aim to not make this PR review into what we would ultimately like in this vignette but instead aim for a good skeleton we can add to over time.

I feel like it needs some examples using synthetic data which I can make (e.g., put in too little data and let the model fail, put in multi-modal data and let the posteriors be wide and ESS low).

I think this would be lovely but would also be very happy with a first version without this and save adding for a later PR?

vignettes/stan-help.Rmd

Co-authored-by: Sam Abbott <s.e.abbott12@gmail.com>

zsusswein

This is really nice! I'm new to {epinowcast} myself and found this a helpful guide. I left a bunch of comments inline -- they're split between minor typos and some suggestions around content to expand on or re-organize. I would consider them all optional and nitpicky.

One other thought: I would personally find it useful if this document mentioned some particular parameters to investigate in problematic fits. I don't know what the hierarchical variance parameters are called (or other frequent trouble spots), but it would speed up my model debugging to know where to look first. That may be out of scope, but thought it was worth mentioning.

vignettes/stan-help.Rmd

seabbs · 2023-12-01T14:34:11Z

Nice comments @zsusswein.

One other thought: I would personally find it useful if this document mentioned some particular parameters to investigate in problematic fits. I don't know what the hierarchical variance parameters are called (or other frequent trouble spots), but it would speed up my model debugging to know where to look first. That may be out of scope, but thought it was worth mentioning.

I think we should make this out of scope for this PR but open an issue to add to this vignette (or possibly to expand into more than one if getting long) as I agree it would be very useful.

Part of the reason I think its out of scope is I think this might need #349 to be completed first otherwise its quite reliant on folk wisdom.

vignettes/stan-help.Rmd

seabbs · 2023-12-05T14:04:58Z

@medewitt would it be helpful to add some suggestions for any of the comments? I think all/most of the reviewers would be happy to help so just give them/me a ping if useful!

medewitt · 2023-12-05T21:49:38Z

@medewitt would it be helpful to add some suggestions for any of the comments? I think all/most of the reviewers would be happy to help so just give them/me a ping if useful!

Sure! Sorry, I got a bit busier this weekend than planned. I should be able to pick this up Wednesday/Thursday this week....

Co-authored-by: Carl A. B. Pearson <pearsonca@users.noreply.github.com>

Co-authored-by: Zachary Susswein <46581799+zsusswein@users.noreply.github.com>

…stan-help

_readme-install-epinowcast.Rmd

seabbs

This looking even better now. A few more comments.

vignettes/stan-help.Rmd

medewitt · 2023-12-11T15:54:58Z

@seabbs almost there. I'll fix the linting error ((default_priors <- enw_reference(data = enw_example("preprocessed"))) to separate lines).

If you feel good about it, I'll add a news item and move it out of draft.

seabbs · 2023-12-12T16:27:16Z

Awesome - I do indeed feel good about it!

seabbs

This is really nice. Thanks for putting so much effort into it @medewitt! I think you've addressed all the reviewer points (thanks @zsusswein and @pearsonca!) so this is good to go from my perspective.

Note I adjusted the pkgdown articles section to split out help with epinowcast from case studies as I think this a little bit clearer.

seabbs

This is really nice. Thanks for putting so much effort into it @medewitt! I think you've addressed all the reviewer points (thanks @zsusswein and @pearsonca!) so this is good to go from my perspective.

Note I adjusted the pkgdown articles section to split out help with epinowcast from case studies as I think this a little bit clearer.

medewitt and others added 11 commits November 21, 2023 06:12

initial skeleton

6662cc6

Merge branch 'main' of https://github.com/epinowcast/epinowcast into …

f57019c

…stan-help

adding reference to bayesian workflow

704ec20

more content

55b7c41

adding more content

6ee78d1

Catching up with main.

14e1b3d

Merge branch 'main' of https://github.com/epinowcast/epinowcast into stan-help

building content on key approaches

8961763

continue to add content on settings

007a5c8

adding more content

fb48a45

Starting to discuss understanding data

b936bdb

medewitt requested review from seabbs and adrian-lison November 29, 2023 16:46

seabbs reviewed Nov 30, 2023

View reviewed changes

.github/workflows/check-cmdstan.yaml Outdated Show resolved Hide resolved

seabbs requested changes Nov 30, 2023

View reviewed changes

Apply suggestions from code review

95d8871

Co-authored-by: Sam Abbott <s.e.abbott12@gmail.com>

zsusswein reviewed Dec 1, 2023

View reviewed changes

initial round of clean up

3b4635b

medewitt added 2 commits December 1, 2023 13:07

[ci skip] catching up pr with edit. more to come

cc58257

Merge branch 'main' into stan-help

7c19d2b

pearsonca reviewed Dec 2, 2023

View reviewed changes

vignettes/stan-help.Rmd Outdated Show resolved Hide resolved

medewitt and others added 4 commits December 5, 2023 16:49

Update vignettes/stan-help.Rmd

8f5186c

Co-authored-by: Carl A. B. Pearson <pearsonca@users.noreply.github.com>

Update vignettes/stan-help.Rmd

6982991

Co-authored-by: Zachary Susswein <46581799+zsusswein@users.noreply.github.com>

satisfy the empty last line linter

cb7c4fc

Merge branch 'main' of https://github.com/epinowcast/epinowcast into …

08f5315

…stan-help

spelling things correctly.

f137485

seabbs reviewed Dec 10, 2023

View reviewed changes

_readme-install-epinowcast.Rmd Outdated Show resolved Hide resolved

seabbs self-requested a review December 10, 2023 17:45

seabbs reviewed Dec 10, 2023

View reviewed changes

vignettes/stan-help.Rmd Outdated Show resolved Hide resolved

vignettes/stan-help.Rmd Outdated Show resolved Hide resolved

vignettes/stan-help.Rmd Show resolved Hide resolved

vignettes/stan-help.Rmd Outdated Show resolved Hide resolved

vignettes/stan-help.Rmd Outdated Show resolved Hide resolved

medewitt added 11 commits December 11, 2023 09:07

updating author list

071332f

move to readme chunk to man

f2dad4f

Remove hidden or commented text

99eddf3

update iter warmup starting chain values

3ce6920

add name to spellcheck

677204f

add warmup language

927f4a0

updating package README with link to vignette

b87c368

going live on epinowcast functions

dc8b62b

Proper directory for chunks

cabc775

Suggested warmup iterations

b57dbaa

latest md readme

37aa5cc

medewitt added 3 commits December 13, 2023 07:12

Fix linting error; print function new line

dd22876

add vignette to pkgdown config

b36c27f

updating news for stan help

86e6ce7

medewitt marked this pull request as ready for review December 13, 2023 13:17

medewitt requested a review from seabbs December 13, 2023 13:21

add a working with epinowcast section to pkgdown

41d45c4

seabbs reviewed Dec 13, 2023

View reviewed changes

seabbs approved these changes Dec 13, 2023

View reviewed changes

seabbs enabled auto-merge December 13, 2023 13:48

seabbs disabled auto-merge December 13, 2023 14:49

seabbs merged commit 88ace98 into main Dec 13, 2023
9 checks passed

seabbs deleted the stan-help branch December 13, 2023 14:50

seabbs mentioned this pull request Dec 13, 2023

Move @medewitt to author #413

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue 289: Getting help with stan vignette #405

Issue 289: Getting help with stan vignette #405

medewitt commented Nov 29, 2023 •

edited

medewitt commented Nov 29, 2023

codecov bot commented Nov 29, 2023 •

edited

seabbs left a comment •

edited

zsusswein left a comment

seabbs commented Dec 1, 2023 •

edited

seabbs commented Dec 5, 2023

medewitt commented Dec 5, 2023

seabbs left a comment

medewitt commented Dec 11, 2023

seabbs commented Dec 12, 2023

seabbs left a comment

seabbs left a comment

Issue 289: Getting help with stan vignette #405

Issue 289: Getting help with stan vignette #405

Conversation

medewitt commented Nov 29, 2023 • edited

Description

Checklist

medewitt commented Nov 29, 2023

codecov bot commented Nov 29, 2023 • edited

Codecov Report

seabbs left a comment • edited

Choose a reason for hiding this comment

zsusswein left a comment

Choose a reason for hiding this comment

seabbs commented Dec 1, 2023 • edited

seabbs commented Dec 5, 2023

medewitt commented Dec 5, 2023

seabbs left a comment

Choose a reason for hiding this comment

medewitt commented Dec 11, 2023

seabbs commented Dec 12, 2023

seabbs left a comment

Choose a reason for hiding this comment

seabbs left a comment

Choose a reason for hiding this comment

medewitt commented Nov 29, 2023 •

edited

codecov bot commented Nov 29, 2023 •

edited

seabbs left a comment •

edited

seabbs commented Dec 1, 2023 •

edited