stantargets: reproducible Stan pipelines at scale

[wlandau]

Submitting Author: Will Landau (@wlandau)
Repository: https://github.com/wlandau/stantargets
Version submitted: 0.0.0.9000
Editor: @melvidoni
Reviewers: @sakrejda @mattwarkentin

Due date for @sakrejda: 2021-03-31

Due date for @mattwarkentin: 2021-03-31
Archive: TBD
Version accepted: TBD

---

• Paste the full DESCRIPTION file inside a code block below:

Package: stantargets
Title: Targets for Stan Workflows
Description: Bayesian data analysis usually incurs long runtimes
  and cumbersome custom code. A specialized pipeline toolkit for
  Bayesians, the 'stantargets' R package leverages
  'targets' and 'cmdstanr' to ease these burdens.
  'stantargets' makes it super easy to set up useful scalable
  Stan pipelines that automatically parallelize the computation
  and skip expensive steps when the results are already up to date.
  Minimal custom code is required, and there is no need to manually
  configure branching, so usage is much easier than 'targets' alone.
  'stantargets' can access all of 'cmdstanr''s major algorithms
  (MCMC, variational Bayes, and optimization) and it supports
  both single-fit workflows and multi-rep simulation studies.
  For the statistical methodology, please refer to 'Stan' documentation
  (Stan Development Team 2020) <https://mc-stan.org/>.
Version: 0.0.0.9000
License: MIT + file LICENSE
URL: https://wlandau.github.io/stantargets/, https://github.com/wlandau/stantargets
BugReports: https://github.com/wlandau/stantargets/issues
Authors@R: c(
  person(
    given = c("William", "Michael"),
    family = "Landau",
    role = c("aut", "cre"),
    email = "will.landau@gmail.com",
    comment = c(ORCID = "0000-0003-1878-3253")
  ),
  person(
    family = "Eli Lilly and Company",
    role = "cph"
  ))
Depends:
  R (>= 3.5.0)
Imports:
  cmdstanr (>= 0.2.0),
  digest (>= 0.6.21),
  fst (>= 0.9.4),
  posterior (>= 0.1.2),
  purrr (>= 0.3.4),
  qs (>= 0.14.1),
  rlang (>= 0.4.8),
  stats,
  targets (>= 0.0.1),
  tarchetypes (>= 0.0.1),
  tibble (>= 3.0.4),
  tools
Suggests:
  dplyr (>= 1.0.2),
  fs (>= 1.5.0),
  knitr (>= 1.28),
  R.utils (>= 2.10.1),
  rmarkdown (>= 2.1),
  testthat (>= 3.0.0),
  visNetwork (>= 2.0.9),
  withr (>= 2.1.2)
Remotes:
  stan-dev/cmdstanr,
  stan-dev/posterior
SystemRequirements: CmdStan >= 2.25.0
Encoding: UTF-8
Language: en-US
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.1.1
VignetteBuilder: knitr
Config/testthat/edition: 3

Scope

• Please indicate which category or categories from our package fit policies this package falls under: (Please check an appropriate box below. If you are unsure, we suggest you make a pre-submission inquiry.):

  • data retrieval
  • data extraction
  • data munging
  • data deposition
  • workflow automation
  • version control
  • citation management and bibliometrics
  • scientific software wrappers
  • field and lab reproducibility tools
  • database software bindings
  • geospatial data
  • text analysis

• Explain how and why the package falls under these categories (briefly, 1-2 sentences):

stantargets is very similar to jagstargets (#425). stantargets leverages the existing workflow automation capabilities of targets to orchestrate computation and skip up-to-date tasks in Bayesian data analysis pipelines. stantargets reduces the burden of user-side custom code that targets would otherwise require, which helps free statisticians to focus more on the models and less on the software engineering.

• Who is the target audience and what are scientific applications of this package?

stantargets is for Bayesian statisticians who develop and run Stan models. Example workflows range from individual analyses of clinical data to large-scale simulation-based calibration studies for validation.

• Are there other R packages that accomplish the same thing? If so, how does yours differ or meet our criteria for best-in-category?

targets already provides the same kind of workflow automation, but it requires more custom code to set up a workflow. stantargets uses specialized domain knowledge to make this process easier. Packages rstan and cmdstanr interface with Stan but do not provide the same kind of workflow automation. In light of the recent preprint by Gelman et al. (2020), I believe the Stan Development Team would be very interested in this kind of workflow automation.

• (If applicable) Does your package comply with our guidance around Ethics, Data Privacy and Human Subjects Research?

N/A

• If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted.

N/A

Technical checks

Confirm each of the following by checking the box.

• I have read the guide for authors and rOpenSci packaging guide.

This package:

• does not violate the Terms of Service of any service it interacts with.
• has a CRAN and OSI accepted license.
• contains a README with instructions for installing the development version.
• includes documentation with examples for all functions, created with roxygen2.
• contains a vignette with examples of its essential functions and uses.
• has a test suite.
• has continuous integration, including reporting of test coverage using services such as Travis CI, Coveralls and/or CodeCov.

Publication options

• Do you intend for this package to go on CRAN?

• Do you intend for this package to go on Bioconductor?

• Do you wish to submit an Applications Article about your package to Methods in Ecology and Evolution? If so:

If JOSS is still an option, I would like to publish there. I have prepared a manuscript at https://github.com/wlandau/stantargets/blob/main/inst/paper.md.

MEE Options

• The package is novel and will be of interest to the broad readership of the journal.
• The manuscript describing the package is no longer than 3000 words.
• You intend to archive the code for the package in a long-term repository which meets the requirements of the journal (see MEE's Policy on Publishing Code)
• (Scope: Do consider MEE's Aims and Scope for your manuscript. We make no guarantee that your manuscript will be within MEE scope.)
• (Although not required, we strongly recommend having a full manuscript prepared when you submit here.)
• (Please do not submit your package separately to Methods in Ecology and Evolution)

Code of conduct

• I agree to abide by rOpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

[melvidoni]

Editor checks:

• Documentation: The package has sufficient documentation available online (README, pkgdown docs) to allow for an assessment of functionality and scope without installing the package.
• Fit: The package meets criteria for fit and overlap
• Automated tests: Package has a testing suite and is tested via a CI service.
• License: The package has a CRAN or OSI accepted license
• Repository: The repository link resolves correctly

---

Editor comments

Thank you for submitting it! I'll be your handling editor this time. Goodpractice is very happy about it:

♥ Aha! Fantastic package! Keep up the prime work!

There are no spelling checks, and covr is also happy. Good job!

I will be searching for reviewers then.

---

Reviewers: @sakrejda @mattwarkentin
Due date: Wednesday, March 31st

[wlandau]

Thanks, @melvidoni! Looking forward to the review process. Please let me know if you would like me to suggest potential reviewers.

[melvidoni]

Quick update. I have contacted about 15 people, and received no acceptance to review. Please bear with me while I find someone. And yes, I have your master list of suggestions.

[wlandau]

Thanks for the update, @melvidoni. I do understand this may take some time.

And yes, I have your master list of suggestions.

Do you mean the folks I mentioned at #401 (comment) for targets? I actually think about stantargets a bit differently because of its focus on Bayesian Statistics. (Likewise for jagstargets in #425.) For my own learning, it would be amazing to include someone from the Stan Development Team. If not, then I think practicing Bayesian statisticians, particularly Stan users, would have a lot to add.

[melvidoni]

Hello @wlandau. Yes, that list. We curated it with some editors, and also have our own list of people. However, given the current world circumstances, I am afraid that finding reviewers may take longer than expected.

[wlandau]

Yes, that list.

Oh good, glad we are on the same page. Sorry, I may have misremembered the earlier parts of this thread.

However, given the current world circumstances, I am afraid that finding reviewers may take longer than expected.

I completely understand.

For what it's worth, dependency packages cmdstanr and posterior are not yet on CRAN, so this review process is extremely unlikely to bottleneck any of my plans. And unlike targets, I am not developing any packages that depend on stantargets.

[mdscheuerell]

Perhaps Krzysztof Sakrejda would be available?

[melvidoni]

Perhaps Krzysztof Sakrejda would be available?

Hello @sakrejda are you interested?

[sakrejda]

Hi Melina, I am interested and the three-week time frame is reasonable for me. I'm not the right person to say much about whether the package aligns well with how targets itself is commonly used but should be able to provide feedback otherwise. If you haven't asked @tjmahr yet he might be interested in reviewing.