Presubmission inquiry for cookiecutter-cmdstanpy-analysis

[teddygroves]

Submitting Author: Teddy Groves (@teddygroves)
Package Name: cookiecutter-cmdstanpy-analysis
One-Line Description of Package: An interactive template for cmdstanpy analyses
Repository Link (if existing): https://github.com/teddygroves/cookiecutter-cmdstanpy-analysis

---

Hi there! I'm not sure whether my project cookiecutter-cmdstanpy-analysis is within scope for pyOpenSci because it is not a package. As far as I know Python packages are not the standard way to distribute interactive project templates.

I'm struggling a bit to get this work reviewed by relevant experts (e.g. JOSS considers it currently not in scope due to insufficient lines of code openjournals/joss-reviews#4760) so if it is not in scope for pyOpenSci I'd really appreciate advice as to alternative options.

Description

cookiecutter-cmdstanpy-analysis is a cookiecutter template that creates a statistical analysis project based on Python and cmdstanpy. The project can be edited so as to implement a very wide range of statistical analysis: in particular it supports any statistical model that can be written using Stan, any data preparation pipeline that can be achieved using standard python tools, and any combination of such models and data preparation choices.

Scope

• Please indicate which category or categories this package falls under:

  • Data retrieval
  • Data extraction
  • Data munging
  • Data deposition
  • Data visualization
  • Reproducibility
  • Geospatial
  • Education
  • Unsure/Other (explain below)

• Explain how and why the package falls under these categories (briefly, 1-2 sentences). Please note any areas you are unsure of:

My cookiecutter helps scientists make their statistical analyses reproducible by providing a template project that follows statistical programming best practice while being modular, automated, encapsulated, documented and tested.

• Who is the target audience and what are the scientific applications of this package?

Scientists writing software that implements a statistical analysis, particularly when the analysis encompasses several data processing options and several custom statistical models.

• Are there other Python packages that accomplish similar things? If so, how does yours differ?

cookiecutter-data-science is a cookiecutter template for data science projects. It is different from cookiecutter-cmdstanpy-analysis because it is less specific (general data science project vs statistical analysis with cmdstanpy) and because it provides an incomplete skeleton project whereas cookiecutter-cmdstanpy-analysis provides a complete working project.

cookiecutter-cmdstanpy-wrapper is a cookiecutter template for python packages that depend on a built Stan model or models, similarly to prophet. This is different from cookiecutter-cmdstanpy-analysis because the target software achieves a different goal: packages depending on built Stan models are typically intended to be tools that can be used in multiple analyses.

• Any other questions or issues we should be aware of:

P.S. *Have feedback/comments about our review process? Leave a comment here

[NickleDave]

Hi @teddygroves and welcome to pyOpenSci.

I completely understand why you would want to a review but unfortunately we consider project templates as out of scope at this time. In your particular case we would need to additionally make sure we have reviewers with domain expertise. We have just started up review again after transferring to a new fiscal host so we're not quite prepared to do that at this time.

I don't mean to discourage you, just trying to be clear about where we're at.
We are actually planning to sunset our own cookiecutter because it is out of date and reduplicates effort done by others.
In the future we may provide a curated list of cookiecutters as described here: pyOpenSci/pyopensci.github.io#91. Perhaps you could contribute yours to that if we do so.

I'd really appreciate advice as to alternative options.

Is there community-developed stan documentation that you could integrate the cookiecutter into?
For example, see the scikit-HEP developer documentation (https://scikit-hep.org/) that their cookie package points to (https://github.com/scikit-hep/cookie).
Likewise, part of what makes the hypermodern python cookiecutter stand out is its extensive documentation:
https://cookiecutter-hypermodern-python.readthedocs.io/en/2022.6.3.post1/
Maybe you could write vignettes for each of the projects you have listed that use the template, explaining how each project used them and the reasoning behind choices they made. I have to admit that I usually put together projects "by hand" but from other developers that maintain and use templates, my understanding is that this is what they find most useful about them.

I hope that feedback is helpful. I'm sorry we can't provide review at this time. Let me know if you have any other questions.

[teddygroves]

Thanks very much for the detailed response and advice!

I’ve already coordinated with the Stan community a little bit and have added a link to cookie cutter-cmdstanpy-analysis to the cmdstanpy docs.

I agree that the docs could use some more specific vignette style examples - thanks a lot for the tip!

I’ll keep track of pyOpenSci’s policies as the review process restarts and especially the issue you linked.

Thanks again for the feedback!