How To Make Your Data Analysis Notebooks More Reproducible
I have included a handful of links to papers, software packages and tutorials/manuals about some tools I mention in my talk. Pull requests or issues on additional ones to include are welcome.
- Statistical Analysis and reproducible research
- Packaging Data Analytical Work Reproducibly Using R (and Friends) (OA preprint). A practical introduction to setting up a research compendium in R.
- The rOpenSci reproducibility guide Slightly dated but still very useful
Examples of Research Compendia on GitHub
Below are a few links to real world examples of research compendia in R. To have a minimal compendium, all you really need is a valid
DESCRIPTION file containing a handful of fields such as type, name, version and dependencies. See Marwick et al 2017 for a detailed description of the different types of compendia.
- Code and data associated with Duffy, James, and Longworth Applied and Environmental Microbiology paper describing the ecology, virulence, and phylogeny of a brood parasite of Daphnia, Blastulidium paedophthorum;
Software packages related to research compendia
rrtoolsby Ben Marwick (also the author of the packaging data analysis paper mentioned above) extends functions in
devtoolsand provides instructions, templates, and functions to make a basic compendium suitable for doing reproducible research with R.
usethisMany of the major function in
rrtoolsare imported from
usethis.A savvy user can get by setting up and maintaining a compendium purely with
goodpractice- Designed to help you build more robust packages, the package does a deep dive on your package contents and provide advice on syntax pitfalls to avoid, code formatting suggestions, and helps you improve overall package structure.
rticlespackage by JJ has numerous journal templates and together with Rstudio addins like word
📈 Data management 📦
piggyback, [docs]: This clever R package allows you to attach arbitrary data (or other) files (upto 2gb each) to a GitHub release. Given GitHub's fast CDN, this would be an easy way to quickly attach large files to a compendium and read them back in a local/collaborator/remote environment very easily. As always be sure to archive a long-term copy on Zenodo.
arkdb[docs]: This package allows you to archive and unarchive databases as flat text files.
🎥For more on setting up data packages, see this excellent talk by Noam Ross at New York R.
Computational environments: Binder and friends
- My Binder is a free binderhub deployment that turns any Git repo into a collection of interactive notebooks. Now with better R support!
- For instructions on how to set this up for your R project, see my notes here
- Introducing Binder 2.0 — share your interactive research environment Paper describing the architecture of Binder in case you were interested in what was happening under the hood
🎥A talk about Binder at Scipy 2018. Also see conference proceedings PDF.
repo2dockerA Python module that will turn any repo (or local folder) into a Docker Image.
Other hosted Binder hubs
- Pangeo binder Pangeo encourages everyone to use it.
- Syzgy Binder + JupyterHub for Compute Canada
Setting up Binder for your analysis
I have captured all the various ways to set up mybinder with a R project in a separate document.
Are you interested in setting up or hosting a binderhub for the R community? Get in touch via the issues.
- Whole Tale
- Computing environments for reproducibility: Capturing the “Whole Tale” - OA paper describing the Whole Tale project.
- Code Ocean - A commercial, blackbox, full-stack service that will accomplish something similar to the above two projects. Code Ocean links will likely start appearing in papers soon.
Software packages related to setting up computational environments
Containerit. Detailed blog post This sweet package will generate a Dockerfile for you by examining the code inside a folder or just from your session info. This is analogous to
repo2dockerbut is very R centric
stevedoreAlthough there are a few docker clients (docker, harbor), this is my recommendation for managing docker containers from inside R.
🔨 Workflows: drake and friends 📦
drake- An R-focused pipeline toolkit for reproducibility and high-performance computing. Install the package from here or CRAN.
- The prequel to the drake R package A blog post by the creator of drake describing his motivation for the package.
- drake manual A detailed
bookdownguide on how to setup and use drake for projects of varying levels of complexity.
- Presentation on drake Slides from a talk by Will Landau (who is here at the conference so go pick his brain if you want to learn more!)
Real world drake examples
- IKEA diagram inspired by IDEA instructions
Many thanks to Chris Holdgraf, Carl Boettiger, Will Landau, and Ben Marwick for various discussions on these topics. Also thanks to Ciera Martinez, Kara Woo, and Nick Tierney for comments on the presentation.