Do you want to improve your neuroimaging research's reproducibility, but you don’t know where to start? Are you overwhelmed by the number of tools and approaches you can use to organize your reproducible research? If yes, you’re in the right place!
This guide presents you with pre-selected practical recommendations and useful resources on how to conduct a fully reproducible neuroimaging research project. The guide covers topics such as basics of reproducibility, preregistration, data/code organization and sharing, and building reproducible neuroimaging workflows.
We built this guide based on subjectively selected resources and experiences of the Computational Neuroimaging Team at Nicolaus Copernicus University in Toruń; thus, not all of our recommendations might be suitable for all labs. Let us know if you find that we missed something essential, or we can improve our workflow!
Check out our Guide to Reproducible Neuroimaging Research
Check out our Guide to Reproducible Neuroimaging Research
- Table of Contents
- Introduction
- Preregistration
- Data sharing
- Data structure
- Data management
- Analysis workflow
- Code management
The progress of science is based on valuable research. While reading the research paper you naturally trust the researchers that the presented results are true and that you can build your research upon them. Recent reports show, however, that scientists are not able to reproduce a large amount of published research. In this section we introduce you to the concept of reproducibility in research, reasons of reproducibility crisis, and motivations to lead reproducible research. Finally, we present you three steps that might improve reproducibility in your research.
Preregistration is the practice of registering your detailed research plan before conducting a study. The preregistered report format requires researchers to submit a description of the confirmatory hypotheses, variables, study methods, and analysis plan prior to data collection.
This practice allows researchers to circumvent the publication bias toward significant findings and prevent the data from taking you hostage. Preregistration also makes the distinction between hypothesis testing and exploratory (hypothesis generating) research more clear. As a result, the obtained results won’t affect the hypothesis and vice versa.
Reporting details about your scientific methods is no longer sufficient to address the complex relationship between science and society. More and more funding bodies require scientists to make their data public after the end of the study. Sharing data in open repositories enables other scientists to reuse your data to answer their research question or to develop new analysis techniques. As a result, society could benefit as much as possible from carrying every single scientific project.
Optimism about propagating reproducible science is challenged by the complex ethical, legal, and social issues it raises. Below, we provide explicit suggestions you may fulfill to change how you conduct the study, before putting your neuroimaging data into a public repository.
Neuroimaging experiments generate a complex set of data that can be organized in many different ways. For many years, the research community has been trying to find a way to effectively store and manage the data collected from the fMRI scanner.
DataLad is a software tool designed to help with anything related to the version control of of digital objects.
BIDSApps, fMRIPrep, etc.
Neuroimaging data analysis involves generation of a code that allows interpretation and validation of the scientific methods and results as well as solving new research questions. Most researchers are not trained in software engineering which often causes undocumented and disorganized code, however sharing an imperfect analysis code is still much better than not sharing at all. (Gorgolewski et al. 2016)
Here, we aim to provide the most optimal scientific practices where data science code quality is focused on correctness and reproducibility. Starting with a fairly standardized setup provides a clean, logical structure of the code.
Contributions of any kind welcome! If there is any possibility that you think, you can improve this project - contribute with us! Report issues and create pull requests via GitHub. We are open to collaborate and create reliable source for reproducible neuroimaging.
There are many ways to get in touch with us! Please see our Contact Page if you want to contact with us. There is also a possibility to contact us by our social media (Twitter, ResearchGate, etc.). If you want to find us, you will!