Other general considerations
Fast, preliminary results vs. slow, precise results.
Some analysis might be slow to run (bootstrapping, some of the diffusion analyses, etc.). Some descriptions note that the faster solutions might be preferred. From the viewpoint of published results, the most precise available solution should be preferred. (A research project takes weeks or more frequently months or even years to be completed, then published. In this time frame even the relatively slow data processing solutions are rather fast (usually taking minutes or occasionally hours to run). So after spending some months with a work, a researcher probably wouldn't want to lose quality to save a few minutes.) Still, in some exploratory phase of the research, faster but less precise solutions might be reasonable options to choose. But these faster and more imprecise results should be used only for a first fast check and not for publications.
- For this reason CogStat may provide two solutions. A "fast preliminary" version and a "slow precise" version
- The user might choose between them on the appropriate dialog with a checkbox (or with other simple solution).
- Because CogStat offers a fast way to check many aspects of the data, the "fast preliminary" option could be the default option.
- When running the fast preliminary version, the output always should warn the user that "This is a fast preliminary calculation. Use the Precise option for calculating the to be published results."
Present the results of suboptimal procedures
Usually for the CogStat pipeline best practice solutions are preferred. However, in some cases it might make sense to use less ideal solutions, when those solutions are widespread and can help readers to compare the actual results with former findings. In those cases the output should warn the user that these suboptimal solutions should be used only for comparing the present result with former results.