Enabling the Verification of Computational Results
We conducted an experiment on the computational physics community to measure 1) how frequently authors shared their code and data directly in their article; 2) Whether authors who didn't would respond to a request for code and data; 3) How far a user skilled in computation, but not an expert in the field could get with the materials received.
procedures_and_descriptions/methodology.md is a description of our research methodology.
procedures_and_descriptions/column_descriptions.md is a description of the columns in our data format.
data/JCP_Dataset.csv an anonymized database of our findings which can be used to get the statistics we report in our article.
scripts/PRECSNumbers.py a script to produce the reported numbers from the included database
scripts/run.sh a script to run the numbers analysis of the database.
The results published in our article were computed using python 3.6.4 in the Arch computing environment. The machine was a desktop with an Intel Core i7-6900K CPU, and two Nvidia GeForce GTX 1080 GPUs.