Skip to content

Latest commit

 

History

History
30 lines (20 loc) · 2.39 KB

badges.md

File metadata and controls

30 lines (20 loc) · 2.39 KB
Raw

Badges and Evaluation Criteria

The artefact evaluation process is based on the artefact Review and Badging Version 1.1.

Available

  • Author-created artefacts relevant to the paper are publically accessible;
  • Relevant data sets are provided or are accessible where possible;
  • A DOI or link to an open-source code repository along with a unique identifier for the object is provided.
  • Tagged version or release for the code used to produce the paper.
  • An open-source license has been applied (for guidance, see https://choosealicense.com/)

For example, Zenodo can be used to deposit a snapshot of the software, providing a DOI.

Functional

The artefacts associated with the research are documented, consistent with the publication, complete, and can be used (exercisable).

  • Documented: At minimum, an inventory of artefacts is included, and sufficient description provided to enable the artefacts to be run.
  • Consistent: The artefacts are relevant to the associated paper, and contribute in some inherent way to the generation of its main results.
  • Complete: To the extent possible, all components relevant to the paper in question are included. (Any proprietary artefacts need not be included or data sets deemed to be too large. If any of these are required to exercise the artefact then this should be documented. Proxies for proprietary or large data should be included (e.g., alternate data sets, or smaller subsets of the data) so as to demonstrate the analysis and give evidence of its veracity.)
  • Exercisable: Scripts and/or software is provided to generate the results of the associated paper (e.g, figures, tables) and can be successfully executed.

Reusable

The artefacts associated with the paper are of a quality that significantly exceeds minimal functionality. That is, they are functional, but are carefully documented and well-structured to the extent that reuse and repurposing is facilitated.

  • Artefacts are carefully documented and well-structured to the extent that reuse and repurposing is facilitated, for example, full documentation of the API / package.
  • Norms and standards of the research community for artefacts of this type are strictly adhered to (e.g., FAIR data and FAIR4RS).
  • The artefact is packaged to enable its reused as a component in another project.