dccvalidator

dccvalidator is a package and Shiny app to perform data validation and QA/QC. It’s used in the AMP-AD and PsychENCODE consortia to validate data prior to data releases.

Installation

You can install dccvalidator from CRAN:

install.packages("dccvalidator")

To install the development version from GitHub, run:

devtools::install_github("Sage-Bionetworks/dccvalidator")

Many functions in dccvalidator use reticulate and the Synapse Python client. See the reticulate documentation for information on how to set R to use a specific version of Python if you don’t want to use the default Python installation on your machine. Whichever Python installation you choose should have synapseclient installed.

Because dccvalidator uses reticulate, it is not compatible with the synapser package..

Check data

dccvalidator provides functions for checking the following common data quality issues:

• Annotation keys and values conform to a controlled vocabulary
• Column names match those of an associated metadata template
• Certain columns are not empty
• Certain columns are complete
• Identifiers match between two metadata files (e.g. all individuals in one file are also present in another)
• Check that identifiers are unique within a file

Data submission validation

This package contains a Shiny app to validate manifests and metadata for AMP-AD studies. It uses the dccvalidator package to check for common data quality issues and gives realtime feedback to the data contributor on errors that need to be fixed. The reporting UI is heavily inspired by the MetaDIG project’s metadata quality reports.

The application also allows users to submit documentation of their study, a description of the methods used, etc.

See the customizing dccvalidator vignette for information on how to spin up a customized version of the application