dataMaid is an R package for documenting and creating reports on data cleanliness.
This github page contains the development version of dataMaid. For the latest stable version download the package from CRAN directly using
To install the development version of dataMaid run the following
commands from within R (requires that the
devtools package is already installed)
A super simple way to get started is to load the package and use the
makeDataReport function on a data frame (if you try to generate several
reports for the same data, then it may be necessary to add the
argument to overwrite the existing report).
library(dataMaid) data(trees) makeDataReport(trees)
This will create a report with summaries and error checks for each
variable in the
trees data frame. The format of the report depends on your OS and whether
you have have a LaTeX installation on your computer, which
is needed for creating pdf reports.
Using dataMaid interactively
The dataMaid package can also be used interactively by running checks for the individual variables or for all variables in the dataset
data(toyData) check(toyData$events) # Individual check of events check(toyData) # Check all variables at once
By default the standard battery of tests is run depending on the
variable type. If we just want a specific test for, say, a numeric
variable then we can specify
that. All available checks can be viewed by calling
the documentation for
an overview of the checks available or how to create and include your own tests.
check(toyData$events, checks = setChecks(numeric = "identifyMissing"))
We can also access the graphics or summary tables that are produced for a variable by calling the
summarize functions. One can visualize a single variable or a full dataset:
#Visualize a variable visualize(toyData$events) #Visualize a dataset visualize(toyData)
The same is true for summaries. Note also that the choice of checks/visualizations/summaries are customizable:
#Summarize a variable with default settings: summarize(toyData$events) #Summarize a variable with user-specified settings: summarize(toyData$events, summaries = setSummaries(all = c("centralValue", "minMax"))
This manuscript has been accepted for publication in JSS and it
provides a detailed introduction to the dataMaid package. At one point
it will be added as a vignette. Moreover, we have created a vignette that describes how to extend dataMaid to include user-defined data screening checks, summaries and visualizations. This vignette is called
We are currently working on an online version of the tool, where users can upload their data and get a report. A prototype is already up and running - we just need to configure the R server correctly.
Until we have set it up online, you can try it out on your own machine: