RADx Data Dictionary Validator 1.2.0

Changed to use Java 17 instead of Java 20 for compatibility with RADx Data Hub Java version.

Installation

To install download ddv.zip and then unzip. Move the extracted ddv tool to into a location on your executable path.

Examples

To use, type ddv --help for a list of options.

RADx Data Dictionaries should be in the RADx Data Dictionary format, which is a CSV based format that is described here.

To validate a data dictionary called mydictionary.csv, type ddv --in mydictionary.csv. The validation report will be written to stdout in TSV format.

To output CSV format add the --format CSV flag. For example,

ddv --in mydictionary.csv --format CSV

To write to a file, you can either, redirect output, for example,

ddv --in mydictionary.csv --format CSV > myreport.csv

or you can specify the --out option, for example,

ddv --in mydictionary.csv --format CSV --out myreport.csv

Programmatic Use

To use in your app add the following Maven dependency.

<dependency>
            <groupId>edu.stanford.bmir.radx</groupId>
            <artifactId>radx-data-dictionary-validator-lib</artifactId>
            <version>1.2.0</version>
</dependency>

Add edu.stanford.bmir.radx.datadictionary to the list of packages that are scanned, for example,

@SpringBootApplication
@ComponentScan(basePackages = "edu.stanford.bmir.radx.datadictionary")
MyApp { ... }

Next, inject the Validator bean where you need it. For example,

@Component
public class ValidateIt {

    private final Validator validator;

    public ValidateIt(Validator validator) {
        this.validator = validator;
    }

    public void doValidation(InputStream in) throws IOException {

        // Parse our data dictionary csv
        var parser = new CsvParser();
        var csv = parser.parseCsv(in);

        // Validate the data dictionary
        var validationReport = validator.validateDataDictionary(csv);
        var validationResults = validationReport.results();

        // Process the validation report as needed
        var errorCount = validationResults.stream()
                .filter(result -> result.validationLevel().equals(ValidationLevel.ERROR))
                .count();

        System.out.println("There are " + errorCount + " errors");

        // Each report contains a list of results
        validationReport.results()
                .forEach(validationResult -> {
                    // For each validation result we have:

                    // Validation level:  ERROR, WARNING, INFO.  WARNING and INFO provide
                    // guidance on how to clean up a data dictionary
                    var validationLevel = validationResult.validationLevel();

                    // The zero-based index of the row in the data dictionary CSV that
                    // the result pertains to
                    // Note that 0 is the header row
                    var rowIndex = validationResult.getRowNumber();

                    // The complete row that the result pertains to
                    var row = validationResult.csvRow();

                    // The human-readable name of the validation result
                    var name = validationResult.name();

                    // The human-readable message of the validation result
                    var message = validationResult.message();

                    // The subject of the validation result.  This is the string (cell value)
                    // that the result pertains to
                    var subject = validationResult.subject();

                });
    }
}