Command Line

After defining your suites and/or checks, you can run them as well as other actions from the command line.

Run Checks

To run checks, use the data_checks.do.run_check command:

usage: python -m data_checks.do.run_check [-h] [--parallel] [--schedule SCHEDULE] 
                                          [--error_logging]
                                          [--alerting]
                                          {CheckClass1}
                                          [{CheckClass1,CheckClass2, ...} ...]

The run_checks command takes in the following arguments:

• checks: List of checks to run by class name.
• --parallel: Run checks and nested rules in parallel. Before enabling see Warning on Fully Parallel Executions. If not specified, checks will be run sequentially.
• --schedule: CRON schedule for the checks. If not specified, checks will be run once.
• --error_logging: Log errors to the console. This may duplicate some error logs in the database. If not specified, errors will only be stored in the database.
• --alerting: Send alerts to the specified endpoint. If not specified, no alerts will be sent.

For example to run MyFirstDataCheck in parallel every minute, log errors to the console, and send alerts to the specified endpoint we would use the following command:

python -m data_checks.do.run_check MyFirstDataCheck --error_logging --alerting --parallel --schedule "* * * * *"

Run Suites

To run suites, use the data_checks command:

usage: python -m data_checks [-h] [--only ONLY] 
                             [--exclude {ConsistencySuite} [{ConsistencySuite} ...]]
                             [--parallel] [--deploy] [--error_logging] [--alerting]

The data_checks command runs all the suites specified in SUITES_MODULE. The command can be customized by passing in the following arguments:

• --only: Only run the specified suite. If not specified, all suites will be run.
• --exclude: Exclude the specified suites. If not specified, no suites will be excluded.
• --parallel: Run suites in parallel. This will run each nested check in parallel and each nested rule in parallel. Before enabling see Warning on Fully Parallel Executions. If not specified, suites will be run sequentially.
• --deploy: Creates database rows for suites, checks, and rules. Then deploys suites and creates a new rule execution row per execution. If not specified, no database rows will be created and will be run once.
• --error_logging: Log errors to the console. This may duplicate some error logs in the database. If not specified, errors will only be stored in the database.

For example to run all suites in every minute, log errors to the console, and deploy the suites with we would use the following command:

python -m data_checks --error_logging --deploy

Silencing Checks' Rules

To silence rules, use the data_checks.do.silence_check command:

usage: python -m data_checks.do.silence [-h] [--until UNTIL] [--delta DELTA] [--hash HASH]

The silence_check command takes in the following arguments:

• --until: Date until which the rule will be silenced. Format should be YYYY-MM-DD:HH:mm:ss.
• --delta: Time delta for which the rule will be silenced. Format: 1h, 1d, 1m, 1w (hour, day, minute, week). Example: 3h for 3 hours.
• --hash: Hash of the rule (stored in the database) to silence. See Database for more information.

Important

Either --until or --delta must be specified. If both are specified, --until will be used. If neither are specified, an error will be thrown.

Analytics

This library comes with a few built-in analytics that can be accessed via the data_checks.do.analytics command:

usage: python -m data_checks.do.analytics [-h] [analytic_func]

The analytics command takes in the following arguments:

• analytic_func: Name of the analytic function to run. If not specified, all analytics will be run.

The available analytics are:

• get_executions_since: Get all executions since yesterday.
• get_silenced_rules: Get all the latest silenced rules.
• get_latest_rules: Get the latest rules in the database.
• get_prev_next_executions: Get the previous and next execution for each suite.
• get_status_counts: Get the success, failure, and total counts the executions of each rule since yesterday.