[Advanced] Suites

Data Suite

The library exposes the DataSuite class (defined in data_checks/classes/data_check.py) which you can use to write your checks. Begin by subclassing the DataSuite class:

from data_checks.classes.data_suite import DataSuite

class MyFirstDataSuite(DataSuite):
    pass

Then override the required class methods:

from data_checks.classes.data_suite import DataSuite
from data_checks.dataset import Dataset
from my_checks import MySecondDataCheck

class MyFirstDataSuite(DataSuite):
    @classmethod
    def checks_overrides(cls) -> dict | None:
        """
        Override the parameters of Checks' rules.
        """
        return {
            "MyFirstDataCheck": {
                "rule_my_first_successful_rule": {
                    "data": "Hello World from Suite"
                }
            }
        }

    @classmethod
    def suite_config(cls) -> dict:
        """
        Define the suite's configuration. This will be stored in the database.
        You can attach any configuration option as long as it is JSON serializable.
        Like checks, you can also define additional features for your suite.
        """
        return {
            "schedule": "* * * * *",
        }

    @classmethod
    def checks(cls) -> list[type | str | Check]:
        """
        Define the checks to be run by the suite. Checks can be specified 
        by passing in the class, the class name, or an instance of the class. 
        If the check is specified by the class or class name, the suite will 
        instantiate the check. If the check is specified by an instance 
        of the class, the suite will use the instance as is.
        """
        return [
            "MyFirstDataCheck",
            MySecondDataCheck(content="Hello") # Within this check, `self.content` will be defined to be "Hello"
        ]

In the above example we define several methods:

• checks(cls) -> dict | None: Defines the checks to be run by the suite. Checks can be specified by passing in the class, the class name, or an instance of the class. If the check is specified by the class or class name, the suite will instantiate the check. If the check is specified by an instance of the class, the suite will use the instance as is. This last option is useful if you want to pass custom instance fields that the check will use.
• checks_overrides(cls) -> dict | None: Override the parameters of Checks' rules (if desired/required). Returns a dictionary where the key is the name of the check and the value in the same format of rules_params (see Instantiating and Running Checks).
• suite_config(cls) -> dict: Define the suite's configuration. This will be stored in the database. You can attach any configuration option as long as it is JSON serializable. Like checks, you can also define additional features for your suite. For example you may want to define a description for the suite. This library provides one built-in feature schedule which allows you to schedule the suite to run at a specific time based on the provided cron expression. For example, the above suite will run every minute.

Defining Suites in SUITES_MODULE

Your suite should be written inside the specified SUITES_MODULE in your settings file. For example, if you set SUITES_MODULE = "my_suites", then you should write your check in my_suites/my_first_data_suite.py. Make sure that SUITES_MODULE and any nested modules are properly defined as directories (i.e. have an __init__.py file). Doing this allows the library to automatically find and run suites.

Instantiating and Running Suites

DataSuite also defines additional methods as well as instance variables that can are used to instantiate and run suites. The DataSuite class exposes the following instance variables:

• name: The name of the suite. This is used to identify the suite in the database. If not provided, the name of the class is used.

You can pass these arguments to the DataSuite constructor. For example:

from my_suites.my_first_data_suite import MyFirstDataSuite

suite = MyFirstDataSuite(
    name="My First Data Suite"
)

creates a suite with the name "My First Data Suite".

You can call a few built-in methods on the suite. These are:

• get_checks(self) -> list[Check]: Get all checks in the suite.
• run_all(self): Run all checks in the suite synchronously.
• run_all_async(self): Run all checks in the suite asynchronously.

More Advanced Suites

The DataSuite class is a simplified and beginner friendly subclass of the base Suite class (data_checks/base/suite.py). The user can also directly subclass the Suite class to create more advanced suites (see References).