Database

Suites, checks, rules, and rule executions are stored in a database. The database is specified in the CHECKS_DATABASE_URL variable in your settings file. Any database that is supported by SQLAlchemy can be used. This library generates the following tables in the database:

Suite Table

Stores data related to suites.

id (INT)	name (VARCHAR)	code (TEXT)	config (TEXT)	created_at (TIMESTAMPTZ)
001	Suite1	def ...	{"schedule": "* */2 * * *"}	2023-08-22 00:00:00.359828-00
002	Suite2	def ...	{"schedule": "* * * * *"}	2023-08-22 01:00:00.359828-00
003	Suite3	def ...	{"schedule": "* */3 * * *"}	2023-08-22 02:00:00.359828-00
004	Suite4	def ...	{"schedule": "* * * * *"}	2023-08-22 03:00:00.359828-00

• id: Unique identifier for the suite.
• name: Name of the suite.
• code: Code of the suite.
• config: Configuration (in JSON) of the suite.
• created_at: Timestamp of when the suite was created.

Check Table

Stores data related to checks.

id (INT)	name (VARCHAR)	code (TEXT)	excluded_rules (VARCHAR)	config (TEXT)	created_at (TIMESTAMPTZ)
001	Check1	def ...	["rule1", "rule2", "rule3"]	{..., "rules_config": {...}}	2023-08-22 00:00:00.359828-00
002	Check2	def ...	["rule1", "rule2", "rule3"]	{..., "rules_config": {...}}	2023-08-22 01:00:00.359828-00
003	Check3	def ...	["rule1", "rule2", "rule3"]	{}	2023-08-22 02:00:00.359828-00
004	Check4	def ...	["rule1", "rule2", "rule3"]	{..., "rules_config": {...}}	2023-08-22 03:00:00.359828-00

• id: Unique identifier for the check.
• name: Name of the check.
• code: Code of the check.
• excluded_rules: List of rules to exclude from the check.
• config: Configuration (in JSON) of the check.
• created_at: Timestamp of when the check was created.

Rule Table

Stores data related to rules.

id (INT)	check_id (INT)	suite_id (INT)	name (VARCHAR)	hash (TEXT)	severity (NUMERIC)	code (TEXT)	config (TEXT)	created_at (TIMESTAMPTZ)
001	001	NULL	Rule1	RULE_HASH1	1	def ...	{"silenced_until": ...}	2023-08-22 00:00:00.359828-00
002	002	002	Rule2	RULE_HASH2	2	def ...	{"silenced_until": ...}	2023-08-22 01:00:00.359828-00
003	003	NULL	Rule3	RULE_HASH3	3	def ...	{}	2023-08-22 02:00:00.359828-00
004	004	004	Rule4	RULE_HASH4	4	def ...	{}	2023-08-22 03:00:00.359828-00

• id: Unique identifier for the rule.
• check_id: ID of the check the rule belongs to.
• suite_id: ID of the suite the rule belongs to.
• name: Name of the rule.
• hash: Hash of the rule. In the following format:

suite:SUITE_NAME::check:CHECK_NAME-group_element::rule:RULE_NAME::params:{args: [ARG1, ARG2, ...], kwargs: {key1: value1, key2: value2, ...}}

• severity: Severity of the rule.
• code: Code of the rule.
• config: Configuration (in JSON) of the rule.
• created_at: Timestamp of when the rule was created.

Rule Execution Table

Stores data related to rule executions.

id (INT)	rule_id (INT)	status (VARCHAR)	params (TEXT)	logs (TEXT)	traceback (TEXT)	exception (TEXT)	created_at (TIMESTAMPTZ)	finished_at (TIMESTAMPTZ)
001	001	success	{"args": [], "kwargs": {}}	hellow world ...	NULL	NULL	2023-08-22 00:00:00.359828-00	2023-08-22 00:00:00.359828-00
002	002	failed	{"args": [], "kwargs": {}}	NULL	lorem ipsum dolor sit ...	lorem ipsum dolor sit ...	2023-08-22 01:00:00.359828-00	2023-08-22 01:00:00.359828-00
003	003	success	{"args": [], "kwargs": {}}	hellow world ...	NULL	NULL	2023-08-22 02:00:00.359828-00	2023-08-22 02:00:00.359828-00
004	004	failed	{"args": [], "kwargs": {}}	NULL	lorem ipsum dolor sit ...	lorem ipsum dolor sit ...	2023-08-22 03:00:00.359828-00	2023-08-22 03:00:00.359828-00

• id: Unique identifier for the rule execution.
• rule_id: ID of the rule the rule execution belongs to.
• status: Status of the rule execution.
• params: Params of the rule execution.
• logs: Logs of the rule execution.
• traceback: Traceback of the rule execution.
• exception: Exception of the rule execution.
• created_at: Timestamp of when the rule execution was created.
• finished_at: Timestamp of when the rule execution finished.

Querying the Database

You can access these tables using SQLAlchemy. Start by importing session_scope and creating a session context. This will automatically create a database session and close it when the context is exited. Then import the models from data_checks.database and use them to query the database. For example:

from data_checks.database.utils.session_utils import session_scope
from data_checks.database import Suite, Check, Rule, RuleExecution

with session_scope() as session:
    print(session.query(Rule).all())
    print(session.query(RuleExecution).all())
    print(session.query(Check).all())
    print(session.query(Suite).all())

# Output:
# [Rule(id=1, name='rule_my_first_successful_rule'), Rule(id=2, name='rule_my_first_failed_rule'), Rule(id=3, name='rule_with_required_arguments')]
# [RuleExecution(id=1), RuleExecution(id=2), RuleExecution(id=3)]
# [Check(id=1, name='MyFirstDataCheck')]
# [Suite(id=1, name='MyFirstDataSuite')]

There are a few pre-built queries that you can use to query the database. These (and other actions like silencing) can be executed through the Command Line.