Skip to content
BranchesTags

Repository files navigation

Data Quality Gate — v2

A configurable data quality engine for Snowflake that runs as a CI/CD gate. Define checks in JSON, execute them against your warehouse, and fail the pipeline when data quality degrades.

DQ Gate solves a specific problem: data pipelines succeed at the infrastructure level (jobs complete, rows load) but silently produce stale, duplicated, or invalid data. This tool lets teams codify their data quality expectations and enforce them automatically, without writing any code.

Features

  • Seven built-in check types — freshness, multi-column uniqueness, custom SQL, schema validation, volume spike, referential integrity, and row count
  • Configuration-driven — all rules defined in JSON files; no code changes needed to add or modify checks
  • Concurrent execution — checks run in parallel, bounded by a configurable semaphore (default: 8 concurrent)
  • Multi-format reporting — HTML, JSON, and CSV report writers; old reports are auto-purged after 30 days
  • Notification channels — Email (SMTP), Pushover, and Microsoft Teams with configurable trigger conditions
  • Incident deduplication and lifecycle tracking — failures are fingerprinted and tracked in a persistent SQLite store; each unique failure opens a single Active incident (New ? Ongoing ? Resolved) and re-notifies only after a configurable interval
  • Table profiling and check suggestion — profile a table or entire schema to generate column-level statistics and a ready-to-paste check configuration
  • Connection resilience — automatic retry with exponential backoff for transient Snowflake failures via Polly
  • SQL safety — identifier sanitization and a statement guard that blocks destructive DML/DDL in custom queries
  • Tag-based filtering — run subsets of checks by tag without maintaining separate config files
  • Dry-run mode — validate all configuration without making any database connections
  • Secret store integration — load sensitive connection credentials from an encrypted file-based secret store
  • Structured logging — Serilog with file and console sinks, enriched with run ID and check context

High-Level Architecture

graph TD
    CLI["CLI Layer — DataQuality.Cli"] --> Core["Business Logic — DataQuality.Core"]
    CLI --> SF["Database Adapter — DataQuality.Snowflake"]
    SF --> Core

    subgraph Core
        Config["Configuration and Validation"]
        Orch[CheckOrchestrator]
        Runners[Check Runners]
        Reports[Report Writers]
        Notify[Notification Channels]
        Incidents[Incident Store]
        Profiler[Table Profiler]
    end

    CLI -- "builds host, wires DI" --> Orch
    Orch -- "resolves via factory" --> Runners
    Runners -- "IConnectionFactory" --> SF
    Orch -- "after execution" --> Reports
    Orch -- "after execution" --> Notify
    Orch -- "per result" --> Incidents
Loading
Component Role
DataQuality.Cli Entry point. Parses CLI arguments, builds the DI container, delegates to the orchestrator.
DataQuality.Core All business logic. Check definitions, runners, orchestration, reporting, notifications, incident tracking, validation, and profiling.
DataQuality.Snowflake Implements IConnectionFactory and IConnection for Snowflake. Wraps connections in a Polly resilience pipeline.

The dependency direction is strict: Cli ? Core and Snowflake; Snowflake ? Core. Core has zero infrastructure dependencies.

Execution Flow

sequenceDiagram
    participant CLI as Program.cs
    participant Host as DI Container
    participant Orch as CheckOrchestrator
    participant Factory as CheckRunnerFactory
    participant Runner as ICheckRunner
    participant DB as Snowflake
    participant IS as IncidentStore

    CLI->>Host: Build host, register services
    Host->>Host: Load and validate JSON configs
    CLI->>Orch: RunAsync()
    Orch->>Orch: Filter by tags, split enabled/disabled
    loop Each enabled check — concurrent
        Orch->>Factory: GetCheckRunner(type)
        Factory->>Runner: Resolve keyed singleton
        Runner->>DB: Execute SQL
        DB-->>Runner: Scalar or rows
        Runner-->>Orch: Result<CheckResult>
    end
    Orch->>IS: Upsert incidents per result
    Orch->>Orch: Aggregate, determine ExitCode
    Orch->>Host: WriteReportsAsync()
    Orch->>Host: NotifyAsync()
    Orch-->>CLI: ExitCode
Loading
  1. JSON files from the config directory are deserialized into polymorphic CheckConfiguration objects.
  2. CheckConfigurationValidator invokes each config's Validate() and throws on any errors before any checks run.
  3. CheckOrchestrator filters by tags and enabled state, then runs checks concurrently via Task.WhenAll bounded by a SemaphoreSlim(8).
  4. Each runner sanitizes SQL identifiers, executes queries, and returns FluentResults.Result<CheckResult>.
  5. CheckResultProcessor converts raw results into outcomes (Pass, Warn, Fail, Error).
  6. IncidentStore fingerprints each result and upserts incident lifecycle records in SQLite.
  7. ReportService writes reports in all requested formats and purges files older than 30 days.
  8. NotificationService dispatches to channels whose trigger conditions are met.
  9. The exit code propagates to the shell — non-zero on any failure or error.

Project Structure

DQ_GATE.slnx
src/
  DataQuality.Cli/               Application host and CLI
    Commands/                    System.CommandLine subcommands (run, list-rules, profile)
    Configuration/               Secret store provider
    Hosting/                     Host builder extensions, AppRunner
    Output/                      Console writer implementations
    Config/                      Check definition JSON files

  DataQuality.Core/              All business logic
    Configuration/Checks/        Polymorphic CheckConfiguration base class
    Configuration/Notifications/ Notification config DTOs
    Connections/                 IConnection, IConnectionFactory interfaces
    Features/                    Check runners and their configurations
    Notifications/               NotificationService and channel implementations
    Orchestration/               CheckOrchestrator, result processing, ExitCode
    Profiling/                   Table profiler, column stats, check suggestor
    Reporting/                   ReportService, HTML/JSON/CSV writers
    Results/                     CheckResult, CheckStatus, error and warning types
    Settings/                    ApplicationSettings, IncidentStoreSettings
    Validation/                  SQL sanitization and statement guard

  DataQuality.Snowflake/         Snowflake adapter
    Resilience/                  Polly retry pipeline and settings
    SnowflakeConnection.cs       IConnection implementation
    SnowflakeConnectionFactory.cs IConnectionFactory implementation

tests/
  DataQuality.Tests/             xUnit + NSubstitute + FluentAssertions

Getting Started

Prerequisites

  • .NET 10 SDK
  • A Snowflake account with a user that has SELECT access to the tables you want to check

Build and run

dotnet build
dotnet test

# Run all checks in the default Config/ directory
dotnet run --project src/DataQuality.Cli -- run

# Target a specific config directory
dotnet run --project src/DataQuality.Cli -- run --config-dir path/to/checks/

# Dry-run: validate config without connecting to Snowflake
dotnet run --project src/DataQuality.Cli -- run --dry-run

# Run only checks tagged "daily"
dotnet run --project src/DataQuality.Cli -- run --tags daily

# Write HTML + JSON reports to a custom directory
dotnet run --project src/DataQuality.Cli -- run --output html json --report-dir /tmp/reports

# Print all configured rules and exit
dotnet run --project src/DataQuality.Cli -- list-rules

# Profile a single table and display column statistics
dotnet run --project src/DataQuality.Cli -- profile MyConnection MYDB.PUBLIC.MY_TABLE

# Profile an entire schema and emit suggested check JSON
dotnet run --project src/DataQuality.Cli -- profile MyConnection MYDB.PUBLIC --output json

CLI Reference

DQ Gate uses subcommands. The top-level binary accepts one of three commands: run, list-rules, or profile.

run — Execute checks

The primary command. Loads check configs, runs all enabled checks, writes reports, and dispatches notifications.

Option Alias Default Description
--config-dir -cd Config Directory containing JSON check configuration files
--report-dir -rd Reports Directory to write generated reports
--output -o html Report format(s): html, json, csv (space or comma-separated; repeatable)
--tags -t (all) Filter checks by tag(s); only checks matching at least one tag are executed
--dry-run -d false Validate configuration without executing any checks
--silent -s false Suppress all console output; process exit code is the only signal
--secretfile -sf ~/.secretstore Path to the encrypted secrets file for connection credentials
--secretpassword -sp (env) Password for the secrets file

The --secretfile and --secretpassword options also read from SECRETSTORE_PATH and SECRETSTORE_PASSWORD environment variables respectively.

list-rules — Inspect loaded configuration

Loads all check configuration files from the default Config/ directory, prints a formatted summary of every rule, and exits without making any database connections.

dotnet run --project src/DataQuality.Cli -- list-rules

profile — Table profiler

Connects to Snowflake, collects column-level statistics (null rate, distinct count, min/max, sample values), and either displays them in the console or emits a ready-to-paste check configuration in JSON format.

profile <connection> <target> [options]
Argument / Option Description
connection Logical connection name from ConnectionStrings in configuration
target db.schema.table (single table) or db.schema (all tables in schema)
--output / -o console (default) or json (emits suggested check config)
--no-limit Scan the full table; by default the profiler samples up to 1,000,000 rows
--secretfile / -sf Path to secrets file
--secretpassword / -sp Password for secrets file 
# Display column stats for a single table
dotnet run --project src/DataQuality.Cli -- profile Snowflake ANALYTICS.PUBLIC.ORDERS

# Generate suggested check config for every table in a schema
dotnet run --project src/DataQuality.Cli -- profile Snowflake ANALYTICS.PUBLIC --output json > checks.json

Configuration

Connection strings

Connections are declared in appSettings.json under ConnectionStrings. The key is the logical name referenced in check definitions via the "connection" property.

{
  "ConnectionStrings": {
    "Snowflake": "account=myaccount;user=myuser;password=mypassword;warehouse=COMPUTE_WH;db=MYDB;schema=PUBLIC"
  }
}

Sensitive values (passwords, private keys) must not be committed. Use the secret store, User Secrets locally, or environment variables in CI:

# Local development — .NET User Secrets
dotnet user-secrets set "ConnectionStrings:Snowflake" "account=...;password=..." --project src/DataQuality.Cli

# CI/CD — environment variable
$env:ConnectionStrings__Snowflake = "account=...;password=..."

Snowflake resilience

The Polly retry pipeline is configured under Resilience in appSettings.json:

{
  "Resilience": {
    "MaxRetryAttempts": 3,
    "BaseDelaySeconds": 2,
    "UseExponentialBackoff": true
  }
}
Setting Default Description
MaxRetryAttempts 3 Maximum retry attempts before propagating the exception
BaseDelaySeconds 2 Base delay between retries in seconds
UseExponentialBackoff true Doubles the delay on each attempt when true; constant delay when false

Incident store

DQ Gate maintains a SQLite database to deduplicate and track the lifecycle of failures:

{
  "IncidentStore": {
    "DatabasePath": "incidents.db",
    "DefaultRenotifyHours": 24
  }
}
Setting Default Description
DatabasePath incidents.db Path to the SQLite file. Relative paths resolve from the current working directory.
DefaultRenotifyHours 24 Hours before an Ongoing incident triggers another notification.

Per-check override: add "renotify_hours" to any check definition to override the global default.

Logging

Logging is configured via the Serilog section in appSettings.json. The default configuration writes to the console. Add a file sink for persistent logs:

{
  "Serilog": {
    "MinimumLevel": { "Default": "Information" },
    "WriteTo": [
      { "Name": "Console" },
      {
        "Name": "File",
        "Args": { "path": "logs/dqgate-.log", "rollingInterval": "Day" }
      }
    ]
  }
}

Check Configuration Files

JSON files in the config directory (default: Config/). Each file is a flat JSON array of check objects. All files in the directory are merged at startup.

Every check shares these base properties:

Property Required Description
type ? Check type discriminator (case-sensitive, must match a registered type)
name ? Human-readable name shown in reports
connection ? Logical connection name from ConnectionStrings
target ? Three-part identifier: database.schema.table
enabled true by default; set to false to disable without removing
tags String array for --tags filtering
renotify_hours Per-check override for the global DefaultRenotifyHours

Supported Check Types

Freshness

Asserts that a timestamp column's maximum value is within an acceptable age from the current time.

{
  "type": "Freshness",
  "name": "Orders Fresh",
  "connection": "Snowflake",
  "target": "SALES.PUBLIC.ORDERS",
  "tags": ["daily", "sales"],
  "dateColumn": "LAST_UPDATED",
  "maxAgeHours": 24,
  "warnAgeHours": 12,
  "timezone": "America/New_York"
}
Property Required Description
dateColumn ? Timestamp column to evaluate
maxAgeHours ? Hours since the latest value before the check fails
warnAgeHours Hours before maxAgeHours to emit a warning; must be less than maxAgeHours
timezone IANA or Windows timezone ID for relative age calculation (defaults to UTC)

Multi-Column Uniqueness

Detects duplicate value combinations across one or more columns.

{
  "type": "MultiColumnUniqueness",
  "name": "Customer Key",
  "connection": "Snowflake",
  "target": "SALES.PUBLIC.CUSTOMERS",
  "columns": ["CUSTOMER_NUMBER", "SOURCE_SYSTEM"]
}
Property Required Description
columns ? One or more column names; the check counts rows where the combination is not unique

Custom SQL

Runs an arbitrary read-only scalar query and optionally asserts the result against expected values.

{
  "type": "CustomSql",
  "name": "No Negative Sales",
  "connection": "Snowflake",
  "target": "SALES.PUBLIC.ORDERS",
  "query": "SELECT COUNT(*) FROM SALES.PUBLIC.ORDERS WHERE TOTAL < 0",
  "operator": "=",
  "expectedValue": "0",
  "warnOperator": ">",
  "warnValue": "-1"
}
Property Required Description
query ? SQL query returning a single scalar value
operator Comparison operator: =, !=, >, >=, <, <=
expectedValue Expected scalar value (requires operator)
warnOperator Operator for the warning threshold
warnValue Value for the warning threshold (requires warnOperator)

Queries are validated by SqlStatementGuard, which blocks: TRUNCATE, DELETE, DROP, ALTER, UPDATE, INSERT, MERGE, CREATE, REPLACE, EXECUTE, EXEC.

Schema

Validates a table's column structure against an inline definition or a baseline file, detecting schema drift.

{
  "type": "Schema",
  "name": "Orders Schema",
  "connection": "Snowflake",
  "target": "SALES.PUBLIC.ORDERS",
  "allow_extra_columns": true,
  "columns": [
    { "name": "ORDER_ID",     "data_type": "NUMBER",        "nullable": false },
    { "name": "LAST_UPDATED", "data_type": "TIMESTAMP_NTZ", "nullable": true  }
  ]
}
Property Required Description
columns ? (unless baseline_file) Expected column definitions with name, data_type, and nullable
baseline_file ? (unless columns) Path to a JSON baseline file for schema drift comparison
allow_extra_columns When true (default), additional columns in the table do not cause a failure

At least one of columns or baseline_file must be provided.

Volume Spike

Compares the latest partition's row count against historical averages and flags abnormal changes.

{
  "type": "VolumeSpike",
  "name": "Orders Load Volume",
  "connection": "Snowflake",
  "target": "SALES.PUBLIC.ORDERS",
  "partition_column": "LOAD_DATE",
  "threshold_type": "Percent",
  "error_threshold": 50,
  "warn_threshold": 30,
  "direction": "Both"
}
Property Required Description
partition_column ? Date/timestamp column used to define partitions
threshold_type ? Percent or Absolute
error_threshold ? Deviation magnitude that triggers a failure
warn_threshold Deviation magnitude that triggers a warning
direction Up, Down, or Both (default: Both)
comparison_mode How the baseline is computed (e.g., rolling average)
count_column Column to count instead of *
history_limit Number of prior partitions to include in the baseline

Referential Integrity

Detects orphaned child records by validating foreign key relationships against parent tables.

{
  "type": "ReferentialIntegrity",
  "name": "Orders ? Customers FK",
  "connection": "Snowflake",
  "target": "SALES.PUBLIC.ORDERS",
  "relationships": [
    {
      "child_columns":  ["CUSTOMER_ID"],
      "parent_target":  "SALES.PUBLIC.CUSTOMERS",
      "parent_columns": ["CUSTOMER_ID"]
    }
  ]
}
Property Required Description
relationships ? One or more relationship objects
child_columns ? per relationship Column(s) on the child (target) table
parent_target ? per relationship Three-part identifier of the parent table
parent_columns ? per relationship Column(s) on the parent table; must match child_columns count

Row Count

Executes a COUNT(*) (with an optional WHERE filter) and evaluates the result against configurable thresholds.

{
  "type": "RowCount",
  "name": "Orders Has Rows",
  "connection": "Snowflake",
  "target": "SALES.PUBLIC.ORDERS",
  "operator": ">",
  "expectedValue": "0",
  "warnOperator": "<",
  "warnValue": "1000",
  "filter": "STATUS = 'ACTIVE'"
}
Property Required Description
operator ? Comparison operator: =, !=, >, >=, <, <=
expectedValue ? Expected row count value
warnOperator Operator for the warning threshold
warnValue Value for the warning threshold
filter Optional WHERE clause (do not include the WHERE keyword)

Reporting

ReportService dispatches to all registered IReportWriter implementations whose format key matches the requested output formats. Reports are written to the directory specified by --report-dir (default: Reports/). Files older than 30 days are automatically purged on each run.

Format key Description
html Interactive HTML report with summary cards and per-check detail (default)
json Machine-readable JSON; includes incident context fields per result
csv Flat CSV; suitable for import into spreadsheets or dashboards

Request multiple formats on the command line:

dotnet run --project src/DataQuality.Cli -- run --output html json csv

All three formats include per-result incident context: IncidentStatus, IncidentId, IncidentOpenedAt, and IncidentOccurrences.

Incident Tracking

DQ Gate maintains a lightweight SQLite database that deduplicates repeating failures and tracks their lifecycle, eliminating alert fatigue from repeated notifications for the same underlying issue.

How it works

  1. Each failing or warning result is assigned a stable fingerprint (SHA-256 of check type + target + rule-specific fields). Renaming a check does not change its fingerprint.
  2. The first failure for a fingerprint opens an Active incident with status New.
  3. Subsequent failures against the same Active incident advance the status to Ongoing — no duplicate notification is sent unless the re-notify window (DefaultRenotifyHours) has elapsed.
  4. When a check passes after a prior failure, the Active incident is automatically Resolved and a resolved notification is dispatched.

Incident statuses in reports

All three report formats include per-result incident context:

Field Values
IncidentStatus New, Ongoing, Resolved, or empty for passing checks
IncidentId UUID of the active incident (empty when passing)
IncidentOpenedAt UTC timestamp when the incident was first opened
IncidentOccurrences Count of consecutive failures against this incident

Incident notification events

Separate from run-level notifications, incident lifecycle events (Opened, EscalationDue, Resolved) are dispatched through IIncidentNotificationChannel implementations, using the same Email/Pushover/Teams credentials configured in the Notifications block.

Notifications

NotificationService dispatches to all enabled INotificationChannel singletons. Channels are configured under Notifications in appSettings.json. Sensitive values (passwords, tokens) should be loaded from the secret store rather than committed to source control.

All channels share these base properties:

Property Description
enabled Set to true to activate the channel
trigger OnFailure (default), Always, or OnSuccess

Email (SMTP)

{
  "Notifications": {
    "Email": {
      "enabled": true,
      "trigger": "OnFailure",
      "host": "smtp.example.com",
      "port": 587,
      "enableSsl": true,
      "username": "user",
      "password": "${secret:Notifications:Email:Password}",
      "from": "noreply@example.com",
      "to": ["team@example.com"],
      "subject": "DQ Gate – Run Summary",
      "includeReport": false
    }
  }
}
Property Default Description
host SMTP server hostname
port 587 SMTP port
enableSsl true Enable TLS/SSL
username SMTP authentication username
password SMTP authentication password
from Sender address
to List of recipient addresses
subject DQ Gate – Run Summary Email subject line
includeReport false Attach the HTML report to the email

Pushover

{
  "Notifications": {
    "Pushover": {
      "enabled": true,
      "trigger": "OnFailure",
      "appToken": "${secret:Notifications:Pushover:AppToken}",
      "userKey": "${secret:Notifications:Pushover:UserKey}",
      "title": "Data Quality Check Failed"
    }
  }
}

Microsoft Teams

{
  "Notifications": {
    "Teams": {
      "enabled": true,
      "trigger": "Always",
      "webhookUrl": "https://outlook.office.com/webhook/..."
    }
  }
}

Secret Store

DQ Gate integrates with a file-based encrypted secret store to keep credentials out of configuration files and source control.

Using the secret store

Provide the path and password at runtime via CLI options or environment variables:

# Via CLI
dotnet run --project src/DataQuality.Cli -- run `
  --secretfile ~/.secretstore `
  --secretpassword "mypassword"

# Via environment variables (recommended for CI)
$env:SECRETSTORE_PATH     = "/run/secrets/.secretstore"
$env:SECRETSTORE_PASSWORD = "mypassword"

The default secret file location is ~/.secretstore (the user's home directory). The password defaults to the SECRETSTORE_PASSWORD environment variable when not specified on the command line.

Referencing secrets in configuration

Once the store is loaded, secrets are injected into appSettings.json using the ${secret:...} interpolation syntax. The path after secret: maps to the key hierarchy in the store:

{
  "ConnectionStrings": {
    "Snowflake": "${secret:ConnectionStrings:Snowflake}"
  },
  "Notifications": {
    "Email": {
      "password": "${secret:Notifications:Email:Password}"
    }
  }
}

Local development alternative

For local development, prefer .NET User Secrets to avoid maintaining a secret file:

dotnet user-secrets set "ConnectionStrings:Snowflake" "account=...;password=..." `
  --project src/DataQuality.Cli

CI/CD Integration

DQ Gate is designed as a pipeline gate. A non-zero exit code signals failure:

Code Meaning
0 All enabled checks passed (warnings do not fail the gate)
1 One or more checks failed
2 One or more checks produced an error (configuration or connection fault)

GitHub Actions example

- name: Run data quality checks
  env:
    SECRETSTORE_PATH: ${{ secrets.SECRETSTORE_PATH }}
    SECRETSTORE_PASSWORD: ${{ secrets.SECRETSTORE_PASSWORD }}
  run: |
    dotnet run --project src/DataQuality.Cli -- run \
      --config-dir Config/ \
      --report-dir Reports/ \
      --output html json \
      --tags daily

- name: Upload DQ reports
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: dq-reports
    path: Reports/

Azure DevOps example

- task: DotNetCoreCLI@2
  displayName: Run data quality checks
  env:
    SECRETSTORE_PASSWORD: $(SecretStorePassword)
  inputs:
    command: run
    projects: src/DataQuality.Cli/DataQuality.Cli.csproj
    arguments: -- run --tags daily --output html json

Table Profiling

The profile command connects to Snowflake, samples up to 1,000,000 rows per table (configurable), and collects per-column statistics:

  • Null rate
  • Distinct value count and ratio
  • Minimum and maximum values
  • Sample values

With --output json, the profiler passes column stats through ProfileSuggestor, which emits a ready-to-paste JSON check configuration covering freshness, uniqueness, and row count candidates. This is the recommended starting point when onboarding new tables.

# Profile a single table and display stats
dotnet run --project src/DataQuality.Cli -- profile Snowflake MYDB.PUBLIC.ORDERS

# Scan full table (no row limit)
dotnet run --project src/DataQuality.Cli -- profile Snowflake MYDB.PUBLIC.ORDERS --no-limit

# Profile all tables in a schema and generate suggested checks
dotnet run --project src/DataQuality.Cli -- profile Snowflake MYDB.PUBLIC --output json > Config/suggested.json

Extending DQ Gate

Adding a new check type

Every check type lives under Features/<CheckName>/ in DataQuality.Core with two files:

File Purpose
<Name>Configuration.cs Extends CheckConfiguration; declares [JsonPropertyName] config properties; overrides Validate() and Describe()
<Name>Check.cs Implements ICheckRunner; exposes public const string Key; executes SQL; returns Result<CheckResult>

Three registration steps are required:

1. Add a [JsonDerivedType] attribute on CheckConfiguration:

[JsonDerivedType(typeof(MyCheckConfiguration), "mychecktype")]

2. Register the keyed runner in ServiceCollectionExtensions.AddDataQualityCore():

services.AddKeyedSingleton<ICheckRunner, MyCheck>(MyCheck.Key);

3. Ensure MyCheck.Key matches the discriminator string (case-insensitive match via ToLowerInvariant()).

Adding a new report format

Implement IReportWriter with a public const string Key constant and register it:

services.AddKeyedSingleton<IReportWriter, MyReportWriter>(MyReportWriter.Key);

Request it at runtime with --output myformat.

Adding a new notification channel

Implement INotificationChannel (and optionally IIncidentNotificationChannel for lifecycle events), bind its options from IOptions<MyChannelConfig>, and register it as a singleton in ServiceCollectionExtensions.

Testing

dotnet test

The test project at tests/DataQuality.Tests/ uses xUnit v3, NSubstitute for mocking, and FluentAssertions for readable assertions. Test structure mirrors the source tree: Checks/, Validation/, Orchestration/. DataQuality.Core exposes internals to the test project via InternalsVisibleTo.

Environment Variables Reference

Variable Used by Description
DOTNET_ENVIRONMENT Host Selects appSettings.{env}.json (default: Production)
SECRETSTORE_PATH --secretfile default Path to the encrypted secret store file
SECRETSTORE_PASSWORD --secretpassword default Decryption password for the secret store
ConnectionStrings__<Name> Configuration Override connection strings directly (useful in containers)

About

Configurable data quality engine for Snowflake with automated checks, reporting, notifications, and data profiling.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Contributors

Languages