Skip to content

[Tests] DatabaseValidatorTests — Environment-dependent Assert.Fail masquerading as unit test #747

@Christophe-Rogiers

Description

@Christophe-Rogiers

Severity: Info
File: tests/Servy.Infrastructure.UnitTests/Helpers/DatabaseValidatorTests.cs lines 15–25

Description:
IsSqliteVersionSafe_CurrentEnvironment_ShouldMatchExpectation is written as a regular [Fact] but its outcome depends entirely on the SQLite version shipped in the binaries of the machine running the test:

if (isSafe)
{
    Assert.True(Version.TryParse(detectedVersion, out var version));
    Assert.True(version >= AppConfig.MinRequiredSqliteVersion);
}
else
{
    // If it fails, we want to know what version caused the failure in the test logs
    Assert.Fail($"Environment is unsafe. Detected: {detectedVersion}. Required: {AppConfig.MinRequiredSqliteVersion}");
}

Two problems:

  1. A developer on an older Windows box (or a CI worker with a bundled older System.Data.SQLite transitive dep) gets a red test for an environment issue, not a code issue. The Assert.Fail message is helpful but the failure mode conflates "code broken" with "box misconfigured".
  2. The companion [Theory] SQLiteVersionComparison_LogicCheck (line 30+) already covers the pure logic (threshold comparison, invalid strings). The environment check is not earning its keep as a unit test — it belongs in an integration suite.

Suggested fix:
Convert the environment-check case to one of:

  • A [Trait("Category", "Integration")] test so the CI run can filter it in/out explicitly.
  • An [SkippableFact] using Xunit.SkippableFact that Skip.IfNot(isSafe, ...) and asserts the shape of detectedVersion rather than the comparison result.
  • A [Fact] that only asserts Version.TryParse(detectedVersion, out _) — the detector works even if the installed version is behind the floor — and move the "is the installed version ≥ floor?" assertion to a deployment-gate check outside the unit test suite.

Metadata

Metadata

Assignees

Labels

testsChanges to test code and coverage.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions