Skip to content

feat: add Linux official evidence drift audit#139

Merged
SSobol77 merged 1 commit into
mainfrom
feature/f4-linux-official-evidence-drift-audit
Jul 10, 2026
Merged

feat: add Linux official evidence drift audit#139
SSobol77 merged 1 commit into
mainfrom
feature/f4-linux-official-evidence-drift-audit

Conversation

@SSobol77

@SSobol77 SSobol77 commented Jul 10, 2026

Copy link
Copy Markdown
Owner

Summary

Add deterministic Linux official evidence drift audit helpers.

This PR is audit-only. It does not add new official evidence records, does not promote new mappings, and does not change generated evidence semantics.

What changed

Added helper APIs in scripts/f4_linter_linux_provisioning.py:

  • linux_official_distro_evidence_matrix()
  • linux_official_distro_evidence_summary()
  • linux_official_distro_evidence_drift_errors()

The helpers make the configured official evidence registry visible as a deterministic contract surface and compare official override records against generated distro evidence.

Official evidence matrix

linux_official_distro_evidence_matrix() returns deterministic rows for the current official override registry.

Each row includes:

  • artifact ID
  • tool ID
  • expected evidence record ID
  • evidence source
  • evidence source type
  • evidence status
  • official source name
  • official source URL
  • official source kind
  • verification scope
  • release blocking state
  • external verification flag

Current matrix contains exactly six rows:

deb/yamllint
deb/shellcheck
deb/clang-tidy
deb/cppcheck
deb/clang-format
deb/checkstyle

Official evidence summary

linux_official_distro_evidence_summary() reports deterministic counts for the official override registry.

Current expected state:

official_override_count == 6
artifact_counts == {"deb": 6}
evidence_status_counts == {"verified-official-source": 6}
evidence_source_type_counts == {"official-distro-metadata": 6}
official_source_kind_counts == {"distro-package-index": 6}
verification_scope_counts == {"package-name-and-executable": 6}
release_blocking_count == 0
non_debian_override_count == 0

Drift audit

linux_official_distro_evidence_drift_errors() compares official override rows against generated distro evidence records.

Current expected result:

linux_official_distro_evidence_drift_errors() == []

The drift helper detects contract mismatches including:

  • unknown artifact/tool policy
  • official override not reflected in generated evidence
  • evidence record ID mismatch
  • evidence source type mismatch
  • evidence status mismatch
  • official source kind mismatch
  • verification scope mismatch
  • official source URL mismatch
  • release blocking mismatch
  • external verification flag mismatch

This is a contract drift audit between the configured official evidence registry and generated ECLI evidence. It is not a live network audit of Debian web pages.

Behavior preserved

Exactly six generated Debian evidence records remain verified-official-source:

  • deb/yamllint
  • deb/shellcheck
  • deb/clang-tidy
  • deb/cppcheck
  • deb/clang-format
  • deb/checkstyle

No Docker helper record is promoted.

No non-Debian mapping is promoted.

All Docker DEB helper records remain repository-local-policy/current-policy-baseline.

RPM/openSUSE/Arch/Slackware mappings remain unchanged.

Tests

Added tests for:

  • exact official evidence matrix rows
  • exact official evidence summary counts
  • empty drift error state
  • targeted non-mutating URL mismatch detection using a copied generated evidence record

Existing promotion, Docker baseline, generated manifest, and negative tamper tests remain passing.

Scope controls

This PR does not:

  • add official evidence records
  • promote additional mappings
  • demote existing mappings
  • change package names
  • change executable names
  • change URLs
  • change OS_PACKAGE_NAMES
  • change versions
  • add checksums
  • run package managers
  • download anything
  • perform network operations
  • vendor binaries or archives
  • add generated JSON artifacts
  • weaken verifier logic
  • change promotion-gate policy
  • touch UI/provider/parser/runtime/TextMate/theme code

Validation

Passed locally / by agent report:

uv run ruff check src tests scripts
uv run ruff format --check src tests scripts
uv run python scripts/check_runtime_imports.py
uv run pytest -q tests/packaging/test_f4_linter_linux_provisioning.py
uv run pytest -q tests/packaging
uv run pytest -q tests/extensions/linters
uv run pytest -q tests/docs
git diff --check

Observed results:

tests/packaging/test_f4_linter_linux_provisioning.py: 121 passed
tests/packaging: 494 passed
tests/extensions/linters: 226 passed
tests/docs: 18 passed

Safety:

No new package names.
No OS_PACKAGE_NAMES changes.
No executable-name changes.
No URL changes.
No package versions as contract fields.
No checksums.
No package-manager calls.
No downloads.
No network operations.
No generated artifacts.
No vendored binaries.
No UI/provider/parser/runtime/TextMate/theme changes.

Next work

A follow-up PR can add a release-facing report/check entry point for this audit helper, still without promoting additional distro families.

Summary by CodeRabbit

  • New Features

    • Added deterministic reporting for official Linux distribution evidence overrides.
    • Added summaries covering evidence status, source type, verification scope, and release-blocking totals.
    • Added drift detection to identify mismatches between configured expectations and generated evidence.
  • Tests

    • Added coverage for Debian evidence reporting, summary counts, and drift validation.
    • Added checks that detect mismatched evidence source URLs.

@coderabbitai

coderabbitai Bot commented Jul 10, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 61b0f89d-feb3-4781-bb3f-39cea3701eb6

📥 Commits

Reviewing files that changed from the base of the PR and between ec7f8e4 and baa39fd.

📒 Files selected for processing (2)
  • scripts/f4_linter_linux_provisioning.py
  • tests/packaging/test_f4_linter_linux_provisioning.py

📝 Walkthrough

Walkthrough

Adds deterministic official distro evidence matrices and summaries, plus drift detection that compares generated Linux distro evidence records with configured override metadata. Tests cover Debian rows, aggregate counts, clean drift checks, and URL mismatches.

Changes

Official distro evidence validation

Layer / File(s) Summary
Evidence matrix and summary
scripts/f4_linter_linux_provisioning.py, tests/packaging/test_f4_linter_linux_provisioning.py
Builds deterministic override rows with expected record IDs and aggregates artifact, status, source, scope, and release-blocking counts.
Generated evidence drift validation
scripts/f4_linter_linux_provisioning.py, tests/packaging/test_f4_linter_linux_provisioning.py
Generates canonical evidence records for overrides, compares selected fields against expected metadata, and tests clean and mismatched results.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant DriftCheck as linux_official_distro_evidence_drift_errors()
  participant Resolver as _generated_official_distro_evidence_record()
  participant Provisioning as Linux provisioning evidence generation
  participant Comparator as _official_distro_evidence_record_drift_errors()
  DriftCheck->>Resolver: resolve override row to policy
  Resolver->>Provisioning: generate canonical distro evidence record
  Provisioning-->>Resolver: generated evidence record
  Resolver-->>DriftCheck: record or resolution error
  DriftCheck->>Comparator: compare generated record with expected row
  Comparator-->>DriftCheck: field mismatch errors
Loading

Possibly related PRs

  • SSobol77/ecli#130: Introduced the related Linux distro evidence record and summary contract.
  • SSobol77/ecli#131: Extends shared official-source metadata and evidence validation logic.
  • SSobol77/ecli#133: Extends Debian official distro evidence overrides and associated tests.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding Linux official evidence drift audit helpers and tests.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/f4-linux-official-evidence-drift-audit

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sonarqubecloud

Copy link
Copy Markdown

@SSobol77 SSobol77 merged commit cade8bb into main Jul 10, 2026
5 checks passed
@SSobol77 SSobol77 deleted the feature/f4-linux-official-evidence-drift-audit branch July 10, 2026 11:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant