The defect
The corpus loader globs *.rules.json and assumes every match has one shape — a ruleset with
rules[] or principles[]. Three shipped files declare a different schema, so the loader
rejects them, prints a warning, and drops them out of every denominator the run reports.
This is the failure the project exists to stop, occurring in the project's own loader. A skipped
rule is at least counted as skipped. These are not counted at all.
Reproduction — first five lines of a first run, from a clean container
docker run --rm node:20 bash -lc '
npm install -g @beyondnet/evolith-cli@1.3.0 --silent
cd /tmp && evolith init --name my-sat --yes >/dev/null 2>&1
cd /tmp/my-sat && evolith validate --engine opa'
[Nest] WARN Skipping non-standard ruleset .../rulesets/architecture/topology-recommendation.rules.json: Schema validation failed: data must have required property 'rules', data must have required property 'principles', data must match a schema in anyOf
[Nest] WARN Skipping non-standard ruleset .../rulesets/infrastructure/helm-enforcement.rules.json: Schema validation failed: ...
[Nest] WARN Skipping non-standard ruleset .../rulesets/infrastructure/opa-sidecar-bundle.rules.json: Schema validation failed: ...
What is actually wrong with each file — two different causes
| file |
declared $schema |
top-level keys |
what it really is |
src/rulesets/infrastructure/helm-enforcement.rules.json |
../schema/rule-definition.schema.json |
id, category, title, description, severity, rationale, validation |
a single rule, never wrapped in { "rules": [ ... ] } |
src/rulesets/infrastructure/opa-sidecar-bundle.rules.json |
../schema/rule-definition.schema.json |
same |
same |
src/rulesets/architecture/topology-recommendation.rules.json |
../schema/topology-recommendation.schema.json |
id, version, progressive, dimensions |
not a ruleset at all — a different artifact type caught by the .rules.json glob |
The two INFRA files are each valid against the schema they themselves declare. They are
rule definitions, and the loader only accepts rule sets. The third is a topology
recommendation config whose only crime is its filename.
Note this means the whole of src/rulesets/infrastructure/ is discarded: those two files are
the only *.rules.json in that directory.
The part that matters: they are in no denominator
Measured against the same captured run (INFRA-001 and INFRA-OPA-001 are the two ids):
corpusTotal: 412 <- does not include them
rulesSelected: 412
Rules Total: 159 <- does not include them
Rules Skipped: 26 <- does not include them
occurrences of "INFRA-001" in 93 lines of output: 0
occurrences of "INFRA-OPA-001" in 93 lines of output: 0
topology-recommendation appears exactly once, and that occurrence is the warning itself.
So the report says 412 corpus / 159 selected as though that were the whole corpus, and two
infrastructure rules — one of them an OPA sidecar bundle policy — are invisible. A run cannot
report them as skipped, because as far as the accounting is concerned they were never there.
Suggested fix, in the order that matters
- The accounting first. A file the loader rejects must land in the report as a
first-class outcome, the same way an undecided rule does. A warning on stderr is not
accounting — it does not survive --format json, and it does not reach the exit code.
This is the load-bearing half: it makes any future instance of this visible instead of
silent.
- Then the three files. Wrap the two
INFRA rules in a ruleset envelope, and either
rename topology-recommendation.rules.json out of the glob or teach the loader to
dispatch on the declared $schema rather than on the file extension.
Doing 2 without 1 fixes three files and leaves the mechanism that hid them intact.
Proving a fix
# no file is dropped at load
docker run --rm node:20 bash -lc '... evolith validate --engine opa' 2>&1 \
| grep -c "Skipping non-standard ruleset" # must be 0
# and the two rules are now accounted for, one way or another
... evolith validate --engine opa --format json | grep -c "INFRA-OPA-001" # must be >= 1
What is not your problem
- Open Dependabot alerts and the OpenSSF Scorecard workflow are known-red and unrelated.
- The bilingual guards apply to the 17-document entry surface only; ruleset JSON is not in it,
so no .es twin is required for this change.
- If
Governance guards goes red on something you did not touch, say so in the PR — it is not
a required check.
Found while re-verifying the README's clean-room capture for #574, which is where the
[Nest] WARN lines are now acknowledged in prose rather than trimmed out of the screenshot.
The defect
The corpus loader globs
*.rules.jsonand assumes every match has one shape — a ruleset withrules[]orprinciples[]. Three shipped files declare a different schema, so the loaderrejects them, prints a warning, and drops them out of every denominator the run reports.
This is the failure the project exists to stop, occurring in the project's own loader. A skipped
rule is at least counted as skipped. These are not counted at all.
Reproduction — first five lines of a first run, from a clean container
What is actually wrong with each file — two different causes
$schemasrc/rulesets/infrastructure/helm-enforcement.rules.json../schema/rule-definition.schema.jsonid, category, title, description, severity, rationale, validation{ "rules": [ ... ] }src/rulesets/infrastructure/opa-sidecar-bundle.rules.json../schema/rule-definition.schema.jsonsrc/rulesets/architecture/topology-recommendation.rules.json../schema/topology-recommendation.schema.jsonid, version, progressive, dimensions.rules.jsonglobThe two
INFRAfiles are each valid against the schema they themselves declare. They arerule definitions, and the loader only accepts rule sets. The third is a topology
recommendation config whose only crime is its filename.
Note this means the whole of
src/rulesets/infrastructure/is discarded: those two files arethe only
*.rules.jsonin that directory.The part that matters: they are in no denominator
Measured against the same captured run (
INFRA-001andINFRA-OPA-001are the two ids):topology-recommendationappears exactly once, and that occurrence is the warning itself.So the report says 412 corpus / 159 selected as though that were the whole corpus, and two
infrastructure rules — one of them an OPA sidecar bundle policy — are invisible. A run cannot
report them as skipped, because as far as the accounting is concerned they were never there.
Suggested fix, in the order that matters
first-class outcome, the same way an undecided rule does. A warning on stderr is not
accounting — it does not survive
--format json, and it does not reach the exit code.This is the load-bearing half: it makes any future instance of this visible instead of
silent.
INFRArules in a ruleset envelope, and eitherrename
topology-recommendation.rules.jsonout of the glob or teach the loader todispatch on the declared
$schemarather than on the file extension.Doing 2 without 1 fixes three files and leaves the mechanism that hid them intact.
Proving a fix
What is not your problem
so no
.estwin is required for this change.Governance guardsgoes red on something you did not touch, say so in the PR — it is nota required check.
Found while re-verifying the README's clean-room capture for #574, which is where the
[Nest] WARNlines are now acknowledged in prose rather than trimmed out of the screenshot.