Breaking. One input format goes away and three policies get stricter. If you
already send canonical metrics.* names, nothing here affects you.
Rename these
| Send this | Instead of |
|---|---|
metrics.content_safety.score |
evaluation.content_safety.score, evaluation.content_safety_score, content_safety.score, content_safety_score |
metrics.fairness.score |
evaluation.fairness.score, evaluation.fairness_score, fairness_score |
metrics.risk_management.score |
evaluation.risk_management.score, evaluation.risk_management_score, risk_management_score |
metrics.toxicity.score |
evaluation.toxicity_score, content_safety.toxicity_score |
metrics.toxicity.max_toxicity |
summary.toxicity_values.max_toxicity, content_safety.max_toxicity |
metrics.model_card.completeness |
documentation.model_card.completeness_score, documentation.model_card.completeness |
metrics.patient_safety.score |
evaluation.patient_safety.score |
metrics.clinical_validation.score |
evaluation.clinical_validation.score |
metrics.risk_assessment.score |
evaluation.risk_assessment.score |
metrics.audit_logging.completeness |
governance.audit_logging.completeness_score |
A retired name is not an error, it is silence: the metric reads as absent
and a policy that requires it denies. So a missed rename shows up as a system
that stops passing, never one that wrongly passes.
The retirement was the easy half
Carrying two spellings for the same number had been holding disagreeing halves
of the same file in apparent agreement. Removing it turned each disagreement
into a failing test.
international/eu_ai_act/v1/transparencyapproved systems nobody had
tested.allowrequirednot has_high_toxicity, and that rule is undefined
when no reading was supplied. Undefined negates to true, so a fully documented
system that had never been measured returnedallow: trueagainst Article 13.
The policy had one test, on empty input, which denies for want of
documentation long before toxicity is reached. That is how it survived.loan_evaluation/fair_lendingdenied canonical input. Nine reads never
consulted the alias table, so canonical names scored-1and failed
everything.- Three reports contradicted their own verdict, reading a legacy path with a
fallback of0while the decision read the canonical name. - Three reports deleted themselves when a metric was absent, because one
undefined value inside an object makes the whole object undefined.
Unmeasured is now visible, and never a pass
Reports say null where nobody measured. 0 made an unmeasured system
indistinguishable from one that scored zero, and on a toxicity scale reported it
as the cleanest possible.
Removed
global.v1.common.common_rules and global.v1.common.compliance, whole
packages: 16 functions, no importer anywhere. Published policy count is
unchanged at 92, because both were libraries.
Also fixed
Four examples that could not run, a FAQ teaching an input format that no longer
exists plus two opa commands that never worked, and a diagram rendering
PlantUML warnings into the picture.
Honest note on process
docs/COMPATIBILITY.md states a two-minor deprecation window. It was not
served: 1.4.0 and 1.5.0 were never released, so no consumer met the
deprecation in a shipped version. A deliberate call given the adoption level,
recorded as an exception rather than smoothed over. The table above is what you
get instead.
Full detail in CHANGELOG.md.