fix(ci): release notes report the shipped binary, not every candidate - #1619
Merged
Conversation
Release notes now carry one row per product — the bytes actually published, with their VirusTotal verdict — instead of a column per candidate. Several candidates are scanned so the selector has an alternative when an opaque classifier flags one, but a reader installing cbm cares about what they receive, not about what we discarded. The rejected candidates' verdicts remain in the published evidence TSVs for anyone auditing the selection, and they stay useful to US as a development signal; in a changelog they are noise, and listing a "microsoft-ml" verdict next to a binary nobody can download invites exactly the wrong conclusion. Also removes a claim that had become FALSE: the section asserted that every scan had a minimum of 50 decisive engines. That floor was removed deliberately (it is VirusTotal fleet availability, not a property of our binary) and the last release observed 31. Published notes would have stated something untrue. The observed range is still reported, now over the shipped binaries. The disclosure of a tolerated Microsoft `!ml` is preserved and narrowed to the binaries that ship. The contract test now pins both directions: a flagged binary that SHIPS must be disclosed, and flagged candidates that were rejected must not appear at all. Its fixture gained a target that draws the tolerated verdict on all three candidates, so the disclosure branch is actually exercised — previously every flagged candidate was rejected in favour of a clean sibling, leaving that path untested. README and SECURITY.md updated: they promised notes "link every candidate result". All five release/VT contract tests pass. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release notes now carry one row per product — the bytes actually published and their VirusTotal verdict — instead of a column per candidate.
Several candidates get scanned so the selector has an alternative when an opaque classifier flags one, but a reader installing cbm cares about the bytes they receive, not the ones we discarded. Per-candidate verdicts stay in the published evidence TSVs for auditing, and they remain useful to us as a development signal — in a changelog they are noise, and printing a
microsoft-mlverdict beside a binary nobody can download invites precisely the wrong conclusion.It also removes a claim that had become false. The section asserted that every scan had a minimum of 50 decisive engines. That floor was deliberately removed — decisive-engine count is VirusTotal fleet availability, not a property of our binary — and the last release observed 31. The published notes would have stated something untrue. The observed range is still reported, now over the shipped binaries.
Disclosure of a tolerated Microsoft
!mlis preserved, narrowed to binaries that actually ship.The contract test now pins both directions: a flagged binary that ships must be disclosed, and flagged candidates that were rejected must not appear at all. Its fixture gained a target drawing the tolerated verdict on all three candidates so the disclosure branch is genuinely exercised — previously every flagged candidate lost to a clean sibling, leaving that path untested.
README and SECURITY.md updated; both promised notes "link every candidate result".
All five release/VT contract tests pass.