-
Notifications
You must be signed in to change notification settings - Fork 49
Add guardrails and update audit sources for cargo-vet workflow #803
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
jerrysxie
wants to merge
3
commits into
OpenDevicePartnership:main
Choose a base branch
from
jerrysxie:chore/cargo-vet-dedupe-guardrails
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -86,6 +86,45 @@ For every crate you review, systematically check ALL of the following: | |
|
|
||
| ## How to Review | ||
|
|
||
| ### Non-Interactive Execution | ||
|
|
||
| **CRITICAL:** All `cargo vet` commands must run non-interactively: | ||
|
|
||
| - **`diff` / `inspect`:** Always set the pager to `cat` to prevent the pager | ||
| from waiting for input. Use `$env:PAGER='cat';` (PowerShell) or `PAGER=cat` | ||
| (POSIX) before the command. | ||
| - **`certify`:** Always pass `--accept-all` along with `--criteria`, `--who`, | ||
| and `--notes` to skip all interactive prompts. | ||
|
|
||
| Never run a `cargo vet` command that could block waiting for terminal input. | ||
|
|
||
| ## Exemptions Are a Last Resort | ||
|
|
||
| Do **not** recommend adding `[[exemptions]]` entries without good reason. | ||
| Each exemption bypasses the audit process entirely and requires explicit manual | ||
| confirmation from the user. Always prefer performing a full or delta audit over | ||
| exempting a crate. If an exemption is truly necessary (e.g., the crate is only | ||
| needed at `safe-to-run` level for dev tooling, or an import source removal | ||
| requires temporary coverage), flag it clearly and let the user decide. | ||
|
|
||
| Every exemption **must** include a `notes` field explaining why the exemption | ||
| exists and under what conditions it can be removed. | ||
|
|
||
| ## Duplicate-Audit Guardrail | ||
|
|
||
| Before recommending or running certification, check whether an identical | ||
| `[[audits.<crate>]]` entry (same who/criteria/version-or-delta/notes) already | ||
| exists in `supply-chain/audits.toml`. | ||
|
|
||
| If an identical entry already exists: | ||
|
|
||
| - Do not recommend re-certifying with the same data | ||
| - Report that the crate is already certified with identical audit content | ||
| - If duplicates already exist, explicitly recommend deduplicating by keeping one | ||
| copy and removing the rest | ||
|
|
||
|
Comment on lines
+101
to
+125
|
||
| Rationale: retried `cargo vet certify` commands can append duplicate blocks. | ||
|
|
||
| ### For Delta Audits | ||
|
|
||
| Use `PAGER=cat cargo vet diff CRATE FROM TO` (POSIX) or | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -60,6 +60,29 @@ Confidence scoring rubric: | |
|
|
||
| ## Step 5: Certify | ||
|
|
||
| ### Exemptions Are a Last Resort | ||
|
|
||
| Do **not** add `[[exemptions]]` entries without explicit user confirmation. | ||
| Each exemption bypasses the audit process entirely and must be justified. | ||
| Valid reasons include: | ||
|
|
||
| - The crate is only needed for `safe-to-run` (test/dev tooling) and a full | ||
| audit is disproportionate | ||
| - An upstream import source was removed and the crate needs temporary coverage | ||
| while a first-party audit is scheduled | ||
| - The user explicitly requests an exemption after reviewing the trade-offs | ||
|
|
||
| Always prefer auditing (full or delta) over exempting. When an exemption is | ||
| unavoidable, present it to the user for manual approval before adding it. | ||
|
|
||
| Every exemption **must** include a `--notes` explaining why the exemption exists | ||
| and under what conditions it can be removed: | ||
|
|
||
| ```shell | ||
| cargo vet add-exemption CRATE VERSION --criteria CRITERIA \ | ||
| --notes "Reason for exemption; plan for resolution" | ||
| ``` | ||
|
|
||
| For each crate that passes (confidence ≥ 70), run: | ||
|
|
||
| ```shell | ||
|
|
@@ -104,9 +127,32 @@ the human reviewer, never the AI agent. | |
|
|
||
| ## Step 6: Verify and Clean Up | ||
|
|
||
| 1. Run `cargo vet` again to confirm everything passes | ||
| 2. Run `cargo vet prune` to remove stale exemptions | ||
| 3. Run `cargo vet` one final time to confirm clean state | ||
| Before final verification, detect and remove identical duplicate `[[audits.*]]` | ||
| entries that may have been appended by retried `cargo vet certify` commands. | ||
|
|
||
| Duplicate-check workflow: | ||
|
|
||
| 1. Scan `supply-chain/audits.toml` for byte-for-byte identical audit blocks | ||
| 2. If duplicates exist, keep one copy (usually the first) and remove the rest | ||
| 3. Re-run `cargo vet` after deduplication to ensure state is still valid | ||
|
|
||
| Suggested duplicate detection commands: | ||
|
|
||
| ```powershell | ||
| # PowerShell: use any local script/command that prints duplicate blocks | ||
| # with crate names and line numbers | ||
| ``` | ||
|
|
||
| ```shell | ||
| # POSIX: optional equivalent using awk/python if available | ||
| # (implementation may vary by environment) | ||
| ``` | ||
|
Comment on lines
+139
to
+149
|
||
|
|
||
| Then run the normal cleanup sequence: | ||
|
|
||
| 4. Run `cargo vet` again to confirm everything passes | ||
| 5. Run `cargo vet prune` to remove stale exemptions | ||
| 6. Run `cargo vet` one final time to confirm clean state | ||
|
|
||
| ## Reviewing Import Sources | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The PowerShell guidance
$env:PAGER='cat'may not actually prevent blocking on Windows becausecatis typically a PowerShell alias, not an external executable that a spawned process (cargo-vet) can invoke viaPAGER. Consider updating the Windows/PowerShell recommendation to a pager that is reliably available as an executable (e.g.,more.com), or explicitly note thatcatrequires a Unix-like toolchain in PATH (Git Bash/MSYS).