vulns/osv_export: merge into existing records instead of overwriting#23112
Merged
Conversation
Records now carry `published` and `database_specific: {source:
"generated"}`. `.run` reads any existing `<id>.json`, preserves its
`published` timestamp and `affected[].ranges` (so the `fixed` boundary
reflects when the annotation was first observed rather than drifting to
the current pkg_version), and only writes when the remaining content has
changed. Records for annotations no longer in core are simply not
visited, so they persist on the default branch.
A transient OSV.dev failure leaves an existing enriched record untouched
rather than stripping its summary/severity/references. The unchanged
comparison uses parsed hashes so key ordering in a hand-formatted
existing file does not cause spurious rewrites.
andrew
added a commit
to Homebrew/advisory-database
that referenced
this pull request
Jul 14, 2026
brew generate-vulns-advisories preserves published/ranges from existing records and never deletes (Homebrew/brew#23112), so wiping first would throw away the persisted fixed boundaries and any curated records.
This was referenced Jul 14, 2026
andrew
added a commit
to Homebrew/advisory-database
that referenced
this pull request
Jul 14, 2026
The workflow refreshes upstream-derived fields but keeps published and affected[].ranges from disk (Homebrew/brew#23112), so a hand-corrected fixed boundary survives. Drop the blanket "do not edit by hand".
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates Homebrew’s OSV advisory exporter (Homebrew::Vulns::OsvExport) so it merges generated advisories into an existing target directory rather than overwriting, enabling the advisory database to accumulate records over time while preserving key historical fields.
Changes:
- Preserve existing on-disk
published(when present) andaffected[].rangeswhile regenerating records, and skip rewrites when the record is otherwise unchanged. - Avoid stripping previously-enriched fields when the upstream OSV API fetch fails (leave existing enriched records untouched).
- Add
publishedplusdatabase_specific: { source: "generated" }to generated records, and extend specs to cover merge/no-rewrite behavior and upstream outage behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| Library/Homebrew/vulns/osv_export.rb | Implement merge-in-place behavior, upstream-outage handling, and add published/database_specific fields. |
| Library/Homebrew/test/vulns/osv_export_spec.rb | Add/adjust tests for the new merge semantics, non-rewrite comparisons, and upstream failure scenarios. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
3 tasks
When migrating a record written before `published` existed, fall back to its `modified` timestamp rather than today's so first-rewrite doesn't jump the publication date forward. Narrow the upstream cache value with `is_a?(Hash)` at the call site instead of `T.cast`, and collapse the failed-fetch guard to a single `next if` now that reassignment isn't needed.
Bo98
approved these changes
Jul 14, 2026
This was referenced Jul 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #23106 so
Homebrew/homebrew-advisory-databasecan be an accumulating source of truth rather than a daily snapshot (per the outcome of Homebrew/homebrew-brew-vulns#111).OsvExport.runnow merges into the target directory instead of blindly writing. For each generated record, if an<id>.jsonalready exists itspublishedtimestamp andaffected[].rangesare carried forward, so thefixedboundary reflects when the annotation was first observed rather than drifting to today'spkg_versionon every run; the write is skipped entirely when nothing else has changed. Records for annotations no longer inhomebrew/coreare simply not visited, so they persist on the default branch. Curated records (which use a different id shape) are never touched.Because
rangesis preserved from disk, a hand-correctedfixedboundary (e.g. backdatinglibquicktimeto_4) survives subsequent runs. A transient OSV.dev failure leaves an existing enriched record untouched rather than stripping itssummary/severity/references; a new record with no existing file is still written without enrichment. The unchanged comparison uses parsed hashes so key ordering in a hand-formatted file does not cause spurious rewrites.Records also gain
publishedanddatabase_specific: {source: "generated"}per the draftCONTRIBUTING.mdin Homebrew/advisory-database#3.regenerate.ymlin that repo drops itsrm -f advisories/*.jsonin Homebrew/advisory-database#2.brewcommands to reproduce the bug?brew lgtm(style, typechecking and tests) locally?Written with Claude Code following the discussion on Homebrew/homebrew-brew-vulns#111 and Homebrew/discussions#6869, with a codex review pass that caught the transient-outage stripping and key-order comparison issues. I read the diff and ran
brew lgtmlocally.