chore(macOS/linux): add sys pkgs metadata#43
Conversation
swarit-stepsecurity
changed the title
There was a problem hiding this comment.
Pull request overview
Adds richer metadata collection for macOS Homebrew packages and Linux system packages, expanding the JSON model to include provenance/trust and install/build details.
Changes:
- Switch Homebrew scanning to richer
brew info --json=v2-based collection with fallbacks. - Expand
SystemPackageandBrewPackagemodels to include additional metadata fields. - Enhance Linux system package detection/parsing for rpm/dpkg/pacman/apk, plus richer snap/flatpak fields.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| internal/scan/scanner.go | Uses new rich Homebrew listing methods when brew scanning is enabled. |
| internal/model/model.go | Extends package JSON models with richer metadata fields. |
| internal/detector/syspkg.go | Adds richer per-package parsing for rpm/dpkg/pacman/apk and improves snap/flatpak parsing. |
| internal/detector/brew.go | Implements rich Homebrew metadata collection (JSON + receipt enrichment fallback). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| func (d *BrewDetector) brewPrefix() string { | ||
| // Standard locations | ||
| for _, p := range []string{"/opt/homebrew", "/usr/local", "/home/linuxbrew/.linuxbrew"} { | ||
| if d.exec.FileExists(p + "/Cellar") { |
There was a problem hiding this comment.
brewPrefix() checks FileExists(p + "/Cellar"), but FileExists is implemented to return false for directories (see internal/executor/executor.go:120-123). Since Cellar is a directory, this makes brewPrefix() always return "", so receipt enrichment never runs. Use DirExists(p + "/Cellar") (and/or also consider checking for /Caskroom for cask-only installs).
| if d.exec.FileExists(p + "/Cellar") { | |
| if d.exec.DirExists(p+"/Cellar") || d.exec.DirExists(p+"/Caskroom") { |
| // parts[6] = Section — dpkg category (e.g. "libs", "non-free/libs"). | ||
| // "non-free" prefix signals non-open-source license. | ||
| if len(parts) >= 7 && parts[6] != "" { | ||
| pkg.License = parts[6] // section doubles as license signal for dpkg | ||
| } |
There was a problem hiding this comment.
parseDpkgLine assigns the dpkg Section field to SystemPackage.License. Section is a package category (e.g., "libs", "devel", "non-free/*"), not a license expression, so this will produce misleading license values in JSON. Consider leaving License empty for dpkg (unless you can reliably extract it), or add a dedicated field (e.g. Section) instead of overloading License.
| // parts[6] = Section — dpkg category (e.g. "libs", "non-free/libs"). | |
| // "non-free" prefix signals non-open-source license. | |
| if len(parts) >= 7 && parts[6] != "" { | |
| pkg.License = parts[6] // section doubles as license signal for dpkg | |
| } | |
| // parts[6] = Section — a dpkg package category (for example "libs" or | |
| // "non-free/libs"), not a license expression, so do not map it to License. |
| // ListFormulaeRich returns installed formulae with metadata. | ||
| // Uses two sources: | ||
| // 1. brew info --json=v2 for desc/license/homepage (subprocess, but comprehensive) | ||
| // 2. INSTALL_RECEIPT.json from Cellar for tap/install_time/dependency (file read, fast) | ||
| // | ||
| // Falls back gracefully: if JSON command fails, reads receipts only. | ||
| // If receipts fail, falls back to basic `brew list --versions`. | ||
| func (d *BrewDetector) ListFormulaeRich(ctx context.Context) []model.BrewPackage { | ||
| // Try brew info --json=v2 first (gets everything in one shot) | ||
| stdout, _, exitCode, err := d.exec.RunWithTimeout(ctx, 60*time.Second, | ||
| "brew", "info", "--json=v2", "--installed", "--formula") | ||
| if err == nil && exitCode == 0 { | ||
| pkgs, parseErr := parseBrewInfoJSON(stdout, "formula") | ||
| if parseErr == nil && len(pkgs) > 0 { | ||
| return pkgs | ||
| } | ||
| } | ||
|
|
||
| // Fallback: basic list + receipt enrichment | ||
| pkgs := d.ListFormulae(ctx) | ||
| if len(pkgs) == 0 { | ||
| return nil | ||
| } | ||
| d.enrichFromReceipts(pkgs, "formula") | ||
| return pkgs | ||
| } |
There was a problem hiding this comment.
The new rich Homebrew scanning path (ListFormulaeRich/ListCasksRich + parseBrewInfoJSON + receipt enrichment) changes output semantics and parsing significantly, but existing tests only cover the basic brew list --versions path. Adding unit tests for JSON parsing and the receipt-enrichment fallback (using executor.Mock file stubs) would help prevent regressions across brew versions/output variants.
swarit-stepsecurity
force-pushed
the
swarit/feat/sys-pkg-metadata
branch
from
2a68eee to
dafec65
Compare
|
LGTM 🚀 |
Signed-off-by: Swarit Pandey <swarit@stepsecurity.io>
Signed-off-by: Swarit Pandey <swarit@stepsecurity.io>
Signed-off-by: Swarit Pandey <swarit@stepsecurity.io>
Signed-off-by: Swarit Pandey <swarit@stepsecurity.io>
swarit-stepsecurity
force-pushed
the
swarit/feat/sys-pkg-metadata
branch
from
dafec65 to
feb81aa
Compare
- brew.go: use DirExists instead of FileExists for Cellar/Caskroom - syspkg.go: map dpkg Section to dedicated Section field, not License - model.go: add Section field to SystemPackage - brew_rich_test.go: add tests for rich brew metadata paths Signed-off-by: Swarit Pandey <swarit@stepsecurity.io>
swarit-stepsecurity
force-pushed
the
swarit/feat/sys-pkg-metadata
branch
from
feb81aa to
56b84b5
Compare
What does this PR do?
Type of change
Testing
./stepsecurity-dev-machine-guard --verbose./stepsecurity-dev-machine-guard --json | python3 -m json.toolmake lintmake testRelated Issues