Skip to content

test: add E2E test for openSUSE Leap ecosystem mapping - #2937

Merged
another-rex merged 11 commits into
google:mainfrom
herdiyana256:feature/os-opensuse-leap-ecosystem-e2e
Aug 7, 2026
Merged

test: add E2E test for openSUSE Leap ecosystem mapping#2937
another-rex merged 11 commits into
google:mainfrom
herdiyana256:feature/os-opensuse-leap-ecosystem-e2e

Conversation

@herdiyana256

@herdiyana256 herdiyana256 commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds an end-to-end test for the openSUSE Leap ecosystem mapping fix that just merged in google/osv-scalibr#2290 (closes google/osv-scalibr#2203), following the same pattern used for the AlmaLinux (#2870) and Mageia (#2873) ecosystem E2E tests.

  • Adds a pinned opensuse/leap:15.5 image fixture (test-opensuse-leap-15.5.Dockerfile).
  • Adds Scanning_openSUSE_Leap_15.5_image / scanning_opensuse_leap_15.5_image cases to TestCommand_OCIImage and TestCommand_OCIImage_JSONFormat, with recorded cassette entries and snapshots showing real SUSE-SU advisories detected for curl and openssl-1_1.

While wiring this up I found the os/rpm extractor was never registered in the artifact preset used by image scanning (internal/scalibrplugin/presets.go), so any RPM-based image, not just openSUSE Leap, currently scans with zero package inventory ("No package sources found"). This is the same root cause already fixed for AlmaLinux in #2870 (commit f995ea5), which hasn't landed on main yet, so I included the same registration here to unblock this test. Updated the snapshots/cassettes it touches (scan/source, internal/output, internal/scalibrplugin).

go.mod points at a branch on my osv-scalibr fork carrying just the openSUSE Leap ecosystem fix cherry-picked onto the same osv-scalibr base already used by the AlmaLinux/Mageia branches. Bumping straight to the latest osv-scalibr main pulls in unrelated breaking API changes (PluginConfig refactor, Maven client signature changes, etc.) that osv-scanner hasn't absorbed yet. This is the same temporary-replace pattern used while #2250 was in flight; happy to drop it once either this or the AlmaLinux/Mageia scalibr bump lands.

Test plan

  • go build ./...: clean
  • go test ./... -short: all packages pass
  • TEST_ACCEPTANCE=true go test ./cmd/osv-scanner/scan/image/... -run 'TestCommand_OCIImage$|TestCommand_OCIImage_JSONFormat$': all pass, including the new openSUSE Leap cases
  • Verified the JSON snapshot shows real detections: curl 8.0.1-150400.5.50.1 matches SUSE-SU-2024:3926-1 and SUSE-SU-2024:4359-1; openssl-1_1 1.1.1l-150500.17.34.1 matches SUSE-SU-2024:3765-1; ecosystem openSUSE:Leap 15.5; base image opensuse/leap correctly detected.

cc @Ly-Joey: this follows up on your review request on google/osv-scalibr#2290 to add E2E coverage for the openSUSE Leap ecosystem mapping.

Adds a pinned opensuse/leap:15.5 image fixture and scan-image test
cases (plain + JSON format), following the same pattern used for the
AlmaLinux (google#2870) and Mageia (google#2873) ecosystem E2E tests.

Depends on the openSUSE Leap ecosystem mapping fix that just merged
in google/osv-scalibr#2290 (closes google/osv-scalibr#2203).
…ckages

The image scan's default "artifact" preset wired up apk, dpkg, chisel,
and homebrew, but never the rpm extractor, so RPM-based images
(AlmaLinux, RHEL, Rocky, Mageia, openSUSE Leap, ...) produced zero
inventory. This is the same root cause already fixed for AlmaLinux in
PR google#2870 (f995ea5), which hasn't landed on main yet; registering it
here so the new openSUSE Leap E2E test (and RPM-based scanning in
general) actually detects packages instead of failing with "No package
sources found".

Also points go.mod at a branch of the osv-scalibr fork carrying just
the openSUSE Leap ecosystem mapping fix (google/osv-scalibr#2290) cherry-picked
onto the same osv-scalibr version already used by the AlmaLinux/Mageia
branches, since bumping straight to the latest osv-scalibr main pulls
in unrelated breaking API changes. Same temporary-replace pattern used
while google#2250 was in flight; drop once osv-scanner's own osv-scalibr bump
catches up.

Updates snapshots/cassettes across scan/image, scan/source, output,
and scalibrplugin affected by rpm now being a registered extractor.
@herdiyana256

herdiyana256 commented Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

Hi @Ly-Joey ,
friendly bump on this one. The dependency fix (osv-scalibr#2290) is merged now, and the E2E tests here are already in place with all checks green. Let me know if there's anything else needed for review, thanks!

@codecov-commenter

codecov-commenter commented Aug 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.31%. Comparing base (d87a911) to head (e3dea47).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2937      +/-   ##
==========================================
+ Coverage   79.29%   79.31%   +0.02%     
==========================================
  Files         118      118              
  Lines        8080     8080              
==========================================
+ Hits         6407     6409       +2     
+ Misses       1314     1313       -1     
+ Partials      359      358       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

another-rex and others added 3 commits August 5, 2026 11:36
osv-scalibr's os/rpm extractor has a Windows-only stub (rpm_dummy.go,
//go:build windows) whose FileRequired() always returns false, so no
RPM packages are ever detected there. This is upstream/pre-existing
and affects any RPM-based image test on windows-latest, not something
introduced by this PR (the same failure mode hits the AlmaLinux google#2870
and Mageia google#2873 E2E PRs for the same reason).
@herdiyana256

Copy link
Copy Markdown
Contributor Author

Pushed a fix for the Windows failure (d2add0f).

Root cause: it's not stdout/stderr mixing or gRPC transport noise, it's that osv-scalibr's os/rpm extractor has a Windows-only stub build (extractor/filesystem/os/rpm/rpm_dummy.go, //go:build windows) whose FileRequired() always returns false and Extract() returns errors.New("not supported"). So on windows-latest, os/rpm never picks up any packages regardless of what's in the artifact preset, hence No package sources found. The "too_many_pings" gRPC lines in the log are just unrelated OSV.dev API transport retries, not the cause.

This isn't something this PR introduced: the AlmaLinux (#2870) and Mageia (#2873) E2E PRs fail windows-latest for the identical reason, since they hit the same os/rpm code path.

Rather than skipping OCI image tests broadly on non-Linux (which would drop coverage for the chisel/apk/homebrew image tests that already pass fine on Windows/macOS), I scoped the skip to just the openSUSE Leap RPM subtests on Windows, following the existing runtime.GOOS skip pattern already used above in this file for the Docker-only tests:

if runtime.GOOS == "windows" && strings.Contains(strings.ToLower(tt.Name), "opensuse") {
    testutility.Skip(t, "Skipping RPM-based test as os/rpm extraction is not supported on Windows")
}

Verified locally: build is clean, and both openSUSE Leap subtests pass in TestCommand_OCIImage / TestCommand_OCIImage_JSONFormat. Once os/rpm gains real Windows support upstream, this skip can be dropped (same as AlmaLinux/Mageia would need).

@another-rex

Copy link
Copy Markdown
Collaborator

Thanks, I'll merge this tomorrow after the release.

@herdiyana256

Copy link
Copy Markdown
Contributor Author

Thanks, appreciate it! Looking forward to it.

@another-rex
another-rex merged commit add393a into google:main Aug 7, 2026
21 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PRP: Route openSUSE Leap packages to correct OSV ecosystem (ID=opensuse-leap falls through to empty namespace, 12,954 advisories unreachable)

3 participants