Feat/advisory cvss support by adewergifosse-ciq · Pull Request #1 · ctrliq/repovis

adewergifosse-ciq · 2026-03-23T23:24:35Z

Add CSAF advisory support with CVSS scoring, refactor data model

Summary

This PR refactors the RepoVis codebase for robustness and correctness, then adds the ability to read CSAF advisory JSON files directly from a local directory — eliminating the need for a separate advisory-parsing script. CVSS v3 scoring data is extracted from advisories and displayed across all output formats.

Changes

Refactor & bug fixes

Introduce PackageInfo/ChangelogEntry dataclasses replacing monkey-patched DNF objects
Fix list-mutation-during-iteration bug in CVE extraction (silent data loss)
Fix unbound buildTime for edge-case CLI arguments
Escape HTML output with html.escape() to prevent XSS
Use csv stdlib module for CSV output instead of manual string building
Replace bare except with except Exception, wildcard imports with explicit imports
Add type hints, PEP 257 docstrings, and PEP 8 naming throughout
Use tempfile.mkdtemp() instead of hardcoded /tmp path
Replace print-to-stderr with logging module

`--advisory-dir` feature

New --advisory-dir and --product-codes CLI options to read CSAF advisory JSONs on the fly
New lib/advisory_read.py module: recursively scans a directory for advisory JSONs, extracts CVE IDs, fix dates, and CVSS v3 scores
Mutually exclusive with existing --cveyaml; full backward compatibility preserved

CVSS v3 scoring in output

Build a global cvss_map: Dict[str, CvssInfo] (CVE scores are CVE-global, not per-package)
HTML: colour-coded severity labels inline with each CVE (Critical/High/Medium/Low)
CSV: new CVSS Scores column (CVE-ID:score:SEVERITY format)
YAML-CVE: separate top-level cvss section; cve_fixes lists remain plain strings
Graceful degradation — when no CVSS data is available, output is identical to before

Documentation

Updated README.md with full CLI reference, refreshed examples, and new sections
New docs/advisory-dir-option.md and docs/cvss-scoring.md

Testing

Tested with CSAF advisory directories for lts-9.2 and rlc-9.2 product codes against Rocky Linux 9.2 LTS repositories in all three output formats (HTML, CSV, YAML-CVE).

- Add PackageInfo/ChangelogEntry dataclasses replacing monkey-patched dnf objects - Fix list-mutation-during-iteration bug in CVE extraction (silent data loss) - Fix unbound buildTime for edge-case CLI arguments - Replace bare except with except Exception - Add type hints, PEP 257 docstrings, and PEP 8 snake_case naming throughout - Replace wildcard imports with explicit imports - Add lib/__init__.py for proper package structure - Use tempfile.mkdtemp() instead of hardcoded /tmp path - Escape HTML output with html.escape() to prevent XSS - Use csv stdlib module for CSV output - Replace deprecated datetime.utcfromtimestamp with timezone-aware alternative - Fix timezone display (DST-aware tzname selection) - Use with-statements for all file I/O - Replace print-to-stderr with logging module - Remove dead code (unused imports, commented-out debug lines)

Add the ability to read CIQ CSAF advisory JSON files from a local directory (--advisory-dir + --product-codes), generating supplemental CVE data on the fly instead of requiring a pre-built YAML file. CVSS v3 baseScore and baseSeverity are extracted from the advisory JSON and stored in a new CvssInfo dataclass on PackageInfo.cvss_data, while cve_dict remains Dict[str, List[str]] for full backward compatibility with existing YAML files and all output formatters. New files: - lib/advisory_read.py: CSAF JSON reader with CVSS extraction - docs/advisory-dir-option.md: feature documentation Modified: - lib/models.py: add CvssInfo dataclass, cvss_data field on PackageInfo - lib/package_read.py: add cve_data param, _merge_cve_data(), _get_cvss_data(), handle mixed string/dict CVE entries - repovis.py: add --advisory-dir/--product-codes CLI args and wiring

- Remove per-package cvss_data from PackageInfo; build a global cvss_map on PackageRead via _build_global_cvss_map() - Pass cvss_map to Output; add _cvss_label() and _cvss_html_span() helpers - HTML: colour-coded severity spans next to each CVE ID - CSV: new CVSS Scores column with CVE-ID:score:SEVERITY format - YAML-CVE: separate top-level 'cvss' section (plain string CVE lists unchanged) - Add CSS severity classes to html_template.html - Replace implementation plan with docs/cvss-scoring.md - Update docs/advisory-dir-option.md with output format details

Copilot

Pull request overview

Refactors RepoVis’ internal data model and ingestion pipeline to support reading CSAF advisory JSON files directly (via --advisory-dir) and to enrich all report formats with CVSS v3 scoring when available.

Changes:

Replace monkey-patched DNF package objects with explicit dataclasses (PackageInfo, ChangelogEntry, CvssInfo) and refactor repository reading into lib/package_read.py.
Add CSAF advisory directory ingestion (lib/advisory_read.py) and wire it into the CLI with --advisory-dir / --product-codes.
Update output generators (HTML/CSV/YAML-CVE) to optionally include CVSS data and refresh docs/README accordingly.

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 13 comments.

Show a summary per file

File	Description
`repovis.py`	Refactored CLI entrypoint; adds advisory-dir options; integrates new reader/output APIs.
`lib/package_read.py`	New DNF repo reader that builds `PackageInfo` records and a global CVSS map.
`lib/models.py`	Adds dataclasses for package/changelog/CVSS data.
`lib/advisory_read.py`	New CSAF JSON scanner that produces supplemental CVE/CVSS fix data.
`lib/Output.py`	Refactored HTML/CSV/YAML-CVE writers; adds CVSS enrichment and safer HTML escaping for repo-derived content.
`lib/html/html_template.html`	Adds CSS styling for CVSS severity labels and reformats template markup.
`README.md`	Updates usage docs, CLI reference, and adds advisory-dir/CVSS sections.
`docs/advisory-dir-option.md`	New detailed documentation for `--advisory-dir` architecture and behavior.
`docs/cvss-scoring.md`	New documentation describing CVSS extraction, global map design, and output formats.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- Rename lib/Output.py → lib/output.py to fix case-sensitive filesystem import - Parse --startdate as UTC midnight instead of local time - Guard SRPM name extraction against malformed entries in advisory_read.py - Escape HTML title with html.escape() for XSS consistency - Wrap output generation in try/finally so cleanup always runs - Use logger.exception() to preserve traceback on repo read failure - Replace O(n²) dedup scan with O(1) set lookup in _build_package_list - Enforce --days/--startdate mutual exclusivity via argparse group - Document minimum Python version as ≥ 3.9 in README - Annotate dnf.module.module_base side-effect import with noqa

skip77 · 2026-04-01T16:53:06Z

Hey, finally got to play with this a bit as used for our GDC builds, and I compared output from main vs. output from this branch. Some things I noticed that we need to patch up:

I don't think --product-codes is necessary as an option. The advisory data includes exact package names/versions (especially src.rpm package name+version), and the DNF data queried does as well. It should be sufficient to match RPM name-version-release found in the DNF query to items found in the advisory JSON. If a match is found, the CVSS/CVE data applies to that package regardless of product status
Having said that, something is odd about the --product-codes option when I specify multiple items. For example, --product-codes lts-8.6 seems to work correctly, but --product-codes "lts-8.6 fips-8-compliant" gives no data. I'd expect the lts-8.6 items to show up, even in the case of an invalid/non-existent product code. This may be a moot point if we end up removing this option altogether
Missing changelog CVEs: Something about the new changes has invalidated the extraction of CVEs from RPM changelogs. Whether I specify --advisory-dir or not, CVE entries don't appear to be extracted from the changelog entries. The desire is to have them in addition to any CVE data pulled from advisories, even if they don't have CVSS scores associated, that's ok.

_get_cves_from_changelog() was applying a redundant date filter on entries already curated by _get_filtered_changelog(). This caused CVEs from the most recent changelog entry to be silently dropped when its date fell before the --days/--startdate cutoff, even though the entry was always displayed in the Changelogs column.

Copilot

Pull request overview

Copilot reviewed 11 out of 12 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

datetime.fromordinal().timestamp() and strptime().timestamp() produce naive datetimes interpreted as local time, but build_time is UTC-based. This could shift filtering by the local UTC offset and include/exclude entries incorrectly near the cutoff. Use datetime.combine(..., tzinfo=timezone.utc) for changelog dates and .replace(tzinfo=timezone.utc) for advisory fix dates so all comparisons are consistently in UTC.

adewergifosse-ciq · 2026-04-02T01:00:12Z

Hey, finally got to play with this a bit as used for our GDC builds, and I compared output from main vs. output from this branch. Some things I noticed that we need to patch up:

I don't think --product-codes is necessary as an option. The advisory data includes exact package names/versions (especially src.rpm package name+version), and the DNF data queried does as well. It should be sufficient to match RPM name-version-release found in the DNF query to items found in the advisory JSON. If a match is found, the CVSS/CVE data applies to that package regardless of product status

Having said that, something is odd about the --product-codes option when I specify multiple items. For example, --product-codes lts-8.6 seems to work correctly, but --product-codes "lts-8.6 fips-8-compliant" gives no data. I'd expect the lts-8.6 items to show up, even in the case of an invalid/non-existent product code. This may be a moot point if we end up removing this option altogether

Missing changelog CVEs: Something about the new changes has invalidated the extraction of CVEs from RPM changelogs. Whether I specify --advisory-dir or not, CVE entries don't appear to be extracted from the changelog entries. The desire is to have them in addition to any CVE data pulled from advisories, even if they don't have CVSS scores associated, that's ok.

@skip77

The product codes are required to filter the relevant CVEs
The missing CVEs were due to the updated script only extracting CVEs from the changelog if their associated date was in the required timeframe. This was fixed in 71b40c4. All included changelogs are read for CVE extraction purposes

skip77 · 2026-04-02T03:37:06Z

Looks great, tests well, and thanks for setting me straight on the package <-> product id relationship.

Merging

adewergifosse-ciq added 3 commits March 24, 2026 07:52

adewergifosse-ciq requested review from Copilot and skip77 March 23, 2026 23:24

Copilot started reviewing on behalf of adewergifosse-ciq March 23, 2026 23:25 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings April 2, 2026 00:49

Copilot started reviewing on behalf of adewergifosse-ciq April 2, 2026 00:50 View session

Copilot AI reviewed Apr 2, 2026

View reviewed changes

Comment thread lib/output.py

Comment thread lib/package_read.py

Comment thread lib/package_read.py Outdated

skip77 approved these changes Apr 2, 2026

View reviewed changes

skip77 merged commit 7a9771b into main Apr 2, 2026

skip77 deleted the feat/advisory-cvss-support branch April 2, 2026 03:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat/advisory cvss support#1

Feat/advisory cvss support#1
skip77 merged 6 commits intomainfrom
feat/advisory-cvss-support

adewergifosse-ciq commented Mar 23, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

skip77 commented Apr 1, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

adewergifosse-ciq commented Apr 2, 2026

Uh oh!

skip77 commented Apr 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

Conversation

adewergifosse-ciq commented Mar 23, 2026

Add CSAF advisory support with CVSS scoring, refactor data model

Summary

Changes

Refactor & bug fixes

--advisory-dir feature

CVSS v3 scoring in output

Documentation

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

skip77 commented Apr 1, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

adewergifosse-ciq commented Apr 2, 2026

Uh oh!

skip77 commented Apr 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

`--advisory-dir` feature