Repository navigation
·
3 commits
to main
since this release
Immutable
release. Only release title and notes can be modified.
Changes
Train/test leakage and CVSS 4.0 scoring in CIRCL/vulnerability-scores,
reported and fixed by Christian Bucher.
In revision 34e6a14, 58% of the labelled test rows had a train row with the
same description, mostly a CVE record and its GitHub advisory.
- Group-aware split (
vulntrain-dataset-generation): rows that share a
normalised description, an identifier or an alias are kept in the same
split. Groups are assigned by a seeded hash, so a vulnerability keeps its
split across rebuilds. Newaliasescolumn, new--test-size,
--validation-sizeand--seedoptions, and duplicate CSAF documents are
removed. - One CVSS 4.0 reading: every v4.0 score, CVE records included, is now
computed from its vector with base metrics only (CVSS-B, switch
CVSS4_BASE_ONLY). Before, advisories were scored with Exploit Maturity
applied while CVE rows kept the CNAbaseScore, so the same vulnerability
could get two different severity labels. - Validation split:
vulntrain-train-severity-classificationselects
checkpoints onvalidationwhen the dataset has one and reports test
metrics separately. compute_metricsno longer fails when an eval split lacks a severity class.- Rejected CVE records are skipped during dataset generation.
- All trainers accept
--no-pushto train locally without pushing to the
Hugging Face Hub (new for the CNVD severity, CWE and description
generation trainers);--repo-idis then optional. - New
tools/dataset/check_split_leakage.py: reports train/test overlap of a
dataset revision and can be used as a gate after each rebuild. - Technical report on these changes:
docs/vulnerability-scores-leakage-report.md
(10.5281/zenodo.23760802).
The rebuilt dataset is revision55a886aofCIRCL/vulnerability-scores.