CVE Validation Library - metrics

[ccoffin]

Purpose:

This will be one of multiple discussions to query the QWG and others on how best to use the upcoming CVE Services Validation Library for performing additional or more accurate CVE Record Format JSON schema validation during record ingest. The Validation Library will allow the CVE Program to perform programmatic validations beyond what the current JSON schema validation can provide. A simple example might be where we want to validate that a provided CWE ID actually exists by referencing the current CWE List. This is not possible using JSON schema validation but would be possible with the Validation Library.

Discussion:

The metrics array is intended to capture a collection of impact scores, and many CNAs use it to provide CVSS. The metrics array is currently 60% of the CVE Record Format schema when counting by number of lines. The metrics array allows CVSS 2.0, 3.0, 3.1, and 4.0 scores and related attributes. The metrics array also includes an “other” object where non-standard impacts can be defined using additional JSON properties not defined with the CVE Record Format schema. The CVE Record Format schema defines each version of CVSS using a separate object and much of the data and attributes are duplicated across the different CVSS versions because of this.

The CVE Record Format schema also includes base score and base severity comparison logic to ensure that the provided score and severity match up (e.g., a score of 1.0 with a severity of High would be flagged). However, a more important comparison that is not present is to make sure that the provided vector string matches the score. Live examples of mismatches can be found at https://vulnogram.org/cve-index/cvss#cvss3. A previous writeup on the comparison logic can be found at https://github.com/ccoffin/cve-schema/blob/develop/rfds/000x-metrics-remove-cvss-score-comparisons.md.

CVSS 3.0, 3.1, and 4.0 require the record author to provide the CVSS version, the vector string, the base score, and the base severity, while CVSS 2.0 requires all but the base severity. A record author can optionally include one or more of the individual CVSS metrics using their property names.

Entering a CVSS 3.1 vector string that is missing one or more of the base metrics passes schema validation. CVSS 4.0 vector strings fail when one or more base metrics are missing. The pattern match for the CVSS 3.1 vector string should probably be updated so that it enforces at least the base metrics.

Schema Location(s):

https://github.com/CVEProject/cve-schema/blob/main/schema/docs/CVE_Record_Format_bundled.json#L1075

Question(s):

• Can we use the Validation Library to improve the quality and accuracy of the data provided by CNAs within the metrics array?
• Can the proposed validation be implemented using only the Validation library, or could it also be accomplished using the JSON schema?
• Should the rather large CVSS vector string pattern matches be moved out of the JSON schema and into the Validation library?
• Should the comparison logic for base score and base severity be moved out of the JSON schema and into the validation library? This would be an easy win for removing CVE Record Format schema complexity. It also makes sense to use the validation library for this kind logic as opposed to the schema.
• Do the different CVSS versions need their own objects within the schema, or could they all be combined within the schema? The CVSS version could then define what metric properties are applicable from the shared set of metric properties.
• Should the optional metrics properties only be defined by CVE Services on ingest? In other words, disallow the record author from adding the individual CVSS metrics and just generate them based on the provided CVSS vector string. This all assumes that consumers find value in them and want them to still be available. If not, maybe we should consider removing the individual CVSS metrics properties altogether.
• Should the base score and base severity values be generated based on the provided vector string? In other words, should we only require and only allow the record author to provide the CVSS vector string, then have CVE Services calculate the base score and base severity using the provided vector string (vector string also includes the CVSS version)? This would essentially eliminate CVSS score and severity mismatches mentioned in the discussion.
• Should we consider disallowing new submissions that include CVSS 2.0 or 3.0 since they are somewhat outdated at this point? We should probably keep the current records as is that include these older scores, but may want to refuse new submissions using them going forward. This could be done with the validation library or could be handled by CVE Services directly.

Examples:

• Fail (or Warn) when the provided base score does not match the provided base severity. (Schema and Validation Library)(currently implemented in the schema)
• Fail (or Warn) when the provided vector string does not match the provided base score. There are live examples of this today, see discussion. (Validation Library Only)
• Fail (or Warn) when the provided vector string does not match the optional individual metric properties (Validation Library Only)
• Fail (or Warn) when optional individual metric properties are provided with a value but are not included in the provided vector string, e.g., providing the CVSS 3.1 exploitCodeMaturity property and value but not also including it in the provided vector string. (Validation Library Only)
• Fail (or Warn) when the provided vector string does not include at least the CVSS base metrics. (Validation Library Only)
• Fail (or Warn) when an outdated CVSS 2.0 or 3.0 object is provided. (Validation Library Only)

[darakian]

Fail (or Warn) when the provided vector string does not match the provided base score. There are live examples of this today, see discussion. (Validation Library Only)

We should drop the score from the schema completely. The vector fully defines the score.

[sei-vsarvepalli]

I would recommend keeping both the CVSS vector and the CVSS score (if CVSS metrics is being submitted). The rationale is that, particularly for security data, we want to preserve both what was evaluated and what conclusion was reached. The CVSS vector represents the inputs and methodology used to perform the assessment, while the CVSS score represents the resulting conclusion. Capturing both allows us to verify that the conclusion is consistent with the underlying assessment data and provides a useful audit trail.

This is also consistent with the data architecture principle of denormalization: when a value is derived from underlying data as part of a decision or assessment, it can be valuable to persist both the source data and the resulting decision rather than requiring the decision to be recomputed and validated at runtime (perhaps repeatedly calculated).

For security data, I would therefore vote for collecting both values and treating the vector as the authoritative assessment inputs and the score as the recorded outcome. We could also validate the two at ingestion time and reject the submission if the supplied score does not match the score calculated from the vector. This gives us the benefits of both traceability and efficient consumption, while also making inconsistencies explicit rather than silently accepting potentially incorrect security data.

[chandanbn]

+1 to Vijay - The CVSS details capture key characteristics of a vulnerability and are important to vulnerability identification.
The vector is not for humans - any letter flip can considerably change the score and is hard to spot.
The score is acting as a checksum. if they don't match, throw an error, don't overwrite it with a new score.

[darakian]

The rationale is that, particularly for security data, we want to preserve both what was evaluated and what conclusion was reached.

I do not accept that rational. The vector IS the conclusion and the score is simply another representation. Requiting both only allows for deviation and thus interpretation conflicts by end users. I do accept that viewing a number rather than a vector can be useful but the calculation can be done server side upon submission.

This is also consistent with the data architecture principle of denormalization: when a value is derived from underlying data as part of a decision or assessment, it can be valuable to persist both the source data and the resulting decision rather than requiring the decision to be recomputed and validated at runtime (perhaps repeatedly calculated).

This can be addressed by running the calculation in cve services. I'm also willing to say that the amount of calculation needed to render a score from a cvss vector is a trivial amount of compute in this the year 2026. I am also willing measure this if you do not accept that claim.

We could also validate the two at ingestion time and reject the submission if the supplied score does not match the score calculated from the vector. This gives us the benefits of both traceability and efficient consumption, while also making inconsistencies explicit rather than silently accepting potentially incorrect security data.

As a short term approach; sure. With a longer term view in mind; we do not need the complexity that the score demands and we would be better off without it.