Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(mariner): use advisory_id for definition file names #271

Merged
merged 9 commits into from
May 15, 2024
Merged

fix(mariner): use advisory_id for definition file names #271

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
2be1bc7
fix(mariner): save defs as array
DmitriyLewen Feb 13, 2024
b94e1d3
test: update tests
DmitriyLewen Feb 13, 2024
af63f0f
revert: save Definition as struct
DmitriyLewen May 15, 2024
1b7c548
refactor: use `advisory_id` as struct
DmitriyLewen May 15, 2024
092ba40
tests: update golden files
DmitriyLewen May 15, 2024
b76577c
fix: update repoURL
DmitriyLewen May 15, 2024
995ea97
refactor: move advisoryID assembly to separate function
DmitriyLewen May 15, 2024
7074604
test: add tests for `AdvisoryID`
DmitriyLewen May 15, 2024
089504c
refactor: mix comment mistake
DmitriyLewen May 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
26 changes: 20 additions & 6 deletions mariner/mariner.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ import (
)

const (
repoURL = "https://github.com/microsoft/CBL-MarinerVulnerabilityData/archive/refs/heads/main.tar.gz//CBL-MarinerVulnerabilityData-main"
repoURL = "https://github.com/microsoft/AzureLinuxVulnerabilityData/archive/refs/heads/main.tar.gz//AzureLinuxVulnerabilityData-main"
knqyf263 marked this conversation as resolved.
Show resolved Hide resolved
cblDir = "mariner" // CBL-Mariner Vulnerability Data
retry = 3

Expand Down Expand Up @@ -142,9 +142,7 @@ func (c Config) update(version, path string) error {
// write definitions
bar := pb.StartNew(len(oval.Definitions.Definition))
for _, def := range oval.Definitions.Definition {
vulnID := def.Metadata.Reference.RefID

if err := c.saveAdvisoryPerYear(filepath.Join(dirPath, definitionsDir), vulnID, def); err != nil {
if err := c.saveAdvisoryPerYear(filepath.Join(dirPath, definitionsDir), def); err != nil {
return xerrors.Errorf("failed to save advisory per year: %w", err)
}

Expand All @@ -154,8 +152,24 @@ func (c Config) update(version, path string) error {

return nil
}
func (c Config) saveAdvisoryPerYear(dirName string, def Definition) error {
// Mariner uses `<ID>_<last_number_from_version>` format for `advisory_id`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK, they use hyphens.

Suggested change
// Mariner uses `<ID>_<last_number_from_version>` format for `advisory_id`.
// Mariner uses `<ID>-<last_number_from_version>` format for `advisory_id`.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should start calling Azure Linux.

Suggested change
// Mariner uses `<ID>_<last_number_from_version>` format for `advisory_id`.
// Azure Linux uses `<ID>_<last_number_from_version>` format for `advisory_id`.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in 995ea97

// But `advisory_id` is not required field.
// Therefore, if `advisory_id` is not exist, we create this field independently.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit

Suggested change
// Therefore, if `advisory_id` is not exist, we create this field independently.
// Therefore, if `advisory_id` does not exist, we create this field independently.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated in 995ea97

// cf. https://github.com/aquasecurity/vuln-list-update/pull/271#issuecomment-2111678641
advisoryID := def.Metadata.AdvisoryID
if advisoryID == "" {
advisoryID = def.ID
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ID is something like oval:com.microsoft.cbl-mariner:obj:31880001. Shouldn't we extract the last digits, 31880001?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch!
Changed in 995ea97
Also added tests - 7074604

// for `0` versions `_0` suffix is omitted.
if def.Version != "" && def.Version[len(def.Version)-1:] != "0" {
advisoryID = fmt.Sprintf("%s_%s", advisoryID, def.Version[len(def.Version)-1:])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
advisoryID = fmt.Sprintf("%s_%s", advisoryID, def.Version[len(def.Version)-1:])
advisoryID = fmt.Sprintf("%s-%s", advisoryID, def.Version[len(def.Version)-1:])

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in 995ea97

}
}
// Use advisory_id for file name to avoid overwriting files when there are 2 definitions for same CVE
// cf. https://github.com/aquasecurity/trivy-db/issues/379
fileName := fmt.Sprintf("%s.json", advisoryID)

func (c Config) saveAdvisoryPerYear(dirName string, vulnID string, def Definition) error {
vulnID := def.Metadata.Reference.RefID
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about using advisory_date field.
But this field does not always exist:

➜ cat cbl-mariner-1.0-oval.xml| grep ' <definition class="vulnerability"' | toilet -l
      2252
➜ cat cbl-mariner-1.0-oval.xml| grep '<advisory_date>' | toilet -l
      2070

So I'm leaving logic with year number from CVE.

if !strings.HasPrefix(vulnID, "CVE") {
log.Printf("discovered non-CVE-ID: %s", vulnID)
return ErrNonCVEID
Expand All @@ -168,7 +182,7 @@ func (c Config) saveAdvisoryPerYear(dirName string, vulnID string, def Definitio
}

yearDir := filepath.Join(dirName, s[1])
if err := utils.Write(filepath.Join(yearDir, fmt.Sprintf("%s.json", vulnID)), def); err != nil {
if err := utils.Write(filepath.Join(yearDir, fileName), def); err != nil {
return xerrors.Errorf("unable to write a JSON file: %w", err)
}
return nil
Expand Down
0 ...r/1.0/definitions/2008/CVE-2008-3914.json → ...en/mariner/1.0/definitions/2008/3173.json
View file
Open in desktop
File renamed without changes.
0 .../1.0/definitions/2018/CVE-2018-25012.json → ...en/mariner/1.0/definitions/2018/4209.json
View file
Open in desktop
File renamed without changes.
0 .../1.0/definitions/2021/CVE-2021-35942.json → ...en/mariner/1.0/definitions/2021/4820.json
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...r/2.0/definitions/2014/CVE-2014-8139.json → ...en/mariner/2.0/definitions/2014/6933.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,4 @@
"TestRef": "oval:com.microsoft.cbl-mariner:tst:1643374850000269"
}
}
}
}
0 .../2.0/definitions/2021/CVE-2021-39924.json → ...en/mariner/2.0/definitions/2021/7412.json
View file
Open in desktop
File renamed without changes.
0 .../2.0/definitions/2022/CVE-2022-21309.json → ...en/mariner/2.0/definitions/2022/7700.json
View file
Open in desktop
File renamed without changes.
28 changes: 28 additions & 0 deletions mariner/testdata/golden/mariner/2.0/definitions/2023/31872-1.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"Class": "vulnerability",
"ID": "oval:com.microsoft.cbl-mariner:def:31872",
"Version": "1",
"Metadata": {
"Title": "CVE-2023-5678 affecting package edk2 for versions less than 20230301gitf80f052277c8-38",
"Affected": {
"Family": "unix",
"Platform": "CBL-Mariner"
},
"Reference": {
"RefID": "CVE-2023-5678",
"RefURL": "https://nvd.nist.gov/vuln/detail/CVE-2023-5678",
"Source": "CVE"
},
"Patchable": "true",
"AdvisoryID": "31872-1",
"Severity": "Medium",
"Description": "CVE-2023-5678 affecting package edk2 for versions less than 20230301gitf80f052277c8-38. A patched version of the package is available."
},
"Criteria": {
"Operator": "AND",
"Criterion": {
"Comment": "Package edk2 is earlier than 20230301gitf80f052277c8-38, affected by CVE-2023-5678",
"TestRef": "oval:com.microsoft.cbl-mariner:tst:31872000"
}
}
}
28 changes: 28 additions & 0 deletions mariner/testdata/golden/mariner/2.0/definitions/2023/31880-1.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
{
"Class": "vulnerability",
"ID": "oval:com.microsoft.cbl-mariner:def:31880",
"Version": "1",
"Metadata": {
"Title": "CVE-2023-5678 affecting package openssl for versions less than 1.1.1k-28",
"Affected": {
"Family": "unix",
"Platform": "CBL-Mariner"
},
"Reference": {
"RefID": "CVE-2023-5678",
"RefURL": "https://nvd.nist.gov/vuln/detail/CVE-2023-5678",
"Source": "CVE"
},
"Patchable": "true",
"AdvisoryID": "31880-1",
"Severity": "Medium",
"Description": "CVE-2023-5678 affecting package openssl for versions less than 1.1.1k-28. A patched version of the package is available."
},
"Criteria": {
"Operator": "AND",
"Criterion": {
"Comment": "Package openssl is earlier than 1.1.1k-28, affected by CVE-2023-5678",
"TestRef": "oval:com.microsoft.cbl-mariner:tst:31880000"
}
}
}
10 changes: 10 additions & 0 deletions mariner/testdata/golden/mariner/2.0/objects/objects.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,16 @@
"ID": "oval:com.microsoft.cbl-mariner:obj:1643374850000669",
"Version": "1643374850",
"Name": "mysql"
},
{
"ID": "oval:com.microsoft.cbl-mariner:obj:31880001",
"Version": "0",
"Name": "openssl"
},
{
"ID": "oval:com.microsoft.cbl-mariner:obj:31872001",
"Version": "0",
"Name": "edk2"
}
]
}
18 changes: 18 additions & 0 deletions mariner/testdata/golden/mariner/2.0/states/states.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,24 @@
"Datatype": "evr_string",
"Operation": "less than or equal"
}
},
{
"ID": "oval:com.microsoft.cbl-mariner:ste:31880002",
"Version": "0",
"Evr": {
"Text": "0:1.1.1k-28.cm2",
"Datatype": "evr_string",
"Operation": "less than"
}
},
{
"ID": "oval:com.microsoft.cbl-mariner:ste:31872002",
"Version": "0",
"Evr": {
"Text": "0:20230301gitf80f052277c8-38.cm2",
"Datatype": "evr_string",
"Operation": "less than"
}
}
]
}
24 changes: 24 additions & 0 deletions mariner/testdata/golden/mariner/2.0/tests/tests.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,30 @@
"State": {
"StateRef": "oval:com.microsoft.cbl-mariner:ste:1643374850000670"
}
},
{
"Check": "at least one",
"Comment": "Package openssl is earlier than 1.1.1k-28, affected by CVE-2023-5678",
"ID": "oval:com.microsoft.cbl-mariner:tst:31880000",
"Version": "0",
"Object": {
"ObjectRef": "oval:com.microsoft.cbl-mariner:obj:31880001"
},
"State": {
"StateRef": "oval:com.microsoft.cbl-mariner:ste:31880002"
}
},
{
"Check": "at least one",
"Comment": "Package edk2 is earlier than 20230301gitf80f052277c8-38, affected by CVE-2023-5678",
"ID": "oval:com.microsoft.cbl-mariner:tst:31872000",
"Version": "0",
"Object": {
"ObjectRef": "oval:com.microsoft.cbl-mariner:obj:31872001"
},
"State": {
"StateRef": "oval:com.microsoft.cbl-mariner:ste:31872002"
}
}
]
}
52 changes: 52 additions & 0 deletions mariner/testdata/happy/cbl-mariner-2.0-preview-oval.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,38 @@
<criterion comment="Package mysql is installed with version 8.0.24 or earlier" test_ref="oval:com.microsoft.cbl-mariner:tst:1643374850000854"/>
</criteria>
</definition>
<definition class="vulnerability" id="oval:com.microsoft.cbl-mariner:def:31880" version="1">
<metadata>
<title>CVE-2023-5678 affecting package openssl for versions less than 1.1.1k-28</title>
<affected family="unix">
<platform>CBL-Mariner</platform>
</affected>
<reference ref_id="CVE-2023-5678" ref_url="https://nvd.nist.gov/vuln/detail/CVE-2023-5678" source="CVE"/>
<patchable>true</patchable>
<advisory_id>31880-1</advisory_id>
<severity>Medium</severity>
<description>CVE-2023-5678 affecting package openssl for versions less than 1.1.1k-28. A patched version of the package is available.</description>
</metadata>
<criteria operator="AND">
<criterion comment="Package openssl is earlier than 1.1.1k-28, affected by CVE-2023-5678" test_ref="oval:com.microsoft.cbl-mariner:tst:31880000"/>
</criteria>
</definition>
<definition class="vulnerability" id="oval:com.microsoft.cbl-mariner:def:31872" version="1">
<metadata>
<title>CVE-2023-5678 affecting package edk2 for versions less than 20230301gitf80f052277c8-38</title>
<affected family="unix">
<platform>CBL-Mariner</platform>
</affected>
<reference ref_id="CVE-2023-5678" ref_url="https://nvd.nist.gov/vuln/detail/CVE-2023-5678" source="CVE"/>
<patchable>true</patchable>
<advisory_id>31872-1</advisory_id>
<severity>Medium</severity>
<description>CVE-2023-5678 affecting package edk2 for versions less than 20230301gitf80f052277c8-38. A patched version of the package is available.</description>
</metadata>
<criteria operator="AND">
<criterion comment="Package edk2 is earlier than 20230301gitf80f052277c8-38, affected by CVE-2023-5678" test_ref="oval:com.microsoft.cbl-mariner:tst:31872000"/>
</criteria>
</definition>
</definitions>
<tests>
<linux-def:rpminfo_test check="at least one" comment="Package unzip is installed with version 6.0 or earlier" id="oval:com.microsoft.cbl-mariner:tst:1643374850000269" version="1643374850">
Expand All @@ -69,6 +101,14 @@
<linux-def:object object_ref="oval:com.microsoft.cbl-mariner:obj:1643374850000669"/>
<linux-def:state state_ref="oval:com.microsoft.cbl-mariner:ste:1643374850000670"/>
</linux-def:rpminfo_test>
<linux-def:rpminfo_test check="at least one" comment="Package openssl is earlier than 1.1.1k-28, affected by CVE-2023-5678" id="oval:com.microsoft.cbl-mariner:tst:31880000" version="0">
<linux-def:object object_ref="oval:com.microsoft.cbl-mariner:obj:31880001"/>
<linux-def:state state_ref="oval:com.microsoft.cbl-mariner:ste:31880002"/>
</linux-def:rpminfo_test>
<linux-def:rpminfo_test check="at least one" comment="Package edk2 is earlier than 20230301gitf80f052277c8-38, affected by CVE-2023-5678" id="oval:com.microsoft.cbl-mariner:tst:31872000" version="0">
<linux-def:object object_ref="oval:com.microsoft.cbl-mariner:obj:31872001"/>
<linux-def:state state_ref="oval:com.microsoft.cbl-mariner:ste:31872002"/>
</linux-def:rpminfo_test>
</tests>
<objects>
<linux-def:rpminfo_object id="oval:com.microsoft.cbl-mariner:obj:1643374850000123" version="1643374850">
Expand All @@ -80,6 +120,12 @@
<linux-def:rpminfo_object id="oval:com.microsoft.cbl-mariner:obj:1643374850000669" version="1643374850">
<linux-def:name>mysql</linux-def:name>
</linux-def:rpminfo_object>
<linux-def:rpminfo_object id="oval:com.microsoft.cbl-mariner:obj:31880001" version="0">
<linux-def:name>openssl</linux-def:name>
</linux-def:rpminfo_object>
<linux-def:rpminfo_object id="oval:com.microsoft.cbl-mariner:obj:31872001" version="0">
<linux-def:name>edk2</linux-def:name>
</linux-def:rpminfo_object>
</objects>
<states>
<linux-def:rpminfo_state id="oval:com.microsoft.cbl-mariner:ste:1643374850000031" version="1643374850">
Expand All @@ -91,5 +137,11 @@
<linux-def:rpminfo_state id="oval:com.microsoft.cbl-mariner:ste:1643374850000670" version="1643374850">
<linux-def:evr datatype="evr_string" operation="less than or equal">0:8.0.24-1.cm1</linux-def:evr>
</linux-def:rpminfo_state>
<linux-def:rpminfo_state id="oval:com.microsoft.cbl-mariner:ste:31880002" version="0">
<linux-def:evr datatype="evr_string" operation="less than">0:1.1.1k-28.cm2</linux-def:evr>
</linux-def:rpminfo_state>
<linux-def:rpminfo_state id="oval:com.microsoft.cbl-mariner:ste:31872002" version="0">
<linux-def:evr datatype="evr_string" operation="less than">0:20230301gitf80f052277c8-38.cm2</linux-def:evr>
</linux-def:rpminfo_state>
</states>
</oval_definitions>