Skip to content

Commit

Permalink
ci: run tests on Windows (google#646)
Browse files Browse the repository at this point in the history
Unsurprisingly this has required a bunch of tests to be updated to
handle slightly different variations in file path handling - this
eventually resulted in me implementing an actual internal snapshot
testing package but I've not included that in here since its sizable on
its own; so please keep that in mind when reviewing . (see
https://github.com/G-Rath/osv-scanner/commit/1273da79e2e26a18d663da482dc5f09258e15c51
for a sneakpeek on what the snapshot-based testing looks like)

~Note that is failing because file -> url path translation is actually
busted; I've opened google#645 to fix this and you can see the passing CI when
both of these changes are merged in google#553~

Resolves google#603
Resolves google#553
  • Loading branch information
G-Rath committed Nov 21, 2023
1 parent eb862fd commit 521f59c
Show file tree
Hide file tree
Showing 13 changed files with 1,195 additions and 36 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- name: Check out code
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/goreleaser.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- name: Check out code
Expand Down
38 changes: 28 additions & 10 deletions cmd/osv-scanner/main_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (

"github.com/go-git/go-git/v5"
"github.com/google/go-cmp/cmp"
"github.com/google/osv-scanner/internal/testutility"
"github.com/google/osv-scanner/internal/version"
)

Expand Down Expand Up @@ -105,6 +106,16 @@ func expectAreEqual(t *testing.T, subject, actual, expect string) {
}
}

// Attempts to normalize any file paths in the given `output` so that they can
// be compared reliably regardless of the file path separator being used.
//
// Namely, escaped forward slashes are replaced with backslashes.
func normalizeFilePaths(t *testing.T, output string) string {
t.Helper()

return strings.ReplaceAll(strings.ReplaceAll(output, "\\\\", "/"), "\\", "/")
}

// normalizeRootDirectory attempts to replace references to the current working
// directory with "<rootdir>", in order to reduce the noise of the cmp diff
func normalizeRootDirectory(t *testing.T, str string) string {
Expand All @@ -115,6 +126,11 @@ func normalizeRootDirectory(t *testing.T, str string) string {
t.Errorf("could not get cwd (%v) - results and diff might be inaccurate!", err)
}

cwd = normalizeFilePaths(t, cwd)

// file uris with Windows end up with three slashes, so we normalize that too
str = strings.ReplaceAll(str, "file:///"+cwd, "file://<rootdir>")

return strings.ReplaceAll(str, cwd, "<rootdir>")
}

Expand All @@ -127,11 +143,11 @@ func testCli(t *testing.T, tc cliTestCase) {
ec := run(tc.args, stdoutBuffer, stderrBuffer)
// ec := run(tc.args, os.Stdout, os.Stderr)

stdout := stdoutBuffer.String()
stderr := stderrBuffer.String()
stdout := normalizeRootDirectory(t, normalizeFilePaths(t, stdoutBuffer.String()))
stderr := normalizeRootDirectory(t, normalizeFilePaths(t, stderrBuffer.String()))

stdout = normalizeRootDirectory(t, stdout)
stderr = normalizeRootDirectory(t, stderr)
tc.wantStdout = normalizeFilePaths(t, tc.wantStdout)
tc.wantStderr = normalizeFilePaths(t, tc.wantStderr)

if ec != tc.wantExitCode {
t.Errorf("cli exited with code %d, not %d", ec, tc.wantExitCode)
Expand Down Expand Up @@ -572,9 +588,10 @@ func TestRun_LockfileWithExplicitParseAs(t *testing.T) {
},
wantExitCode: 127,
wantStdout: "",
wantStderr: `
open <rootdir>/path/to/my:file: no such file or directory
`,
wantStderr: testutility.ValueIfOnWindows(
"open <rootdir>/path/to/my:file: The system cannot find the path specified.",
"open <rootdir>/path/to/my:file: no such file or directory",
),
},
{
name: "",
Expand All @@ -585,9 +602,10 @@ func TestRun_LockfileWithExplicitParseAs(t *testing.T) {
},
wantExitCode: 127,
wantStdout: "",
wantStderr: `
open <rootdir>/path/to/my:project/package-lock.json: no such file or directory
`,
wantStderr: testutility.ValueIfOnWindows(
"open <rootdir>/path/to/my:project/package-lock.json: The filename, directory name, or volume label syntax is incorrect.",
"open <rootdir>/path/to/my:project/package-lock.json: no such file or directory",
),
},
// one lockfile with local path
{
Expand Down
52 changes: 52 additions & 0 deletions internal/output/fixtures/sarif-commit-output_windows.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
**Your dependency is vulnerable to [OSV-2023-72](https://osv.dev/list?q=OSV-2023-72)**.

## [OSV-2023-72](https://osv.dev/vulnerability/OSV-2023-72)

<details>
<summary>Details</summary>

> OSS-Fuzz report: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=56057
>
> ```
> Crash type: Heap-buffer-overflow WRITE 4
> Crash state:
> perfetto::trace_processor::TrackEventParser::ParseTrackDescriptor
> perfetto::trace_processor::TrackEventModule::ParseTracePacketData
> perfetto::trace_processor::ProtoTraceParser::ParseTracePacket
> ```
>
</details>

---

### Affected Packages

| Source | Package Name | Package Version |
| --- | --- | --- |
| lockfile:/usr/local/google/home/rexpan/Documents/Project/engine/deps_flatten.txt | https://fuchsia.googlesource.com/third_party/android.googlesource.com/platform/external/perfetto | b8da07095979310818f0efde2ef3c69ea70d62c5 |

## Remediation

To fix these vulnerabilities, update the vulnerabilities past the listed fixed versions below.

### Fixed Versions

| Vulnerability ID | Package Name | Fixed Version |
| --- | --- | --- |
| OSV-2023-72 | perfetto | 9a7f09383dd39f19e662d428321ca708a2a600a3 |

If you believe these vulnerabilities do not affect your code and wish to ignore them, add them to the ignore list in an
`osv-scanner.toml` file located in the same directory as the lockfile containing the vulnerable dependency.

See the format and more options in our documentation here: https://google.github.io/osv-scanner/configuration/

Add or append these values to the following config files to ignore this vulnerability:

`\usr\local\google\home\rexpan\Documents\Project\engine/osv-scanner.toml`

```
[[IgnoredVulns]]
id = "OSV-2023-72"
reason = "Your reason for ignoring this vulnerability"
```
130 changes: 130 additions & 0 deletions internal/output/fixtures/sarif-output_windows.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
**Your dependency is vulnerable to [CVE-2022-24713](https://osv.dev/list?q=CVE-2022-24713)**
(Also published as: [RUSTSEC-2022-0013](https://osv.dev/vulnerability/RUSTSEC-2022-0013), [GHSA-m5pq-gvj9-9vr8](https://osv.dev/vulnerability/GHSA-m5pq-gvj9-9vr8), ).

## [RUSTSEC-2022-0013](https://osv.dev/vulnerability/RUSTSEC-2022-0013)

<details>
<summary>Details</summary>

> The Rust Security Response WG was notified that the `regex` crate did not
> properly limit the complexity of the regular expressions (regex) it parses. An
> attacker could use this security issue to perform a denial of service, by
> sending a specially crafted regex to a service accepting untrusted regexes. No
> known vulnerability is present when parsing untrusted input with trusted
> regexes.
>
> This issue has been assigned CVE-2022-24713. The severity of this vulnerability
> is "high" when the `regex` crate is used to parse untrusted regexes. Other uses
> of the `regex` crate are not affected by this vulnerability.
>
> ## Overview
>
> The `regex` crate features built-in mitigations to prevent denial of service
> attacks caused by untrusted regexes, or untrusted input matched by trusted
> regexes. Those (tunable) mitigations already provide sane defaults to prevent
> attacks. This guarantee is documented and it's considered part of the crate's
> API.
>
> Unfortunately a bug was discovered in the mitigations designed to prevent
> untrusted regexes to take an arbitrary amount of time during parsing, and it's
> possible to craft regexes that bypass such mitigations. This makes it possible
> to perform denial of service attacks by sending specially crafted regexes to
> services accepting user-controlled, untrusted regexes.
>
> ## Affected versions
>
> All versions of the `regex` crate before or equal to 1.5.4 are affected by this
> issue. The fix is include starting from `regex` 1.5.5.
>
> ## Mitigations
>
> We recommend everyone accepting user-controlled regexes to upgrade immediately
> to the latest version of the `regex` crate.
>
> Unfortunately there is no fixed set of problematic regexes, as there are
> practically infinite regexes that could be crafted to exploit this
> vulnerability. Because of this, we do not recommend denying known problematic
> regexes.
>
> ## Acknowledgements
>
> We want to thank Addison Crump for responsibly disclosing this to us according
> to the [Rust security policy][1], and for helping review the fix.
>
> We also want to thank Andrew Gallant for developing the fix, and Pietro Albini
> for coordinating the disclosure and writing this advisory.
>
> [1]: https://www.rust-lang.org/policies/security
</details>

## [GHSA-m5pq-gvj9-9vr8](https://osv.dev/vulnerability/GHSA-m5pq-gvj9-9vr8)

<details>
<summary>Details</summary>

> > This is a cross-post of [the official security advisory][advisory]. The official advisory contains a signed version with our PGP key, as well.
>
> [advisory]: https://groups.google.com/g/rustlang-security-announcements/c/NcNNL1Jq7Yw
>
> The Rust Security Response WG was notified that the `regex` crate did not properly limit the complexity of the regular expressions (regex) it parses. An attacker could use this security issue to perform a denial of service, by sending a specially crafted regex to a service accepting untrusted regexes. No known vulnerability is present when parsing untrusted input with trusted regexes.
>
> This issue has been assigned CVE-2022-24713. The severity of this vulnerability is "high" when the `regex` crate is used to parse untrusted regexes. Other uses of the `regex` crate are not affected by this vulnerability.
>
> ## Overview
>
> The `regex` crate features built-in mitigations to prevent denial of service attacks caused by untrusted regexes, or untrusted input matched by trusted regexes. Those (tunable) mitigations already provide sane defaults to prevent attacks. This guarantee is documented and it's considered part of the crate's API.
>
> Unfortunately a bug was discovered in the mitigations designed to prevent untrusted regexes to take an arbitrary amount of time during parsing, and it's possible to craft regexes that bypass such mitigations. This makes it possible to perform denial of service attacks by sending specially crafted regexes to services accepting user-controlled, untrusted regexes.
>
> ## Affected versions
>
> All versions of the `regex` crate before or equal to 1.5.4 are affected by this issue. The fix is include starting from `regex` 1.5.5.
>
> ## Mitigations
>
> We recommend everyone accepting user-controlled regexes to upgrade immediately to the latest version of the `regex` crate.
>
> Unfortunately there is no fixed set of problematic regexes, as there are practically infinite regexes that could be crafted to exploit this vulnerability. Because of this, we do not recommend denying known problematic regexes.
>
> ## Acknowledgements
>
> We want to thank Addison Crump for responsibly disclosing this to us according to the [Rust security policy](https://www.rust-lang.org/policies/security), and for helping review the fix.
>
> We also want to thank Andrew Gallant for developing the fix, and Pietro Albini for coordinating the disclosure and writing this advisory.
</details>

---

### Affected Packages

| Source | Package Name | Package Version |
| --- | --- | --- |
| lockfile:/path/to/sub-rust-project/Cargo.lock | regex | 1.5.1 |

## Remediation

To fix these vulnerabilities, update the vulnerabilities past the listed fixed versions below.

### Fixed Versions

| Vulnerability ID | Package Name | Fixed Version |
| --- | --- | --- |
| GHSA-m5pq-gvj9-9vr8 | regex | 1.5.5 |
| RUSTSEC-2022-0013 | regex | 1.5.5 |

If you believe these vulnerabilities do not affect your code and wish to ignore them, add them to the ignore list in an
`osv-scanner.toml` file located in the same directory as the lockfile containing the vulnerable dependency.

See the format and more options in our documentation here: https://google.github.io/osv-scanner/configuration/

Add or append these values to the following config files to ignore this vulnerability:

`\path\to\sub-rust-project/osv-scanner.toml`

```
[[IgnoredVulns]]
id = "CVE-2022-24713"
reason = "Your reason for ignoring this vulnerability"
```

0 comments on commit 521f59c

Please sign in to comment.