golang `stdlib` scan inconsistency for fresh go toolchain versions: bare metal vs docker #679

tommiv · 2023-11-27T11:58:50Z

I've recently faced a weird issue in our CI process – it reports some vulnerabilities for go v1.20, while the go.mod file claims go 1.21.4. At the same time, my local version on a dev machine reports "no vulnerabilities found".

So, given this /tmp/osv-issue/go.mod file (not much to see here, the original one is way bigger, but the issue persists even with a minimal one):

module some_module_name

go 1.21.4

require (
)

I have the following results:

[MacOS v14.1.1] [clean] bare metal binary, installed via homebrew:

osv-scanner -v
osv-scanner version: 1.4.3
commit: n/a
built at: n/a

Scanned /tmp/osv-issue/go.mod file and found 1 package
No vulnerabilities found

[MacOS v14.1.1] [clean] bare metal binary, manual download darwin_amd64 from github releases:

curl -OL https://github.com/google/osv-scanner/releases/download/v1.4.3/osv-scanner_1.4.3_darwin_amd64
chmod +x osv-scanner_1.4.3_darwin_amd64
./osv-scanner_1.4.3_darwin_amd64 -v
osv-scanner version: 1.4.3
commit: 6316373e47d7e3e4b4fd3630c4bbc10987738de6
built at: 2023-11-02T00:53:14Z

./osv-scanner_1.4.3_darwin_amd64 -L /tmp/osv-issue/go.mod
Scanned /tmp/osv-issue/go.mod file and found 1 package
No vulnerabilities found

[vulnerable] official docker way (copypasted some commands from docs)

docker run -it -v ${PWD}:/src ghcr.io/google/osv-scanner:v1.4.3 -v            
osv-scanner version: 1.4.3
commit: 6316373e47d7e3e4b4fd3630c4bbc10987738de6
built at: 2023-11-02T00:53:14Z

 docker run -it -v /tmp/osv-issue:/src ghcr.io/google/osv-scanner:v1.4.3 -L /src/go.mod
Scanned /src/go.mod file and found 1 package
╭──────────────────────────────┬──────┬───────────┬─────────┬─────────┬────────────╮
│ OSV URL                      │ CVSS │ ECOSYSTEM │ PACKAGE │ VERSION │ SOURCE     │
├──────────────────────────────┼──────┼───────────┼─────────┼─────────┼────────────┤
│ https://osv.dev/GO-2023-2185 │      │ Go        │ stdlib  │ 1.20.10 │ src/go.mod │
│                              │      │           │         │         │            │
│ https://osv.dev/GO-2023-2186 │      │ Go        │ stdlib  │ 1.20.10 │ src/go.mod │
│                              │      │           │         │         │            │
╰──────────────────────────────┴──────┴───────────┴─────────┴─────────┴────────────╯

As you can see, the result in docker is quite different, while it runs binaries compiled against exactly the same codebase (based on the commit hash) – it reports vulnerabilities for go stdlib v.1.20.10.

It seems that osv-scanner uses the original golang toolchain to get the report (it fails to run if go is not installed), so I've tried to install golang in a clean container of alpine:latest and indeed it emits

go version
go version go1.20.11 linux/amd64

Vulnerabilities above are actually already fixed in v1.20.11 but the ghcr.io/google/osv-scanner:v1.4.3 image was built three weeks ago, probably before the patch had become available.

The bottom line here is: in my case our CI reports false positives.

P.S. TBH I have no idea how one could solve that with the official image, since it's on the alpine side to provide the latest go toolchain version, so I expect this issue will be closed as "won't fix", just wanted to bring the problem to maintainers attention.

The text was updated successfully, but these errors were encountered:

another-rex · 2023-11-28T02:04:41Z

This seems to be a result of: #453. The assumption is made that the go binary found on the PATH will be the version that is building the go package, which might not always be the case.

I think we want to do a couple of things here:

Remove golang from the docker image, as it is not necessary to even be there (example is the action.dockerfile image) (This will break/skip gouvlncheck call analysis though.)
Add an option to override the go version, so you can input your own actual go version to show correct results.

tommiv · 2023-11-28T09:40:38Z

@another-rex

Remove golang from the docker image, as it is not necessary to even be there

I don't know of osv-scanner internals, but this would definetely break golang stdlib checks mentioned in #453, here's the quick demo:

docker run -it alpine:latest
apk add curl
cd /tmp
curl -OL https://github.com/google/osv-scanner/releases/download/v1.4.3/osv-scanner_1.4.3_linux_amd64
chmod +x osv-scanner_1.4.3_linux_amd64
printf "module test\ngo 1.121\n" > go.mod
./osv-scanner_1.4.3_linux_amd64 -L go.mod

This will emit the following:

cannot get go standard library version, go might not be installed: exec: "go": executable file not found in $PATH
Scanned /tmp/go.mod file and found 0 packages
No package sources found, --help for usage information.

So, I guess, providing some way to switch to a desired go toolchain version is prefered. However I understood from #453 that as of now, there's no clear way to do this.

UPD: As an alternative, not suitable for everyone though, one can use golang:$VERSION-alpine image and just download osv-scanner from github releases. In this approach though it will be necessary to stick to a particular version of osv-scanner or use additional scripting/software to obtain the latest release tag from the github releases page, since github itself doesn't provide an alias url for the "latest" release.

another-rex · 2023-11-28T23:52:16Z

cannot get go standard library version, go might not be installed: exec: "go": executable file not found in $PATH

This error has been converted into a warning in the next release (#622) (v1.5.0 is currently planned to release next week), and as long as there is a package in the go.mod file, it should successfully scan.

another-rex · 2023-11-29T04:54:33Z

After discussing this with the team, we are going to change the implementation to read the go.mod file for the go version, and if no patch version is specified in go.mod, we assume .0 patch version. This will replace finding the go version in $PATH proposed in #453.

It looks like after 1.21, go version in go.mod is enforced as the minimum version.

Fix #679

spencerschrock · 2024-02-21T01:49:07Z

I was playing around with this when trying out Go call analysis.

The toy project uses an old version of Go intentionally.
Based on my understanding of https://osv.dev/vulnerability/GO-2023-2102:
http.ListenAndServe is vulnerable for versions before 1.20.10 (or 1.21.3)

package main

import (
    "log"
    "net/http"
)

func main() {
    err := http.ListenAndServe(":8080", nil)
    if err != nil {
        log.Fatal(err)
    }
}

the example go.mod declares a vulnerable version:

module example.com/foo/bar

go 1.19

But osv-scanner seems to also care what my Go version is when determining reachability (go.mod still says 1.19).

$ go version
go version go1.22.0 linux/amd64

$ osv-scanner -r .
Scanning dir .
Scanned /tmp/go/go.mod file and found 1 package
Uncalled vulnerabilities 
│ https://osv.dev/GO-2023-2375 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2041 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2043 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2102 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2185 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2186 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2382 │      │ Go        │ stdlib  │ 1.19    │ go.mod │

Vs if I use an older Go toolchain (example go.mod is still 1.19).

$ go version
go version go1.21.2 linux/amd64
$ osv-scanner -r .
Scanning dir .
Scanned /tmp/go/go.mod file and found 1 package
(called vulnerabilities section)
│ https://osv.dev/GO-2023-2102 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2185 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2382 │      │ Go        │ stdlib  │ 1.19    │ go.mod │

Uncalled vulnerabilities
│ https://osv.dev/GO-2023-2375 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2041 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2043 │      │ Go        │ stdlib  │ 1.19    │ go.mod │
│ https://osv.dev/GO-2023-2186 │      │ Go        │ stdlib  │ 1.19    │ go.mod │

Am I misunderstanding what #704 did? I'm trying to understand this behavior before enabling it for Scorecard because our weekly scan runs with the latest version of the Go toolchain. So I don't want to dismiss everything as uncalled based on the Go toolchain version we use.

hogo6002 · 2024-02-21T04:24:07Z

Thanks @spencerschrock for reporting this.

As expected, osv-scanner should not care what the local Go version is when determining reachability. It should only checks Go version that is defined in go.mod.

I think the issue here is because when we run govulncheck, we didn't set GOVERSION to the same one as go.mod (By default, GOVERSION represents the installed go version.). And govulncheck takes this variable to determine vulns.

To fix it, I will raise a PR to assign GOVERSION to the right value before executing govulncheck. How do you think @another-rex

$ export GOVERSION=go1.19.0
$ go run ./cmd/osv-scanner/ -r ../test3

│ https://osv.dev/GO-2023-2375 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2102 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2185 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2382 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ Uncalled vulnerabilities     │      │           │         │         │                 │
│ https://osv.dev/GO-2023-2041 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2043 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2186 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │

exit status 1

$ export GOVERSION=go1.22.0
$ go run ./cmd/osv-scanner/ -r ../test3
│ Uncalled vulnerabilities     │      │           │         │         │                 │
│ https://osv.dev/GO-2023-2375 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2041 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2043 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2102 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2185 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2186 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │
│ https://osv.dev/GO-2023-2382 │      │ Go        │ stdlib  │ 1.19    │ ../test3/go.mod │

spencerschrock · 2024-02-21T07:13:32Z

Ah I didn't know about GOVERSION. I tried clearing the Env variables mentioned in the docs, thinking it had to do with GOROOT (or similar) but setting a new value for GOVERSION sounds perfect.
https://pkg.go.dev/golang.org/x/vuln/scan#Cmd

GoVulncheck uses local installed GO version to determine vulnerabilities if the env `GOVERSION` is not defined. Set the `GOVERSION` value to the one defined in go.mod. detail: #679 --------- Co-authored-by: Rex P <106129829+another-rex@users.noreply.github.com>

another-rex added enhancement New feature or request good first issue Good for newcomers and removed good first issue Good for newcomers labels Nov 29, 2023

another-rex assigned hogo6002 Nov 30, 2023

hogo6002 mentioned this issue Dec 11, 2023

Get go stdlib version from go.mod #704

Merged

G-Rath mentioned this issue Dec 18, 2023

fix: get stdlib version for Go from go.mod #713

Closed

hogo6002 closed this as completed in #704 Dec 27, 2023

hogo6002 added a commit that referenced this issue Dec 27, 2023

Get go stdlib version from go.mod (#704)

dd30864

Fix #679

another-rex mentioned this issue Dec 27, 2023

Fix vulnerabilities that OSV-Scanner found #724

Merged

another-rex mentioned this issue Jan 10, 2024

Assume latest patch version if version does not exist #740

Merged

hogo6002 mentioned this issue Feb 22, 2024

Align GoVulncheck Go version with go.mod #818

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

golang `stdlib` scan inconsistency for fresh go toolchain versions: bare metal vs docker #679

golang `stdlib` scan inconsistency for fresh go toolchain versions: bare metal vs docker #679

tommiv commented Nov 27, 2023

another-rex commented Nov 28, 2023 •

edited

tommiv commented Nov 28, 2023 •

edited

another-rex commented Nov 28, 2023

another-rex commented Nov 29, 2023

spencerschrock commented Feb 21, 2024

hogo6002 commented Feb 21, 2024 •

edited

spencerschrock commented Feb 21, 2024 •

edited

golang stdlib scan inconsistency for fresh go toolchain versions: bare metal vs docker #679

golang stdlib scan inconsistency for fresh go toolchain versions: bare metal vs docker #679

Comments

tommiv commented Nov 27, 2023

[MacOS v14.1.1] [clean] bare metal binary, installed via homebrew:

[MacOS v14.1.1] [clean] bare metal binary, manual download darwin_amd64 from github releases:

[vulnerable] official docker way (copypasted some commands from docs)

another-rex commented Nov 28, 2023 • edited

tommiv commented Nov 28, 2023 • edited

another-rex commented Nov 28, 2023

another-rex commented Nov 29, 2023

spencerschrock commented Feb 21, 2024

hogo6002 commented Feb 21, 2024 • edited

spencerschrock commented Feb 21, 2024 • edited

golang `stdlib` scan inconsistency for fresh go toolchain versions: bare metal vs docker #679

golang `stdlib` scan inconsistency for fresh go toolchain versions: bare metal vs docker #679

another-rex commented Nov 28, 2023 •

edited

tommiv commented Nov 28, 2023 •

edited

hogo6002 commented Feb 21, 2024 •

edited

spencerschrock commented Feb 21, 2024 •

edited