Cosign builds aren't reproducible

[naveensrinivasan]

Description
Checkout 1.3.0 and run a build
git checkout -b repro-builds v1.3.0

make
CGO_ENABLED=0 go build -trimpath -ldflags "-X github.com/sigstore/cosign/cmd/cosign/cli/options.GitVersion=v1.3.0 -X github.com/sigstore/cosign/cmd/cosign/cli/options.gitCommit=a91aa202a01b830dafa969bb46f168e9c44580bd -X github.com/sigstore/cosign/cmd/cosign/cli/options.gitTreeState="clean" -X github.com/sigstore/cosign/cmd/cosign/cli/options.buildDate='1980-01-01T00:00:00Z'" -o cosign ./cmd/cosign

sha256sum ./cosign
9cd98af02981f709d0a2c281aede8b45afb4bef023face3ae177510c7ba5b02e  ./cosign

Download from the release

curl -L -o cosgin https://github.com/sigstore/cosign/releases/download/v1.3.0/cosign-linux-amd64

sha256sum ./cosign
c7a6819408e4580e561ba4e060afdaeb161af353f8f8fefc33171f7d62cd98a1  ./cosign

[naveensrinivasan]

cc @cpanato

[cpanato]

i will take a look on that

[cpanato]

@naveensrinivasan something looks wrong with the sha256sum from the binary that you downloaded from github

the checksum file in the release says that binary have a sha256 sum of 9604a5eb171748113f92a67495556007dde6f45804f0b38d3e55c3bc7e151774 cosign-linux-amd64 and not c7a6819408e4580e561ba4e060afdaeb161af353f8f8fefc33171f7d62cd98a1

I've downloaded using the same command as you and got the same shasum as in the file

$ curl -L -o cosgin https://github.com/sigstore/cosign/releases/download/v1.3.0/cosign-linux-amd64
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   657  100   657    0     0   1052      0 --:--:-- --:--:-- --:--:--  1061
100 77.6M  100 77.6M    0     0  13.1M      0  0:00:05  0:00:05 --:--:-- 19.0M

$ sha256sum cosgin
9604a5eb171748113f92a67495556007dde6f45804f0b38d3e55c3bc7e151774  cosgin

$ sha256sum ./cosgin
9604a5eb171748113f92a67495556007dde6f45804f0b38d3e55c3bc7e151774  ./cosgin

also using wget

$ wget https://github.com/sigstore/cosign/releases/download/v1.3.0/cosign-linux-amd64
--2021-11-10 10:43:23--  https://github.com/sigstore/cosign/releases/download/v1.3.0/cosign-linux-amd64
Resolving github.com (github.com)... 140.82.112.4
Connecting to github.com (github.com)|140.82.112.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/335952417/6e6fd0a1-aebb-4f07-8d97-e51e808f1825?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211110%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211110T094324Z&X-Amz-Expires=300&X-Amz-Signature=b26fed6a4f4ebc1895e0029434bade47890c0301cddcb21ea6faa33832a72de1&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=
335952417&response-content-disposition=attachment%3B%20filename%3Dcosign-linux-amd64&response-content-type=application%2Foctet-stream [following]
--2021-11-10 10:43:24--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/335952417/6e6fd0a1-aebb-4f07-8d97-e51e808f1825?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211110%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211110T094324Z&X-Amz-Expires=300&X-Amz-Signature=b26fed6a4f4ebc1895e0029434bade47890c0301cddcb21ea6faa33832a72de1&X-Amz-SignedHeaders=host&actor_id=0&ke
y_id=0&repo_id=335952417&response-content-disposition=attachment%3B%20filename%3Dcosign-linux-amd64&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.110.133, 185.199.108.133, 185.199.109.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.110.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 81393043 (78M) [application/octet-stream]
Saving to: ‘cosign-linux-amd64’

cosign-linux-amd64                                    100%[=======================================================================================================================>]  77.62M  27.4MB/s    in 2.8s

2021-11-10 10:43:29 (27.4 MB/s) - ‘cosign-linux-amd64’ saved [81393043/81393043]

$ sha256sum cosign-linux-amd64
9604a5eb171748113f92a67495556007dde6f45804f0b38d3e55c3bc7e151774  cosign-linux-amd64

[cpanato]

i got a successful reproducible build using a container

Dockerfile definition

FROM debian:jessie

RUN apt-get update && apt-get install -y unzip wget tar ca-certificates git build-essential

RUN wget https://storage.googleapis.com/golang/go1.17.2.linux-amd64.tar.gz
RUN tar xvf go1.17.2.linux-amd64.tar.gz
RUN mkdir -p /opt/go && cp -r go /opt/go/go1.17.2
RUN cd /opt/go/go1.17.2/src && GOROOT_BOOTSTRAP=/go ./make.bash

ENV PATH "/opt/go/go1.17.2/bin:$PATH"

RUN mkdir -p /github.com/sigstore/
RUN cd /github.com/sigstore && git clone https://github.com/sigstore/cosign
RUN cd /github.com/sigstore/cosign && git checkout v1.3.0

ENV GOPATH /go

ENTRYPOINT ["go"]

build the container: docker build . -t cosing-repro-build

in the local cosign directory you can run the docker to build the binary

docker run -it --rm -v $(pwd):$(pwd) -w $(pwd) -e CGO_ENABLED=0 6b46a57893d4 build --ldflags "-X github.com/sigstore/cosign/cmd/cosign/cli/options.GitVersion=v1.3.0 -X github.com/sigstore/cosign/cmd/cosign/cli/options.gitCommit=a91aa202a01b830dafa969bb46f168e9c44580bd -X github.com/sigstore/cosign/cmd/cosign/cli/options.gitTreeState="clean" -X github.com/sigstore/cosign/cmd/cosign/cli/options.buildDate='2021-11-03T00:05:51Z'" -trimpath -o cosign -o cosign github.com/sigstore/cosign/cmd/cosign

the binary will be out in the host and then you can check the sha256sum

$ sha256sum ./cosign
9604a5eb171748113f92a67495556007dde6f45804f0b38d3e55c3bc7e151774  ./cosign

which is the same as in the https://github.com/sigstore/cosign/releases/download/v1.3.0/cosign_checksums.txt

I've run 10 times the same and always got the same sha256 output

[cpanato]

/cc @mattmoor @dlorenc @dekkagaijin

[dlorenc]

Is it a go version issue? Are we sure we're using the same version of go in the release?

[cpanato]

Is it a go version issue? Are we sure we're using the same version of go in the release?

in the release v1.3.0 the go version was 1.17.2, now we are in 1.17.3

[cpanato]

i think we can close this issue
if not feel free to reopen or create a new one