Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: "go build" fails if go.sum is not writable #30185

Closed
atombender opened this issue Feb 12, 2019 · 11 comments
Closed

cmd/go: "go build" fails if go.sum is not writable #30185

atombender opened this issue Feb 12, 2019 · 11 comments
Assignees
Milestone

Comments

@atombender
Copy link

@atombender atombender commented Feb 12, 2019

What version of Go are you using (go version)?

$ go version
1.11.5

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/alex/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/alex/.go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/opt/go/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/opt/go/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/alex/Trash/gotest/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/qs/wpmg19r12_9_nz7pvvs2_82r0000gn/T/go-build642111807=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I have a Dockerfile that runs go mod download followed by go build. Both the download and the build are run as an unprivileged user against a read-only codebase whose files are owned by root and not writable by anyone else.

A rudimentary dockerfile:

FROM golang:1.11-alpine AS build
RUN apk add --no-cache curl git ca-certificates \
  && addgroup -S build && adduser -S -G build build
WORKDIR /srv
COPY . ./
USER build
RUN go mod download
RUN go mod verify
RUN CGO_ENABLED=0 go build -mod=readonly -o ./server ./cmd/server

Possibly related to #29452?

What did you expect to see?

I expect go build to not attempt to write go.sum, especially if -mod=readonly is provided. I can obviously chmod the files to be writable, but this still shouldn't fail.

What did you see instead?

go: writing go.sum: open /srv/go.sum: permission denied
@agnivade

This comment has been minimized.

Copy link
Contributor

@agnivade agnivade commented Feb 12, 2019

/cc @bcmills

@dsymonds

This comment has been minimized.

Copy link
Member

@dsymonds dsymonds commented Aug 18, 2019

This was biting me just now. I believe the source of the difference is that my local dev machine has a newer version of Go than the continuous build I'm using, so when it runs go get -d ./... it gets a slightly different set of dependencies, then fails because it can't write go.sum.

I don't think failing to write go.sum should cause the go tool to exit unsuccessfully.

@dsymonds

This comment has been minimized.

Copy link
Member

@dsymonds dsymonds commented Aug 18, 2019

It is also made worse in my situation by the go tool writing its temporary files to the package directory (where the go.mod and go.sum would end up) rather than using $TMPDIR.

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Aug 19, 2019

It is also made worse in my situation by the go tool writing its temporary files to the package directory (where the go.mod and go.sum would end up) rather than using $TMPDIR.

That part is necessary in general:

  • To ensure that we don't lose existing information if the go command is killed partway through a write, we write to a temporary file and then rename it over the existing file.
  • That rename operation fails if the temporary file is not on the same filesystem as the destination.
  • The most portable way to ensure that both files are on the same filesystem is to construct them in the same directory.
@dsymonds

This comment has been minimized.

Copy link
Member

@dsymonds dsymonds commented Aug 19, 2019

I'm not sure I see why the go tool is so different to the multitude of tools that have existed for decades that quite happily use $TMPDIR for the same effect.

@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@bcmills bcmills modified the milestones: Backlog, Go1.14 Oct 10, 2019
@griesemer griesemer added the Tools label Oct 16, 2019
@griesemer

This comment has been minimized.

Copy link
Contributor

@griesemer griesemer commented Oct 16, 2019

Related to #34860?

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Oct 16, 2019

Somewhat? They're mostly orthogonal, I think, but this issue can make the failure mode for #34860 worse (or less explicable).

@dowgird

This comment has been minimized.

Copy link

@dowgird dowgird commented Oct 23, 2019

This makes guru fail on (some) projects fetched from Github. Reproduction & error message pasted below to make discovery easier.

go get github.com/GoogleCloudPlatform/osconfig
guru "-scope" "github.com/GoogleCloudPlatform/osconfig" "-tags=" "callstack" "$GOPATH/src/github.com/GoogleCloudPlatform/osconfig/inventory/inventory.go:#1629"

Result (snippet):

 [redacted]/go/pkg/mod/github.com/golang/protobuf@v1.3.2/ptypes/duration/duration.pb.go:8:8: could not import github.com/golang/protobuf/proto (go/build: importGo github.com/golang/protobuf/proto:      exit status 1
    error writing go.mod: open [redacted]/go/pkg/mod/github.com/golang/protobuf@v1.3.2/go.mod298498081.tmp: permission denied
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Oct 31, 2019

Change https://golang.org/cl/204521 mentions this issue: cmd/go: default to mod=readonly when the go.mod file is read-only

gopherbot pushed a commit that referenced this issue Nov 1, 2019
Updates #30185
Updates #33326
Updates #34822

Change-Id: Ie13651585898d1bbbf4f779b97ee50b6c7e7ad50
Reviewed-on: https://go-review.googlesource.com/c/go/+/204521
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Nov 6, 2019

The use of $TMPDIR will be addressed incidentally by the fix for #34634.

The remainder is working as designed per #30667.

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Nov 6, 2019

Duplicate of #34054

@bcmills bcmills marked this as a duplicate of #34054 Nov 6, 2019
@bcmills bcmills closed this Nov 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
10 participants
You can’t perform that action at this time.