Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: add a new flag to disable unzipping after go mod download #32605

Open
aofei opened this issue Jun 13, 2019 · 2 comments

Comments

@aofei
Copy link
Contributor

@aofei aofei commented Jun 13, 2019

The go mod download caches the module files into the $GOPATH/pkg/mod/cache/download by default and extracts them into the $GOPATH/pkg/mod after the downloads are complete. This's fine. But today I encountered a problem that reminds me that we may need to disable the unzipping operation at some point.


So, I ran a program similar to the following using a non-root user:

package main

import (
	"io/ioutil"
	"log"
	"os"
	"os/exec"
)

func main() {
	tempDir, err := ioutil.TempDir("", "foobar")
	if err != nil {
		log.Fatalf("failed to create temp dir: %v", err)
	}

	cmd := exec.Command("go", "mod", "download", "-json", "golang.org/x/text@latest")
	cmd.Env = append(os.Environ(), "GO111MODULE=on", "GOPATH="+tempDir)
	cmd.Dir = tempDir
	if err := cmd.Run(); err != nil {
		log.Fatalf("failed to download module: %v", err)
	}

	if err := os.RemoveAll(tempDir); err != nil {
		log.Fatalf("failed to remove temp dir: %v", err)
	}
}

Then I got the following failed output:

2019/06/14 01:03:21 failed to remove temp dir: unlinkat /var/folders/j6/2330_fdx4tn9t9tx4y117w700000gn/T/foobar128162500/pkg/mod/golang.org/x/text@v0.3.2/codereview.cfg: permission denied
exit status 1

Actually, I just need to download the module files and save them somewhere else (not in the $GOPATH/pkg/mod/cache/download), then clear the download history after the download is complete. But I got stuck in that unlinkat operation.

In my scenario, I don't actually need the go command to extract the zip files I downloaded into the $GOPATH/pkg/mod, but the go command doesn't ask me if it can do that. This not only caused unnecessary computing, but also causes problems such as the inability to remove the unzipped files if you don't have enough permissions.

So I think we should probably add a new flag like -disable-unzip to go mod download to disable the unzipping operation.

@FiloSottile

This comment has been minimized.

Copy link
Member

@FiloSottile FiloSottile commented Jun 14, 2019

/cc @bcmills

aofei added a commit to goproxy/goproxy that referenced this issue Jul 29, 2019
@dmitshur

This comment has been minimized.

Copy link
Member

@dmitshur dmitshur commented Nov 5, 2019

There are a few alternative solutions to this problem that can be considered, in addition to a new flag.

  1. Before doing os.RemoveAll(tempDir), you can insert a call to exec.Command("go", "clean", "-modcache"). That'll make it succeed without permission errors.

  2. In Go 1.13, by default, the go command downloads from the module mirror at https://proxy.golang.org. If that behavior is satisfactory for you, you can bypass the go command and download the module zip file from the module mirror directly.

  3. In the x/mod repository, there is a new zip package. /cc @jayconrod It brings us closer to being able to make a stand-alone tool that could create a module zip in a safe way. Such a tool can be better suited for your custom needs than the go command.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.