Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
cmd/compile, cmd/link: support arbitrary import file map configurations #20579
Because of multiple versions of packages being used for different builds, proper support for package management in the go command is very likely going to need better support for caching build artifacts (long-standing issue #4719). I have a prototype of this caching that fixes #4719 but also various more specific bugs, like #11193, #14271, #15752, #15799, #18981, #19340, #20137, #20512 and almost certainly others I've missed.
But it's a big change. I'd like to publish an experimental go command (named xgo) that people can download and try, but I want xgo to use an unmodified Go 1.9 release, so that people trying it don't have to opt in to the instability of Go tip toolchain.
The toolchain support required by the caching prototype is fairly straightforward. The compiler and linker both need to support loading a definition of how to resolve imports instead of using the default file-name-construction lookup. For vendoring we added half of what the compiler needs, namely -importmap, which maps from import path to "fully-qualified import path" (the path containing /vendor/). The other half is a map from fully-qualified import path to exact package file to open for that import. The compiler needs this for imports it will see, and the linker needs this for all imports in the program.
The -importmap flag works for a few imports but doesn't scale well, since it must be repeated for every import, and some systems place limits on command-line size. Instead, I propose to add an -importcfg flag to both the compiler and linker that takes a single file name argument. The named file contains a sequence of lines like
There is one more change required in the compiler: it assumes that it can use the file name extension to decide whether to expect a package archive or a Go object. A trivial change lets it look at the first line of the file to make that decision (it was already reading the first line anyway) instead of using the file name.
Because I have a working prototype of proper build caching using this functionality, I'm confident these two changes are sufficient to enable the "experiment using an extra binary and the standard Go 1.9 toolchain" approach.
The -importcfg change changes very little of the code and is implemented such that it clearly has no effect if the flag is not used. As such, it should be safe to add this in Go 1.9 even now just before the beta.
Allows reading -importmap options from a file instead of putting them all on the command line, and adds the ability to specify the file location of specific packages. In effect, -importcfg is a generalization of and supersedes -importmap, -importsuffix, and -I. Of course, those flags will continue to be supported, for compatibility with other tools. Having this flag in Go 1.9 will let us try some experiments involving package management without needing guinea pigs to build a custom Go toolchain. This flag also helps with #14271 at some later point. For #20579. Change-Id: If005dbc2b01d8fd16cbfd3687dfbe82499f4bc56 Reviewed-on: https://go-review.googlesource.com/44850 Run-TryBot: Russ Cox <firstname.lastname@example.org> Reviewed-by: Ian Lance Taylor <email@example.com>
Adds the ability to specify the file location of each imported package, like in the -importcfg added to cmd/compile in a related CL. In effect, -importcfg is a generalization of and supersedes -installsuffix and -L. Of course, those flags will continue to be supported, for compatibility with other tools. Having this flag in Go 1.9 will let us try some experiments involving package management without needing guinea pigs to build a custom Go toolchain. This flag also helps with #14271 at some later point. For #20579. Change-Id: Ie4c171bcd3aa2faa446ac340e36516f2f9853882 Reviewed-on: https://go-review.googlesource.com/44851 Run-TryBot: Russ Cox <firstname.lastname@example.org> Reviewed-by: Ian Lance Taylor <email@example.com>