Breaking change: u-root works exactly when go build or go list also work.

[hugelgupf]

Important

TL;DR: You won't be able to run u-root from any arbitrary directory after this change (when using GO111MODULE=on|auto). u-root must be run somewhere with a Go module or Go workspace.

Go workspaces replace this use case, and there is a tool to create Go workspaces on the fly called goanywhere. Another usable method is described by the mkuimage README.

This change pulls in u-root/gobusybox#110, in which we stop supporting module builds without a go.mod. In essence, from this point forward, the u-root tool will support builds exactly when go build and go list also work. u-root/gobusybox#110 reduces a lot of maintenance burden, trivially starts supporting Go workspaces and Go module replace directives, and increases gobusybox code's readability.

Multi-module builds can be done easily with the 2 standard Go methods, as described in the mkuimage README. Go workspaces, vendored Go workspaces, Go modules, and vendored Go modules are all supported. Completely offline builds can easily done with vendored Go modules or workspaces.

For ease of use, the goanywhere tool was created. goanywhere creates a temporary directory with a Go workspace of the given commands, and then execs a binary passing along the Go command paths. Use like goanywhere ./u-root/cmds/core/{init,gosh} ./cpu/cmds/cpud -- u-root [other u-root args]. This approaches the easy usability of u-root up to this point.

goanywhere does not work with templates. goanywhere only accepts file system paths at this time.

This also fixes the issue where binary mode builds didn't always work in the same cases where gbb builds worked. Both are now always supported in the exact same circumstances.

This change also pulls in u-root/mkuimage#28, in which we support generic template YAML files named ".mkuimage.yaml" in the current working directory or any of its parents. Templates support the existing command expansions, but also support configs (invoked with u-root -config=$config) in which one can specify build configuration and mixed-builder (bb/binary) builds. They will be documented in a README at a later time.

Also check out the new README.

Closes #2814
Closes #2603
Closes #2450
Closes #2200
Closes #2198
Closes #2149
Closes #1863
Closes #1851
Closes #1024
Closes #985

[codecov]

Codecov Report

Attention: Patch coverage is 90.90909% with 3 lines in your changes are missing coverage. Please review.

Project coverage is 78.30%. Comparing base (e2dc8b2) to head (ce88b10).

Files	Patch %	Lines
u-root.go	90.90%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##           main    #2923       +/-   ##
=========================================
+ Coverage      0   78.30%   +78.30%     
=========================================
  Files         0      416      +416     
  Lines         0    42409    +42409     
=========================================
+ Hits          0    33207    +33207     
- Misses        0     9202     +9202     

Flag	Coverage Δ
.-amd64	90.69% <90.90%> (?)
cmds/...-amd64	67.64% <ø> (?)
integration/generic-tests/...-amd64	20.32% <ø> (?)
integration/generic-tests/...-arm	11.63% <ø> (?)
integration/generic-tests/...-arm64	23.59% <ø> (?)
integration/gotests/...-amd64	73.93% <ø> (?)
integration/gotests/...-arm	75.30% <ø> (?)
integration/gotests/...-arm64	75.13% <ø> (?)
pkg/...-amd64	76.65% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[rminnich]

nice

[hugelgupf]

I'll set up some more tools in the other repo before I really merge this -- I'd like e.g. an easy way to test compilation of every template.

[hugelgupf]

PTAL

[MDr164]

Really nice! It's fairly large so looking over everything might take time but so far this looks very promising. The gist is to strip out the image building and rely on mkuimage and gobusybox alone, right? Seems reasonable to me.

[hugelgupf]

Really nice! It's fairly large so looking over everything might take time but so far this looks very promising. The gist is to strip out the image building and rely on mkuimage and gobusybox alone, right? Seems reasonable to me.

That is a side-effect here. The behavior changes I described in the first comment (which are pulled in through that) are important to take a look at.

[rminnich]

this makes a lot of sense. the nature of u-root is utterly changed from 12y ago, as is the nature of Go, and this seems like a long overdue reworking of it all.