Build linuxkit/grub off of master

[deitch]

- What I did

• Separate the building of grub into its own modular element, as opposed to in each of the 3 tools that require it - mkimage-iso-efi, mkimage-raw-efi, mkimage-qcow2-efi . We do not use grub for bios.
• Replace dependency on the largely abandoned coreos/grub fork with the maintained upstream grub. In the last several months to year, the same team that did much of the work on coreos/grub has upstreamed the important parts.

Note that this is compiling directly from master (a specific commit). We are not using the official release - which is available in apk - since that is based on the most recent release, 2.02, which is almost 2 years old.

As discussed with @rn and @justincormack .

Question: would like some additional testing, even if only smoke-testing style, "can we build it and get the image we want" (final OCI image has a single file in it). Any thoughts on it?

- How I did it

Build tools/linuxkit/grub

- How to verify it

cd tools/grub
docker build -t linuxkit/grub .

Or more properly:

lkt pkg build .

- Description for the changelog

Extract grub build to its own module.

- A picture of a cute animal (not mandatory but encouraged)

[GordonTheTurtle]

Please sign your commits following these rules:
https://github.com/moby/moby/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:

$ git clone -b "grub" git@github.com:deitch/linuxkit.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

Amending updates the existing PR. You DO NOT need to open a new one.

[deitch]

Note: this is still based off of alpine:3.8. Once this is in, we can put in the effort to switch to 3.9. One improvement at a time.

[deitch]

Will push out packages for x86_64 and arm64 once someone else has signed off.

[justincormack]

This doesn't replace the old grub builds is the intention to do that later?

[deitch]

This doesn't replace the old grub builds is the intention to do that later?

Yes, several PRs:

1. Get this separated and working
2. Get it to work on alpine:3.9 base
3. Replace the direct builds on all of the dependent packages

[deitch]

@rn and @justincormack , can we get this moved ahead please?

[rn]

Added a few comments. This looks good, but the comments should get addressed and we also need to have the commit for the mkimage-* images to use this before committing because otherwise CI won't test it

[deitch]

So it seems, I cannot sign it due to something with trust. Push worked, just not signing.

Is it that we need a new hub repo set up correctly with trust for linuxkit/grub?

$ DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE=$(...) lkt pkg push tools/grub
Building and pushing "linuxkit/grub:9aaae2ad8ab52206ebb5c014d27bffeddee8c9ff"
Error: remote trust data does not exist for docker.io/linuxkit/grub: notary.docker.io does not have trust data for docker.io/linuxkit/grub
No image pulled, continuing with build
Sending build context to Docker daemon  4.608kB
...
Successfully built 2d2a3e3ab9e7
Successfully tagged linuxkit/grub:9aaae2ad8ab52206ebb5c014d27bffeddee8c9ff-amd64
Pushing linuxkit/grub:9aaae2ad8ab52206ebb5c014d27bffeddee8c9ff-amd64
The push refers to repository [docker.io/linuxkit/grub]
d7ea740c3251: Pushed
9aaae2ad8ab52206ebb5c014d27bffeddee8c9ff-amd64: digest: sha256:566954b876e5c4043b5759a05c63956a3de00831559a880ae8bd00a0da12915a size: 527
Signing and pushing trust metadata
Enter passphrase for root key with ID 7780670:
password invalid, operation has failed.
exit status 1

[deitch]

OK, everything resolved except:

1. Cannot sign, issue listed above
2. Did not update the downstream dependencies (mkimage-*) due to previous step.

If someone can figure out the signing issue and help?

[justincormack]

Yes, it probably needs initialising. The instructions and keys are somewhere around that you have access to...

[deitch]

Update: I managed to get it to build correctly off of upstream grub, and build the various mkimage-*-efi images. Still running into 2 problems:

1. Refuses to boot (or boots but has no output).
   • With lkt run qemu, I get the usual qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.80000001H:ECX.svm [bit 2] but nothing after that, even as qemu appears to be running contentedly.
   • With virtual box, I get error: invalid video mode specification: text. booting in blind mode and no output after that. I am sure I am doing something wrong....
2. While the tpm support definitely has been upstreamed into grub (at least into master, not the 2-yr-old 2.02 release), it is as-yet-unclear if the full EFI support has been upstreamed. Most distributions have some form of linuxefi/initrdefi custom module (including coreos/grub). I am trying to determine if that was upstreamed.

[deitch]

And thanks to @rn for initialized the repo.

[deitch]

mjg confirmed that linuxefi module was not upstreamed (yet) into mainline grub (see second issue I raised above). I will be looking into the impact of that, still tbd.

In the meantime, any help people can offer on the first issue, kindly do.

[deitch]

The impact of losing linuxefi is limited to loss of the shim. In other words, if you try to use a system that has SecureBoot enabled, it will fail. All other cases will boot, and tpm support is fully in.

I am willing to live with that.

The only remaining issue, then, is my system with no video output. I do not yet know if this is something due to the changes in grub, or something else entirely. Can @rn or @justincormack weigh in here? Other than that, I think this is ready to go.

[deitch]

I stand corrected. I had a slightly longer conversation with Matthew. It appears that linux command uses the legacy entrypoint into the kernel, which misses certain critical elements (tcg2 logs, memory overwrite protection, some other things which Matthew can run rings around me on), while linuxefi uses the EFI entrypoint into the kernel, enabling the above (in addition to SecureBoot shim support).

Our choices basically are:

• Stick with the orphaned coreos grub until the EFI entrypoint is upstreamed.
• Move to master and live with the lacking functionality.

Thoughts?

[rn]

My suggestion would be stick with the orphaned coreOS grub for the really soon now release and then move to master after.

[deitch]

In that case, I am going to do the following:

1. Take the "centralization of grub" from here, but rather than going to grub upstream, go to the orphaned CoreOS grub.
2. Leave this PR open (or open a new one) that switches to upstream, and leave that one open until we get linuxefi module support in there.

Once that is in place, I would appreciate some assistance in figuring out the empty boot screen issue.

[deitch]

Replaced by #3505