Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build linuxkit/grub off of master #3309

Open
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
4 participants
@deitch
Copy link
Collaborator

deitch commented Mar 18, 2019

- What I did

  • Separate the building of grub into its own modular element, as opposed to in each of the 3 tools that require it - mkimage-iso-efi, mkimage-raw-efi, mkimage-qcow2-efi . We do not use grub for bios.
  • Replace dependency on the largely abandoned coreos/grub fork with the maintained upstream grub. In the last several months to year, the same team that did much of the work on coreos/grub has upstreamed the important parts.

Note that this is compiling directly from master (a specific commit). We are not using the official release - which is available in apk - since that is based on the most recent release, 2.02, which is almost 2 years old.

As discussed with @rn and @justincormack .

Question: would like some additional testing, even if only smoke-testing style, "can we build it and get the image we want" (final OCI image has a single file in it). Any thoughts on it?

- How I did it

Build tools/linuxkit/grub

- How to verify it

cd tools/grub
docker build -t linuxkit/grub .

Or more properly:

lkt pkg build .

- Description for the changelog

Extract grub build to its own module.

- A picture of a cute animal (not mandatory but encouraged)

sulphur-crested cockatoo

@GordonTheTurtle

This comment has been minimized.

Copy link
Collaborator

GordonTheTurtle commented Mar 18, 2019

Please sign your commits following these rules:
https://github.com/moby/moby/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:

$ git clone -b "grub" git@github.com:deitch/linuxkit.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

Amending updates the existing PR. You DO NOT need to open a new one.

@deitch

This comment has been minimized.

Copy link
Collaborator Author

deitch commented Mar 18, 2019

Note: this is still based off of alpine:3.8. Once this is in, we can put in the effort to switch to 3.9. One improvement at a time.

@deitch deitch force-pushed the deitch:grub branch from 82c9593 to 595e18e Mar 18, 2019

@GordonTheTurtle GordonTheTurtle removed the dco/no label Mar 18, 2019

@deitch deitch force-pushed the deitch:grub branch from 595e18e to 8b508fd Mar 18, 2019

@deitch

This comment has been minimized.

Copy link
Collaborator Author

deitch commented Mar 18, 2019

Will push out packages for x86_64 and arm64 once someone else has signed off.

Show resolved Hide resolved tools/grub/Dockerfile Outdated
Show resolved Hide resolved tools/grub/Dockerfile Outdated
@justincormack

This comment has been minimized.

Copy link
Collaborator

justincormack commented Mar 18, 2019

This doesn't replace the old grub builds is the intention to do that later?

@deitch deitch force-pushed the deitch:grub branch from 8b508fd to 113222c Mar 18, 2019

@deitch

This comment has been minimized.

Copy link
Collaborator Author

deitch commented Mar 18, 2019

This doesn't replace the old grub builds is the intention to do that later?

Yes, several PRs:

  1. Get this separated and working
  2. Get it to work on alpine:3.9 base
  3. Replace the direct builds on all of the dependent packages
@deitch

This comment has been minimized.

Copy link
Collaborator Author

deitch commented Mar 21, 2019

@rn and @justincormack , can we get this moved ahead please?

@rn
Copy link
Member

rn left a comment

Added a few comments. This looks good, but the comments should get addressed and we also need to have the commit for the mkimage-* images to use this before committing because otherwise CI won't test it

Show resolved Hide resolved tools/grub/Dockerfile Outdated
Show resolved Hide resolved tools/grub/Dockerfile Outdated
Show resolved Hide resolved tools/grub/build.yml

@deitch deitch force-pushed the deitch:grub branch from 113222c to e510327 Mar 22, 2019

@deitch

This comment has been minimized.

Copy link
Collaborator Author

deitch commented Mar 22, 2019

So it seems, I cannot sign it due to something with trust. Push worked, just not signing.

Is it that we need a new hub repo set up correctly with trust for linuxkit/grub?

$ DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE=$(...) lkt pkg push tools/grub
Building and pushing "linuxkit/grub:9aaae2ad8ab52206ebb5c014d27bffeddee8c9ff"
Error: remote trust data does not exist for docker.io/linuxkit/grub: notary.docker.io does not have trust data for docker.io/linuxkit/grub
No image pulled, continuing with build
Sending build context to Docker daemon  4.608kB
...
Successfully built 2d2a3e3ab9e7
Successfully tagged linuxkit/grub:9aaae2ad8ab52206ebb5c014d27bffeddee8c9ff-amd64
Pushing linuxkit/grub:9aaae2ad8ab52206ebb5c014d27bffeddee8c9ff-amd64
The push refers to repository [docker.io/linuxkit/grub]
d7ea740c3251: Pushed
9aaae2ad8ab52206ebb5c014d27bffeddee8c9ff-amd64: digest: sha256:566954b876e5c4043b5759a05c63956a3de00831559a880ae8bd00a0da12915a size: 527
Signing and pushing trust metadata
Enter passphrase for root key with ID 7780670:
password invalid, operation has failed.
exit status 1

@deitch deitch force-pushed the deitch:grub branch from e510327 to a6cfca0 Mar 22, 2019

@deitch

This comment has been minimized.

Copy link
Collaborator Author

deitch commented Mar 22, 2019

OK, everything resolved except:

  1. Cannot sign, issue listed above
  2. Did not update the downstream dependencies (mkimage-*) due to previous step.

If someone can figure out the signing issue and help?

@justincormack

This comment has been minimized.

Copy link
Collaborator

justincormack commented Mar 22, 2019

Yes, it probably needs initialising. The instructions and keys are somewhere around that you have access to...

@deitch deitch force-pushed the deitch:grub branch from a6cfca0 to 46bae07 Mar 24, 2019

Build linuxkit/grub from upstream master, and consume in images
Signed-off-by: Avi Deitcher <avi@deitcher.net>

@deitch deitch force-pushed the deitch:grub branch from 46bae07 to 38498ac Mar 24, 2019

@deitch

This comment has been minimized.

Copy link
Collaborator Author

deitch commented Mar 24, 2019

Update: I managed to get it to build correctly off of upstream grub, and build the various mkimage-*-efi images. Still running into 2 problems:

  1. Refuses to boot (or boots but has no output).
    • With lkt run qemu, I get the usual qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.80000001H:ECX.svm [bit 2] but nothing after that, even as qemu appears to be running contentedly.
    • With virtual box, I get error: invalid video mode specification: text. booting in blind mode and no output after that. I am sure I am doing something wrong....
  2. While the tpm support definitely has been upstreamed into grub (at least into master, not the 2-yr-old 2.02 release), it is as-yet-unclear if the full EFI support has been upstreamed. Most distributions have some form of linuxefi/initrdefi custom module (including coreos/grub). I am trying to determine if that was upstreamed.
@deitch

This comment has been minimized.

Copy link
Collaborator Author

deitch commented Mar 24, 2019

And thanks to @rn for initialized the repo.

@deitch

This comment has been minimized.

Copy link
Collaborator Author

deitch commented Mar 26, 2019

mjg confirmed that linuxefi module was not upstreamed (yet) into mainline grub (see second issue I raised above). I will be looking into the impact of that, still tbd.

In the meantime, any help people can offer on the first issue, kindly do.

@deitch

This comment has been minimized.

Copy link
Collaborator Author

deitch commented Apr 9, 2019

The impact of losing linuxefi is limited to loss of the shim. In other words, if you try to use a system that has SecureBoot enabled, it will fail. All other cases will boot, and tpm support is fully in.

I am willing to live with that.

The only remaining issue, then, is my system with no video output. I do not yet know if this is something due to the changes in grub, or something else entirely. Can @rn or @justincormack weigh in here? Other than that, I think this is ready to go.

@deitch

This comment has been minimized.

Copy link
Collaborator Author

deitch commented Apr 9, 2019

I stand corrected. I had a slightly longer conversation with Matthew. It appears that linux command uses the legacy entrypoint into the kernel, which misses certain critical elements (tcg2 logs, memory overwrite protection, some other things which Matthew can run rings around me on), while linuxefi uses the EFI entrypoint into the kernel, enabling the above (in addition to SecureBoot shim support).

Our choices basically are:

  • Stick with the orphaned coreos grub until the EFI entrypoint is upstreamed.
  • Move to master and live with the lacking functionality.

Thoughts?

@rn

This comment has been minimized.

Copy link
Member

rn commented Apr 13, 2019

My suggestion would be stick with the orphaned coreOS grub for the really soon now release and then move to master after.

@deitch

This comment has been minimized.

Copy link
Collaborator Author

deitch commented Apr 15, 2019

In that case, I am going to do the following:

  1. Take the "centralization of grub" from here, but rather than going to grub upstream, go to the orphaned CoreOS grub.
  2. Leave this PR open (or open a new one) that switches to upstream, and leave that one open until we get linuxefi module support in there.

Once that is in place, I would appreciate some assistance in figuring out the empty boot screen issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.