Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add kernel-install plugin that calls ukify #27262

Merged
merged 20 commits into from
May 6, 2023
Merged

Commits on May 3, 2023

  1. 90-loaderentry: make sure that variables are set

    We unconditionally use the variables later on, so let's make sure
    that they were passed as expected.
    keszybz committed May 3, 2023
    Configuration menu
    Copy the full SHA
    d353316 View commit details
    Browse the repository at this point in the history
  2. ukify: use UPPERCASE for parameter names

    We generally nowadays use UPPERCASE for parameters in variuos help text.
    Let's be consistent here too, and also drop duplicated 'usage:':
    $ ukify -h
    usage: ukify [options…] LINUX INITRD…
           ukify -h | --help
    
    Build and sign Unified Kernel Images
    
    positional arguments:
      LINUX                 vmlinuz file [.linux section]
      INITRD…               initrd files [.initrd section]
    ...
    keszybz committed May 3, 2023
    Configuration menu
    Copy the full SHA
    50f4add View commit details
    Browse the repository at this point in the history
  3. ukify: add missing header

    This file is installed, so it should have the long header.
    keszybz committed May 3, 2023
    Configuration menu
    Copy the full SHA
    d9c8f07 View commit details
    Browse the repository at this point in the history
  4. meson: avoid building executables that won't be installed

    When executable() or custom_target() has install: that is conditional as is
    false (i.e. not install:true), it won't be built by default. (build_by_default:
    defaults to install:). But if that program is added to public_programs, it will
    be build by default because it is pulled in by the test, effectively defeating
    the disablement.
    
    While at it, make 'ukify' follow the same pattern as 'kernel-install'.
    They will be used later together.
    keszybz committed May 3, 2023
    Configuration menu
    Copy the full SHA
    9a01fe3 View commit details
    Browse the repository at this point in the history
  5. meson: allow building .standalone on demand

    We can always build the standalone version whenever we build the normal version
    (the dependencies are the same). In most builds standalone binaries would be
    disabled. But it is occasionally useful to have them for testing, so move the
    conditional to install:, so the binaries can be build by giving the explicit
    target name.
    
    The default of 'build_by_default' for executable() is sadly true (since meson
    0.38.0), so need to specify build_by_default: too.
    
    Also add systemd-shutdown.standalone to public_programs for additional testing.
    keszybz committed May 3, 2023
    Configuration menu
    Copy the full SHA
    b62ee35 View commit details
    Browse the repository at this point in the history
  6. test_ukify: fix loop iteration

    We'd try to access 'linux' or 'initrd' after failing to set it.
    keszybz committed May 3, 2023
    Configuration menu
    Copy the full SHA
    cb3b451 View commit details
    Browse the repository at this point in the history
  7. test_ukify: fix two failing tests

    Fixup for 22ad038 and
    3fc5eed. It seems that the tests are
    not executed properly in CI. Nevertheless, test-ukify appears in logs:
    
    rpm-build:fedora-rawhide-x86_64:
     409/1191 systemd / test-ukify    OK   0.16s
    
    This is strange.
    keszybz committed May 3, 2023
    Configuration menu
    Copy the full SHA
    3f7e77f View commit details
    Browse the repository at this point in the history

Commits on May 5, 2023

  1. ukify: rework option parsing to support a config file

    In some ways this is similar to mkosi: we have a argparse.ArgumentParser()
    with a bunch of options, and a configparser.ConfigParser() with an
    overlapping set of options. Many options are settable in both places, but
    not all. In mkosi, we define this in three places (a dataclass, and a
    function for argparse, and a function for configparser). Here, we have one
    huge list of ConfigItem instances. Each instance specifies the full metadata
    for both parsers. Argparse generates a --help string for all the options,
    and we also append a config file sample to --help based on the ConfigItem
    data:
    
    $ python src/ukify/ukify.py --help|tail -n 25
    config file:
      [UKI]
      Linux = LINUX
      Initrd = INITRD…
      Cmdline = TEXT|@path
      OSRelease = TEXT|@path
      DeviceTree = PATH
      Splash = BMP
      PCRPKey = KEY
      Uname = VERSION
      EFIArch = ia32|x64|arm|aa64|riscv64
      Stub = STUB
      PCRBanks = BANK…
      SigningEngine = ENGINE
      SecureBootPrivateKey = SB_KEY
      SecureBootCertificate = SB_CERT
      SignKernel = SIGN_KERNEL
    
      [PCRSignature:NAME]
      PCRPrivateKey = PATH
      PCRPublicKey = PATH
      Phases = PHASE-PATH…
    
    While writing this I needed to check the argument parsing, so I added
    a --summary switch. It just pretty-prints the resulting option dictionary:
    
    $ python src/ukify/ukify.py /efi//3a9d668b4db749398a4a5e78a03bffa5/6.2.11-300.fc38.x86_64/linux /efi//3a9d668b4db749398a4a5e78a03bffa5/6.2.11-300.fc38.x86_64/initrd --pcr-private-key=PRIV.key --pcr-public-key=PUB.key --config=man/ukify-example.conf --summary
    Host arch 'x86_64', EFI arch 'x64'
    {'_groups': [0, 'initrd', 'system'],
     'cmdline': 'A1 B2 C3',
     'config': 'man/ukify-example.conf',
     'devicetree': None,
     'efi_arch': 'x64',
     'initrd': [PosixPath('initrd1'),
                PosixPath('initrd2'),
                PosixPath('initrd3'),
                PosixPath('/efi/3a9d668b4db749398a4a5e78a03bffa5/6.2.11-300.fc38.x86_64/initrd')],
     'linux': PosixPath('/efi/3a9d668b4db749398a4a5e78a03bffa5/6.2.11-300.fc38.x86_64/linux'),
     'measure': None,
     'os_release': PosixPath('/etc/os-release'),
     'output': 'linux.efi',
     'pcr_banks': ['sha1', 'sha384'],
     'pcr_private_keys': [PosixPath('PRIV.key'),
                          PosixPath('pcr-private-initrd-key.pem'),
                          PosixPath('pcr-private-system-key.pem')],
     'pcr_public_keys': [PosixPath('PUB.key'),
                         PosixPath('pcr-public-initrd-key.pem'),
                         PosixPath('pcr-public-system-key.pem')],
     'pcrpkey': None,
     'phase_path_groups': [None,
                           ['enter-initrd'],
                           ['enter-initrd:leave-initrd',
                            'enter-initrd:leave-initrd:sysinit',
                            'enter-initrd:leave-initrd:sysinit:ready']],
     'sb_cert': PosixPath('mkosi.secure-boot.crt'),
     'sb_key': PosixPath('mkosi.secure-boot.key'),
     'sections': [],
     'sign_kernel': None,
     'signing_engine': None,
     'splash': None,
     'stub': PosixPath('/usr/lib/systemd/boot/efi/linuxx64.efi.stub'),
     'summary': True,
     'tools': None,
     'uname': None}
    
    With --summary, existence of input paths is not checked. I think we'll
    want to show them, instead of throwing an error, but in red, similarly to
    'bootctl list'.
    
    This also fixes tests which were failing with e.g.
    E       FileNotFoundError: [Errno 2] No such file or directory: '/ARG1'
    =========================== short test summary info ============================
    FAILED ../src/ukify/test/test_ukify.py::test_parse_args_minimal - FileNotFoun...
    FAILED ../src/ukify/test/test_ukify.py::test_parse_args_many - FileNotFoundEr...
    FAILED ../src/ukify/test/test_ukify.py::test_parse_sections - FileNotFoundErr...
    =================== 3 failed, 10 passed, 3 skipped in 1.51s ====================
    keszybz committed May 5, 2023
    Configuration menu
    Copy the full SHA
    5143a47 View commit details
    Browse the repository at this point in the history
  2. ukify: PeError → PEError

    We don't lowercase acronyms in systemd usually.
    Remove unnused f'' prefix to avoid a pylint warning.
    keszybz committed May 5, 2023
    Configuration menu
    Copy the full SHA
    7081db2 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    47a6df4 View commit details
    Browse the repository at this point in the history
  4. 60-ukify: kernel-install plugin that calls ukify to create a UKI

    60-ukify.install calls ukify with a config file, so singing and policies and
    splash will be done through the ukify config file, without 60-ukify.install
    knowing anything directly.
    
    In meson.py, the variable for loaderentry.install.in is used just once, let's
    drop it. (I guess this approach was copied from kernel_install_in, which is
    used in another file.)
    
    The general idea is based on cvlc12's systemd#27119, but now in Python instead of
    bash.
    keszybz committed May 5, 2023
    Configuration menu
    Copy the full SHA
    ca1abaa View commit details
    Browse the repository at this point in the history
  5. test-kernel-install: test 60-ukify.install and 90-uki-copy.install

    We install a kernel with layout=uki and uki_generator=ukify, and test
    that a UKI gets installed in the expected place. The two plugins cooperate,
    so it's easiest to test them together.
    keszybz committed May 5, 2023
    Configuration menu
    Copy the full SHA
    f9a6cb0 View commit details
    Browse the repository at this point in the history
  6. test/60-ukify: override stub location in tests

    Without this, build would fail if the stub is not available in /usr/lib/.
    keszybz committed May 5, 2023
    Configuration menu
    Copy the full SHA
    3f80c13 View commit details
    Browse the repository at this point in the history
  7. TODO: remove two entries

    0ccfd35 implemented one of the items, and this
    pull requests handles the other one.
    keszybz committed May 5, 2023
    Configuration menu
    Copy the full SHA
    a4b329e View commit details
    Browse the repository at this point in the history
  8. ukify: appease mypy

    Note to self: PEP 585 introduced using collection types as types,
    and is available since 3.9. PEP 604 allows writing unions with "|",
    but is only available since 3.10, so not yet here because we maintain
    compat with 3.9.
    keszybz committed May 5, 2023
    Configuration menu
    Copy the full SHA
    a758f95 View commit details
    Browse the repository at this point in the history
  9. test_ukify: propagate failure

    Oops. This explains why the tests were "passing" in CI even
    though a direct pytest invocation would fail.
    keszybz committed May 5, 2023
    Configuration menu
    Copy the full SHA
    041f536 View commit details
    Browse the repository at this point in the history
  10. test_ukify: rework how --flakes argument is appended

    The usual approach is to put 'addopts = --flakes' in setup.cfg. Unfortunately
    this fails badly when pytest-flakes is not installed:
      ERROR: usage: test_ukify.py [options] [file_or_dir] [file_or_dir] [...]
      test_ukify.py: error: unrecognized arguments: --flakes
    
    pytest-flakes is not packaged everywhere, and this test is not very important,
    so let's just do it only if pytest-flakes is available. We now detect if
    pytest-flakes is available and only add '--flakes' conditionally. This
    unfortunately means that when invoked via 'pytest' or directly as
    'src/ukify/test/test_ukify.py', '--flakes' will not be appended automatically.
    But I don't see a nice way to achieve previous automatic behaviour.
    
    (I first considered making 'setup.cfg' templated. But then it is created
    in the build directory, but we would need it in the source directory for
    pytest to load it automatically. So to load the file, we'd need to give an
    argument to pytest anyway, so we don't gain anything with this more complex
    approach.)
    keszybz committed May 5, 2023
    Configuration menu
    Copy the full SHA
    55be961 View commit details
    Browse the repository at this point in the history
  11. ci: install pytest-flakes

    Some web searches say that it's packaged for those distros and not the others…
    
    v2:
    - drop arch. https://aur.archlinux.org/packages/python-pytest-flakes exists,
      but installation fails in CI.
    keszybz committed May 5, 2023
    Configuration menu
    Copy the full SHA
    bac1882 View commit details
    Browse the repository at this point in the history
  12. man: describe all the changes to ukify

    As in mkosi(1), let's describe the config file and commandline options
    together. This is nice for us, because we don't need to duplicate descriptions
    and we're less likely to forget to update one place or the other. This is also
    nice for users, because they can easily figure out what can be configured
    where.
    
    The options are now ordered by config file section.
    
    --summary was not described before.
    
    More examples are added.
    keszybz committed May 5, 2023
    Configuration menu
    Copy the full SHA
    248be6e View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    46886f1 View commit details
    Browse the repository at this point in the history