Introduce kernel-64k

[rlmenge]

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

• The toolchain has been rebuilt successfully (or no changes were made to it)
• The toolchain/worker package manifests are up-to-date
• Any updated packages successfully build (or no packages were changed)
• Packages depending on static components modified in this PR (Golang, *-static subpackages, etc.) have had their Release tag incremented.
• Package tests (%check section) have been verified with RUN_CHECK=y for existing SPEC files, or added to new SPEC files
• All package sources are available
• cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json, .github/workflows/cgmanifest.json)
• LICENSE-MAP files are up-to-date (./LICENSES-AND-NOTICES/SPECS/data/licenses.json, ./LICENSES-AND-NOTICES/SPECS/LICENSES-MAP.md, ./LICENSES-AND-NOTICES/SPECS/LICENSE-EXCEPTIONS.PHOTON)
• All source files have up-to-date hashes in the *.signatures.json files
• sudo make go-tidy-all and sudo make go-test-coverage pass
• Documentation has been updated to match any changes to the build system
• Ready to merge

---

Summary

Note: when reviewing this PR it may be easier to look at the individual commits to see what differences are present with the generic kernel.

kernel-64k is a new aarch64 kernel which has 64k page sizes.

kernel-64k contains a config_aarch64 which differs from the kernel in that it sets CONFIG_ARM64_64K_PAGES (Kconfig). This offering is to help with HPC scenarios.

The kernel package will still be offered and will retain the default 4k page size.

Associated PRs:
https://dev.azure.com/mariner-org/mariner/_git/CBL-Mariner-Pipelines/pullrequest/21015
https://dev.azure.com/mariner-org/mariner/_git/CBL-Mariner-Pipelines/pullrequest/21017

Change Log

• Introduce new aarch64 package kernel-64k

Does this affect the toolchain?

NO

Associated issues

• https://microsoft.visualstudio.com/OS/_workitems/edit/54880630

Build Test Methodology

• Buddy Build https://dev.azure.com/mariner-org/mariner/_build/results?buildId=681352&view=results
• Buddy Build with reverted commit blocking builds: https://dev.azure.com/mariner-org/mariner/_build/results?buildId=681457&view=results

Test Methodology

Azure VM

1. confirm that default page size is 4k
   
2. reboot using new kernel
   

$ systemd-analyze
Startup finished in 1.704s (kernel) + 1.204s (initrd) + 4.941s (userspace) = 7.851s 
graphical.target reached after 4.706s in userspace.

Baremetal

[rlmenge]

kernel-64k should also be added to the tdnf.conf. Holding off to see if #11055 goes in.

kernel-64k will also need an updated .pem similar to #10961 if that PR is approved

[reubeno]

If we are to build another variant of the kernel that is expected to move forward in lock-step with the standard one, what's our plan for how to keep them from diverging?

(FYI, for a previous project we managed to build multiple variants of the kernel without needing to duplicate the spec, loose source files, and full config file. It entailed having multiple kernel .spec files that lived in the same directory, but which primarily just set a few macros and %included a common .inc file that contained the real guts of the spec. We also used a simple scheme for taking a base defconfig and merging targeted changes onto it for variants. Would be happy to discuss more / provide pointers if helpful. It's also possible that approach had gotchas we were lucky not to trip over.)

[rlmenge]

If we are to build another variant of the kernel that is expected to move forward in lock-step with the standard one, what's our plan for how to keep them from diverging?

I considered using entangled specs check; however it only ensures our versions are aligned and won't actually check if the specs or configs are aligned. Upon reflection, that may be enough to make sure we are at least checking we need to update these files. I have made the change :)

(FYI, for a previous project we managed to build multiple variants of the kernel without needing to duplicate the spec, loose source files, and full config file. It entailed having multiple kernel .spec files that lived in the same directory, but which primarily just set a few macros and %included a common .inc file that contained the real guts of the spec. We also used a simple scheme for taking a base defconfig and merging targeted changes onto it for variants. Would be happy to discuss more / provide pointers if helpful. It's also possible that approach had gotchas we were lucky not to trip over.)

I think having a uniform base spec could be useful! It would require changing our current kernel.spec which might make more sense for a separate PR? As for the defconfigs, we have historically not used these as we provide an explict config so that we know exactly what is being built instead of only seeing the final config after the build since defconfigs allow the kernel to change kernel configs under the hood. If having an explicit config is no longer a requirement we can definitely reduce the overhead using defconfigs!

I considered keeping kernel-64k in the kernel folder, as it would prevent having to apply patches multiple times, service files, and key files. We still would have a separate kernel-64k.spec, config, and signature files. I decided against moving the kernel-64k.spec into the kernel folder because while kernel-uki.spec is in the kernel folder, kernel-uki relies on the kernel's vmlinuz to build and kernel-64k is technically a separate kernel (think if a CVE pops up in 64k page sizes).

[reubeno]

I decided against moving the kernel-64k.spec into the kernel folder because while kernel-uki.spec is in the kernel folder, kernel-uki relies on the kernel's vmlinuz to build and kernel-64k is technically a separate kernel (think if a CVE pops up in 64k page sizes).

It sounds like that's a good reason for kernel-uki to be in the kernel directory, but I'm not sure I understand what down-side there is for kernel-64k.spec being there too.

Regarding defconfig, I think I used the wrong term. What I meant was that it seems the config and config_aarch64 should be shared between kernel.spec and kernel-64k.spec; we just need to replace/append the divergent kconfig options, right?

Agreed that we could use some of the existing checks we have to catch some divergence, but better than low-fidelity checks to verify alignment, it would be great to see if we can guarantee alignment through how we construct the specs.

[rlmenge]

It sounds like that's a good reason for kernel-uki to be in the kernel directory, but I'm not sure I understand what down-side there is for kernel-64k.spec being there too.

No downside, just preference. Today, kernels which produce their own vmlinuz are put in their own SPEC folder. For 3.0 that includes kernel-uvm, kernel-mshv, and in extended kernel-rt. In 2.0, that is kernel-mos, kernel-hci, kernel-azure, and those in 3.0. I created a workitem to investigate if we can reduce the duplication since I do think this is a worthwhile effort: https://microsoft.visualstudio.com/OS/_workitems/edit/55119067

Regarding defconfig, I think I used the wrong term. What I meant was that it seems the config and config_aarch64 should be shared between kernel.spec and kernel-64k.spec; we just need to replace/append the divergent kconfig options, right?

So typically what you are describing is done through defconfigs. However, including an explicit patch for the specific 64k changes would be tricky as the kernel configs themselves do change quite frequently even in stable releases. Four out of the last six updates for 6.6 included config implementation changes (for example #10767 (comment)). If the configs for the 64k pagesizes changes for instance, we should be ok because the kernel.spec has a check to ensure we are at least building a functional kernel but the patch itself may just need to be updated a lot. I would want to know how feasible this is with the automation we have today.

Agreed that we could use some of the existing checks we have to catch some divergence, but better than low-fidelity checks to verify alignment, it would be great to see if we can guarantee alignment through how we construct the specs.

100%, having it built into the way the spec is built would be ideal

[reubeno]

To one of your earlier points, I do think it's reasonable to handle this all in separate changes.

However, including an explicit patch for the specific 64k changes would be tricky as the kernel configs themselves do change quite frequently even in stable releases.

Patches aren't the only way to merge content. Particularly given that the order of lines in a kconfig file shouldn't really make a difference (right?), couldn't we do a simple concatenation of a "base" config file and a kernel variant-specific "append" file?

Today, kernels which produce their own vmlinuz are put in their own SPEC folder.

Yes, and I'm challenging that 🙂 For what it's worth, I'm happy to chat offline; we have some proof points from one other project that this can be feasible, but I don't understand whether there are custom (post-?)build steps we perform for the kernel that would throw a wrench in the works of that approach.

[eric-desrochers]

kernel-64k should also be added to the tdnf.conf. Holding off to see if #11055 goes in.

kernel-64k will also need an updated .pem similar to #10961 if that PR is approved

[tdnf] -> #11151

[rlmenge]

Rebased and added patch.

Confirmed builds: https://dev.azure.com/mariner-org/mariner/_build/results?buildId=686970&view=results

And boots in ARM vm: