Skip to content
This repository has been archived by the owner on May 12, 2021. It is now read-only.

Create Linux kernel configuration script #8

Closed
jodh-intel opened this issue Feb 20, 2018 · 7 comments
Closed

Create Linux kernel configuration script #8

jodh-intel opened this issue Feb 20, 2018 · 7 comments
Assignees
Labels
stale Issue or PR was not updated in a timely fashion

Comments

@jodh-intel
Copy link
Contributor

As @sameo noted in kata-containers/linux#5 (comment), we should have a script that can generate a valid kernel configuration file.

The script will generate a valid .config kernel configuration file to be used by the kernel's build system.

However, the script will allow individual options and groups of options to be documented as the kernel's config file does not support comments (well, it does, but only for "commented out" / disabled CONFIG_ options).

Each group of config options in the script should document:

  • Why it has been enabled.
  • If there is any known impact (positive or negative) on boot speed / general performance.
  • If there is any known impact (positive or negative) on memory density.

See also: #7.

@jcvenegas
Copy link
Member

@jodh-intel do you think is needed to do using Kconfig tools? Or you think is better an script that takes an exithing config and checks for the minimal set of configs ?

@jodh-intel
Copy link
Contributor Author

I was imagining a script that would do something similar to https://github.com/kata-containers/packaging/blob/master/scripts/configure-hypervisor.sh. The script would read a YAML config file and would create a (partial) .config file. You could then run make olddefconfig so that Kconfig would add in any missing CONFIG_* options and (hopefully?) generate a usable kernel.

The config file would be something like the following (in pseudo-YAML):

config:
    CONFIG_FOO:
      type: [module|built-in|disabled]
      reasons: [boot-speed|memory-density|security]
      architectures: all
      notes: free-form multi-line text to describe the option goes here.

  CONFIG_BAR:
      ...

  CONFIG_BAZ:
      ...

The config file would not list all config options - it would only list those that we have to set or explicitly disable. However, I don't know whether such a plan would even work.

Any thoughts @grahamwhaley, @sboeuf, @bergwolf?

@sboeuf
Copy link

sboeuf commented Jul 24, 2018

@jodh-intel I think it works technically as long as you apply the defaults for the rest of the config. But this partial .config will need to be explicit enough to get a kernel that boots eventually.
It sounds interesting to me, because this would help users of Kata to get their own kernel with the minimal config expected by Kata. This way they don't need to do the translation of what is in our specific .config for our current vmlinuz.

@jodh-intel
Copy link
Contributor Author

Related: #56.

bergwolf added a commit that referenced this issue Jan 3, 2019
Reverts #269

The added kernel patch doesn't apply and is causing all CI to fail. FYI, the current guest kernel version is 4.14.67. Please rebase and re-submit your PR cleanly @Pennyzct.

e.g., http://jenkins.katacontainers.io/job/kata-containers-runtime-centos-7-4-PR/995/console

NFO: Applying patch /tmp/jenkins/workspace/kata-containers-runtime-centos-7-4-PR/go/src/github.com/kata-containers/packaging/kernel/patches/0003-backport-Arm64-KVM-Dynamic-IPA-and-52bit-IPA-support.patch
patching file arch/arm/include/asm/kvm_arm.h
patching file arch/arm/include/asm/kvm_host.h
Hunk #1 succeeded at 266 (offset -7 lines).
Hunk #2 succeeded at 314 with fuzz 2 (offset -40 lines).
patching file arch/arm/include/asm/kvm_mmu.h
Hunk #1 FAILED at 35.
Hunk #2 succeeded at 47 with fuzz 2 (offset -5 lines).
Hunk #3 succeeded at 266 with fuzz 2 (offset -96 lines).
1 out of 3 hunks FAILED -- saving rejects to file arch/arm/include/asm/kvm_mmu.h.rej
patching file arch/arm/include/asm/stage2_pgtable.h
patching file arch/arm64/include/asm/cpufeature.h
Hunk #1 FAILED at 262.
Hunk #2 succeeded at 284 (offset -246 lines).
1 out of 2 hunks FAILED -- saving rejects to file arch/arm64/include/asm/cpufeature.h.rej
patching file arch/arm64/include/asm/kvm_arm.h
Hunk #1 succeeded at 102 (offset -5 lines).
Hunk #2 succeeded at 116 (offset -5 lines).
Hunk #3 succeeded at 305 (offset -7 lines).
patching file arch/arm64/include/asm/kvm_asm.h
Hunk #1 succeeded at 27 (offset -3 lines).
Hunk #2 FAILED at 73.
1 out of 2 hunks FAILED -- saving rejects to file arch/arm64/include/asm/kvm_asm.h.rej
patching file arch/arm64/include/asm/kvm_host.h
Hunk #1 succeeded at 48 (offset -5 lines).
Hunk #2 FAILED at 61.
Hunk #3 succeeded at 399 with fuzz 2 (offset -41 lines).
Hunk #4 FAILED at 503.
2 out of 4 hunks FAILED -- saving rejects to file arch/arm64/include/asm/kvm_host.h.rej
patching file arch/arm64/include/asm/kvm_hyp.h
patching file arch/arm64/include/asm/kvm_mmu.h
Hunk #1 succeeded at 154 (offset 13 lines).
Hunk #2 succeeded at 229 with fuzz 1 (offset -17 lines).
Hunk #3 succeeded at 409 with fuzz 2 (offset -110 lines).
patching file arch/arm64/include/asm/stage2_pgtable-nopmd.h
patching file arch/arm64/include/asm/stage2_pgtable-nopud.h
patching file arch/arm64/include/asm/stage2_pgtable.h
patching file arch/arm64/kvm/guest.c
Hunk #1 succeeded at 305 (offset -86 lines).
patching file arch/arm64/kvm/hyp-init.S
Hunk #1 FAILED at 65.
1 out of 1 hunk FAILED -- saving rejects to file arch/arm64/kvm/hyp-init.S.rej
patching file arch/arm64/kvm/hyp/Makefile
Hunk #1 succeeded at 18 (offset -1 lines).
patching file arch/arm64/kvm/hyp/s2-setup.c
Reversed (or previously applied) patch detected! Assume -R? [n]
Apply anyway? [n]
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file arch/arm64/kvm/hyp/s2-setup.c.rej
patching file arch/arm64/kvm/hyp/switch.c
Hunk #1 FAILED at 198.
Hunk #2 succeeded at 226 (offset -37 lines).
1 out of 2 hunks FAILED -- saving rejects to file arch/arm64/kvm/hyp/switch.c.rej
patching file arch/arm64/kvm/hyp/tlb.c
Hunk #1 succeeded at 29 (offset -1 lines).
Hunk #2 succeeded at 38 (offset -1 lines).
patching file arch/arm64/kvm/reset.c
Hunk #4 FAILED at 86.
Hunk #5 succeeded at 133 (offset -4 lines).
1 out of 5 hunks FAILED -- saving rejects to file arch/arm64/kvm/reset.c.rej
patching file include/linux/irqchip/arm-gic-v3.h
Hunk #1 succeeded at 374 (offset -14 lines).
patching file include/uapi/linux/kvm.h
Hunk #1 succeeded at 739 (offset -11 lines).
Hunk #2 FAILED at 962.
1 out of 2 hunks FAILED -- saving rejects to file include/uapi/linux/kvm.h.rej
patching file virt/kvm/arm/arm.c
Hunk #1 succeeded at 115 (offset -5 lines).
Hunk #2 succeeded at 207 (offset -6 lines).
Hunk #3 succeeded at 236 (offset -6 lines).
Hunk #4 succeeded at 463 (offset -35 lines).
Hunk #5 FAILED at 546.
Hunk #6 succeeded at 1164 with fuzz 2 (offset -145 lines).
Hunk #7 succeeded at 1261 (offset -147 lines).
1 out of 7 hunks FAILED -- saving rejects to file virt/kvm/arm/arm.c.rej
patching file virt/kvm/arm/mmu.c
Hunk #1 succeeded at 43 with fuzz 2 (offset -2 lines).
Hunk #2 succeeded at 147 (offset -2 lines).
Hunk #3 succeeded at 216 (offset -35 lines).
Hunk #4 succeeded at 226 (offset -35 lines).
Hunk #5 succeeded at 245 (offset -35 lines).
Hunk #6 succeeded at 255 (offset -35 lines).
Hunk #7 succeeded at 272 (offset -35 lines).
Hunk #8 succeeded at 371 with fuzz 2 (offset 40 lines).
Hunk #9 FAILED at 340.
Hunk #10 FAILED at 370.
Hunk #11 FAILED at 388.
Hunk #12 FAILED at 408.
Hunk #13 succeeded at 753 (offset -143 lines).
Hunk #14 succeeded at 842 (offset -143 lines).
Hunk #15 succeeded at 850 (offset -143 lines).
Hunk #16 succeeded at 859 (offset -143 lines).
Hunk #17 succeeded at 881 (offset -143 lines).
Hunk #18 succeeded at 1024 (offset -182 lines).
Hunk #19 succeeded at 1114 (offset -188 lines).
Hunk #20 succeeded at 1148 (offset -188 lines).
Hunk #21 succeeded at 1176 (offset -188 lines).
Hunk #22 succeeded at 1190 (offset -188 lines).
Hunk #23 succeeded at 1333 (offset -196 lines).
Hunk #24 succeeded at 1552 (offset -220 lines).
Hunk #25 succeeded at 1852 (offset -219 lines).
4 out of 25 hunks FAILED -- saving rejects to file virt/kvm/arm/mmu.c.rej
patching file virt/kvm/arm/vgic/vgic-its.c
Hunk #1 succeeded at 233 (offset -8 lines).
Hunk #2 succeeded at 696 (offset -56 lines).
Hunk #3 succeeded at 721 (offset -56 lines).
Hunk #4 succeeded at 736 (offset -56 lines).
Hunk #5 succeeded at 746 (offset -56 lines).
Hunk #6 succeeded at 1214 (offset -80 lines).
Hunk #7 succeeded at 1232 (offset -80 lines).
Hunk #8 succeeded at 1279 (offset -80 lines).
Hunk #9 succeeded at 2135 (offset -82 lines).
Hunk #10 succeeded at 2207 (offset -82 lines).
Hunk #11 succeeded at 2256 (offset -82 lines).
patching file virt/kvm/arm/vgic/vgic-mmio-v3.c
Hunk #1 succeeded at 326 (offset -38 lines).
Hunk #2 succeeded at 343 (offset -38 lines).
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for :.* : True
Logical operation result is TRUE
Running script : #!/bin/bash

Fixes: #296
Signed-off-by: Peng Tao <bergwolf@gmail.com>
bergwolf added a commit that referenced this issue Jan 4, 2019
Reverts #269

The added kernel patch doesn't apply and is causing all CI to fail. FYI, the current guest kernel version is 4.14.67. Please rebase and re-submit your PR cleanly @Pennyzct.

e.g., http://jenkins.katacontainers.io/job/kata-containers-runtime-centos-7-4-PR/995/console

NFO: Applying patch /tmp/jenkins/workspace/kata-containers-runtime-centos-7-4-PR/go/src/github.com/kata-containers/packaging/kernel/patches/0003-backport-Arm64-KVM-Dynamic-IPA-and-52bit-IPA-support.patch
patching file arch/arm/include/asm/kvm_arm.h
patching file arch/arm/include/asm/kvm_host.h
Hunk #1 succeeded at 266 (offset -7 lines).
Hunk #2 succeeded at 314 with fuzz 2 (offset -40 lines).
patching file arch/arm/include/asm/kvm_mmu.h
Hunk #1 FAILED at 35.
Hunk #2 succeeded at 47 with fuzz 2 (offset -5 lines).
Hunk #3 succeeded at 266 with fuzz 2 (offset -96 lines).
1 out of 3 hunks FAILED -- saving rejects to file arch/arm/include/asm/kvm_mmu.h.rej
patching file arch/arm/include/asm/stage2_pgtable.h
patching file arch/arm64/include/asm/cpufeature.h
Hunk #1 FAILED at 262.
Hunk #2 succeeded at 284 (offset -246 lines).
1 out of 2 hunks FAILED -- saving rejects to file arch/arm64/include/asm/cpufeature.h.rej
patching file arch/arm64/include/asm/kvm_arm.h
Hunk #1 succeeded at 102 (offset -5 lines).
Hunk #2 succeeded at 116 (offset -5 lines).
Hunk #3 succeeded at 305 (offset -7 lines).
patching file arch/arm64/include/asm/kvm_asm.h
Hunk #1 succeeded at 27 (offset -3 lines).
Hunk #2 FAILED at 73.
1 out of 2 hunks FAILED -- saving rejects to file arch/arm64/include/asm/kvm_asm.h.rej
patching file arch/arm64/include/asm/kvm_host.h
Hunk #1 succeeded at 48 (offset -5 lines).
Hunk #2 FAILED at 61.
Hunk #3 succeeded at 399 with fuzz 2 (offset -41 lines).
Hunk #4 FAILED at 503.
2 out of 4 hunks FAILED -- saving rejects to file arch/arm64/include/asm/kvm_host.h.rej
patching file arch/arm64/include/asm/kvm_hyp.h
patching file arch/arm64/include/asm/kvm_mmu.h
Hunk #1 succeeded at 154 (offset 13 lines).
Hunk #2 succeeded at 229 with fuzz 1 (offset -17 lines).
Hunk #3 succeeded at 409 with fuzz 2 (offset -110 lines).
patching file arch/arm64/include/asm/stage2_pgtable-nopmd.h
patching file arch/arm64/include/asm/stage2_pgtable-nopud.h
patching file arch/arm64/include/asm/stage2_pgtable.h
patching file arch/arm64/kvm/guest.c
Hunk #1 succeeded at 305 (offset -86 lines).
patching file arch/arm64/kvm/hyp-init.S
Hunk #1 FAILED at 65.
1 out of 1 hunk FAILED -- saving rejects to file arch/arm64/kvm/hyp-init.S.rej
patching file arch/arm64/kvm/hyp/Makefile
Hunk #1 succeeded at 18 (offset -1 lines).
patching file arch/arm64/kvm/hyp/s2-setup.c
Reversed (or previously applied) patch detected! Assume -R? [n]
Apply anyway? [n]
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file arch/arm64/kvm/hyp/s2-setup.c.rej
patching file arch/arm64/kvm/hyp/switch.c
Hunk #1 FAILED at 198.
Hunk #2 succeeded at 226 (offset -37 lines).
1 out of 2 hunks FAILED -- saving rejects to file arch/arm64/kvm/hyp/switch.c.rej
patching file arch/arm64/kvm/hyp/tlb.c
Hunk #1 succeeded at 29 (offset -1 lines).
Hunk #2 succeeded at 38 (offset -1 lines).
patching file arch/arm64/kvm/reset.c
Hunk #4 FAILED at 86.
Hunk #5 succeeded at 133 (offset -4 lines).
1 out of 5 hunks FAILED -- saving rejects to file arch/arm64/kvm/reset.c.rej
patching file include/linux/irqchip/arm-gic-v3.h
Hunk #1 succeeded at 374 (offset -14 lines).
patching file include/uapi/linux/kvm.h
Hunk #1 succeeded at 739 (offset -11 lines).
Hunk #2 FAILED at 962.
1 out of 2 hunks FAILED -- saving rejects to file include/uapi/linux/kvm.h.rej
patching file virt/kvm/arm/arm.c
Hunk #1 succeeded at 115 (offset -5 lines).
Hunk #2 succeeded at 207 (offset -6 lines).
Hunk #3 succeeded at 236 (offset -6 lines).
Hunk #4 succeeded at 463 (offset -35 lines).
Hunk #5 FAILED at 546.
Hunk #6 succeeded at 1164 with fuzz 2 (offset -145 lines).
Hunk #7 succeeded at 1261 (offset -147 lines).
1 out of 7 hunks FAILED -- saving rejects to file virt/kvm/arm/arm.c.rej
patching file virt/kvm/arm/mmu.c
Hunk #1 succeeded at 43 with fuzz 2 (offset -2 lines).
Hunk #2 succeeded at 147 (offset -2 lines).
Hunk #3 succeeded at 216 (offset -35 lines).
Hunk #4 succeeded at 226 (offset -35 lines).
Hunk #5 succeeded at 245 (offset -35 lines).
Hunk #6 succeeded at 255 (offset -35 lines).
Hunk #7 succeeded at 272 (offset -35 lines).
Hunk #8 succeeded at 371 with fuzz 2 (offset 40 lines).
Hunk #9 FAILED at 340.
Hunk #10 FAILED at 370.
Hunk #11 FAILED at 388.
Hunk #12 FAILED at 408.
Hunk #13 succeeded at 753 (offset -143 lines).
Hunk #14 succeeded at 842 (offset -143 lines).
Hunk #15 succeeded at 850 (offset -143 lines).
Hunk #16 succeeded at 859 (offset -143 lines).
Hunk #17 succeeded at 881 (offset -143 lines).
Hunk #18 succeeded at 1024 (offset -182 lines).
Hunk #19 succeeded at 1114 (offset -188 lines).
Hunk #20 succeeded at 1148 (offset -188 lines).
Hunk #21 succeeded at 1176 (offset -188 lines).
Hunk #22 succeeded at 1190 (offset -188 lines).
Hunk #23 succeeded at 1333 (offset -196 lines).
Hunk #24 succeeded at 1552 (offset -220 lines).
Hunk #25 succeeded at 1852 (offset -219 lines).
4 out of 25 hunks FAILED -- saving rejects to file virt/kvm/arm/mmu.c.rej
patching file virt/kvm/arm/vgic/vgic-its.c
Hunk #1 succeeded at 233 (offset -8 lines).
Hunk #2 succeeded at 696 (offset -56 lines).
Hunk #3 succeeded at 721 (offset -56 lines).
Hunk #4 succeeded at 736 (offset -56 lines).
Hunk #5 succeeded at 746 (offset -56 lines).
Hunk #6 succeeded at 1214 (offset -80 lines).
Hunk #7 succeeded at 1232 (offset -80 lines).
Hunk #8 succeeded at 1279 (offset -80 lines).
Hunk #9 succeeded at 2135 (offset -82 lines).
Hunk #10 succeeded at 2207 (offset -82 lines).
Hunk #11 succeeded at 2256 (offset -82 lines).
patching file virt/kvm/arm/vgic/vgic-mmio-v3.c
Hunk #1 succeeded at 326 (offset -38 lines).
Hunk #2 succeeded at 343 (offset -38 lines).
Build step 'Execute shell' marked build as failure
Performing Post build task...
Match found for :.* : True
Logical operation result is TRUE
Running script : #!/bin/bash

Fixes: #296
Signed-off-by: Peng Tao <bergwolf@gmail.com>
@grahamwhaley
Copy link
Contributor

Right, let's start trying to add a little 'meat' to the whole config fragment stuff. I didn't find too much in any other Issue.

We have discussed seeing if we can use the 'config merge' script before. that lives in the linux kernel sources here afaict.
And then there is an interesting (recent) presentation on the ability for the kernel to merge fragments here that briefly shows how the kernel makefile can take multiple config args, and how to invoke merge_config.sh.

I'm going to have a little play with merge_config.sh and either the allnoconfig or tinyconfig. I suspect allnoconfig is the best base choice, as iirc tinyconfig may be an x86 only thing.

@egernst
Copy link
Member

egernst commented Jul 17, 2019

Stale -- I think we have forward progress now on this wrt config fragments. Please re-open if you disagree @jodh-intel

@egernst egernst added the stale Issue or PR was not updated in a timely fashion label Jul 17, 2019
@devimc
Copy link

devimc commented Aug 14, 2019

closing issue since kernel fragments support already landed

@devimc devimc closed this as completed Aug 14, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
stale Issue or PR was not updated in a timely fashion
Projects
None yet
Development

No branches or pull requests

6 participants