Automate retrieval of Linux defconfig

[ricardon]

The Yocto Project kconfig scripts expects a defconfig when building Linux. We add configuration fragments on top of the ARCH_defconfig for all the CPU architectures we support.

Instead of using patches to update our defconfig, it could be possible to pull it directly from the Linux source tree.

In the past we tried to get rid of the defconfig by passing
--alldefconfig to the Yocto kconfig script. Looking at the script's
comments, --alldefconfig does the following:

alldefconfig: Fills in any missing symbols with Kconfig default

Which I don't think is what we want.

Perhaps this could be implemented with an unpack_append task that copies
${TARGET_ARCH}/configs/{TARGET_ARCH}_defconfig into ${S}/defconfig.

[nareshgbhat]

Do you want me split the linux-yocto-efi-test_4.10.bb into 2 parts and add linux-yocto-efi-test_4.10.bbappend and move few functions over bbappend ?

or

Should I just go ahead just modify with the above functionality as you have suggested and remove alldefconfig

[ricardon]

I don't think we need a linux-yocto-efi-test_4.10.bbappend. the do_unpack_append can be written in the current linux-yocto-efi-test_4.10.bb. It would do something like this

do_unpack_append() {
cp ${S}/arch/{some_arch}/configs/{default_config_filename} ${WORKDIR}/patches/defconfig
}
Of course, we need logic to set some_arch and default_config_filename correctly. For x86 it will be x86 and x86_64_defconfig and i386_defconfig.

Also, do_unpack is written in python. Hence, the append, needs to either be written in python or have it written as a sh function and called from do_unpack_append with bb.build.exec_func('sh_function', d).

[nareshgbhat]

Thanks Ricardo, I looked into the meta/classes/kernel-yocto.bbclass the function do_kernel_configme() If we are not using "allnoconfig", "alldefconfig" than the 3rd condition will be keep a copy of defconfig (a copy of x86_64-defconfig or i386_defconfig or defconfig from arm64) in ${WORKDIR}

if [ -f ${WORKDIR}/defconfig ]; then
config_flags="-n"
fi

i.e nothing but the implementation something like

Retrieve the defconfig from respective architecture of kernel source and keep it as defconfig inside kernel source directory.

do_unpack_append() {
bb.build.exec_func('unpack_configure', d)
}
unpack_configure() {
if [ "${TARGET_ARCH}" = "x86_64" ]; then
cp -pRv git/arch/x86/configs/x86_64_defconfig git/defconfig
fi
if [ "${TARGET_ARCH}" = "x86" ]; then
cp -pRv git/arch/x86/configs/i386_defconfig git/defconfig
fi
if [ "${TARGET_ARCH}" = "aarch64" ]; then
cp -pRv git/arch/arm64/configs/defconfig git/defconfig
fi
}

Let me know your thoughts on above implementation.

[ricardon]

Hi Naresh,

Yes, that is exactly what I had in mind. Perhaps I would explicitly use some bitbake variables for the copy. For instance:
cp -pRv ${S}/git/arch/x86/configs/x86_64_defconfig ${WORKDIR}/git/defconfig

Also, I am not sure where exactly the defconfig is put. I think it is unde ${WORKDIR}/qemu${TARGET_ARCH}

As a stretch, the arch/${arch}/configs/${configarch}_defconfig patter is repeated for the three supported archs. It would be great if the copy could be done in a single line by simply using variables. Unfortunately the tree lines have subtle differences. For instance, for x86 i386 and x86_64 are under the same directory; arm64 does not have a prefix in the defconfig filename. Thus, it is not trivial. If you find a way of making it nice it would be great. If not, I am happy with the three separate copy lines. Ah, also it would be good to make the two last ifs as elifs and maybe a final else that returns an error.

[nareshgbhat]

I think the simple way to implement this feature is to use of 2 variables
KBUILD_DEFCONFIG = "defconfig"
This will use default kernel config from kernel source. As this has been mentioned - https://github.com/01org/luv-yocto/blob/master/meta/classes/kernel-yocto.bbclass#L102

And create a SCC file e.g. using patches http://www.yoctoproject.org/docs/1.6.1/kernel-dev/kernel-dev.html#scc-reference
e.g.
kconf non-hardware network.cfg
kconf non-hardware sbbr.cfg

And apply the SCC file using below variable.

KERNEL_FEATURES_append_aarch64 += " features/qemuaarch64.scc"

I have't tried it yet, I am going to try this method. Let me know if you have any suggestions.

[ricardon]

Yes this sounds good to me. I am not if we need the KERNEL_FEATURES_append_parts. Wouldn't it be sufficient to have the .scc files in the SRC_URI? I really don't know.

Also, this would need to rename our current .cfg fragments. Would KERNEL_FEATURES work with .cfg file?

Something to ponder.

[nareshgbhat]

I could able to solve the automatic retrieval of defconfig and merge the config fragment as .cfg files in a very simple way.

STEP 1. We have to remove the defconfig from
meta-luv/recipes-kernel/linux/linux-yocto-efi-test/qemuarm64/defconfig
and also defconfig patch file://qemuarm64/network.cfg from meta-luv/recipes-kernel/linux/linux-yocto-efi-test_4.10.bb recipe

STEP 2. This is how automatic defconfig is retrieved
In do_kernel_metadata function there is a condition as you mentioned in your comment - https://github.com/01org/luv-yocto/blob/master/meta/classes/kernel-yocto.bbclass#L102 which check for the KBUILD_DEFCONFIG variable. If it is defined as defconfig in recipe, this will pick up defconfig from path arch/arm64/configs/defconfig

I have verified and confirmed the defconfig as follows,

$ bitbake -c devshell linux-yocto-efi-test

You will get a devshell for kernel and verified the defconfig and other patches diff,

root@nareshbhat:/test-next-1/luv-yocto/build/tmp/work-shared/qemuarm64/kernel-source# ls -l .kernel-meta/
total 28
drwxrwxr-x 3 1000 1000 4096 May 4 10:52 cfg
-rw-rw-r-- 1 1000 1000 218 May 4 10:51 config.queue
drwxrwxr-x 2 1000 1000 4096 May 4 10:51 configs
-rw-rw-r-- 1 1000 1000 3021 May 4 10:51 meta-series
-rw-rw-r-- 1 1000 1000 716 May 4 10:51 non-hardware_frags.txt
drwxrwxr-x 2 1000 1000 4096 May 4 10:51 patches
-rw-rw-r-- 1 1000 1000 700 May 4 10:51 patch.queue
lrwxrwxrwx 1 1000 1000 11 May 4 10:51 series -> patch.queue
root@nareshbhat:/test-next-1/luv-yocto/build/tmp/work-shared/qemuarm64/kernel-source# ls -l .kernel-meta/configs/
total 32
-rw-r--r-- 1 1000 1000 11739 May 4 10:51 defconfig
-rw-rw-r-- 1 1000 1000 123 May 4 10:51 efi.cfg
-rw-rw-r-- 1 1000 1000 56 May 4 10:51 highmem.cfg
-rw-rw-r-- 1 1000 1000 258 May 4 10:51 network.cfg
-rw-rw-r-- 1 1000 1000 679 May 4 10:51 pstore.cfg
-rw-rw-r-- 1 1000 1000 59 May 4 10:51 sbbr.cfg
root@nareshbhat:~/test-next-1/luv-yocto/build/tmp/work-shared/qemuarm64/kernel-source# vimdiff .kernel-meta/configs/defconfig arch/arm64/configs/defconfig

STEP 3. The third step is merging. With the switch case "-m -r" for merge_config.sh in file meta/classes/kernel-yocto.bbclass and under function do_kernel_configme generates the .config file which is being used by the function do_configure
This is happening because you can notice the below addtask statement at the end of the function do_kernel_configme()
addtask kernel_configme before do_configure after do_patch

If this solution looks feasible. I will cleanup and issue patches on ML.

[nareshgbhat]

Patches are posted on ML - https://lists.01.org/pipermail/luv/2017-May/001766.html

[nareshgbhat]

v1 version of patches posted on ML - https://lists.01.org/pipermail/luv/2017-May/001800.html

[ricardon]

Hi Naresh I took a look at your patches. I'll post my comments tomorrow. Thanks and BR, Ricardo

…

On May 16, 2017 1:11 PM, "Naresh Bhat" ***@***.***> wrote: v1 version of patches posted on ML - https://lists.01.org/ pipermail/luv/2017-May/001800.html — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#116 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAldaN4gxWIhJLh5UA95XRFlVlFOPUQPks5r6gMKgaJpZM4MbtSM> .

[nareshgbhat]

v2 version set of patches got merged - https://lists.01.org/pipermail/luv/2017-June/001834.html I think we can close the issue.

[ricardon]

Thanks for the patches!