docker: add dwarves package for pahole tool

[nuclearcat]

As reported in maillist https://groups.io/g/kernelci/message/1489 recent builds are failing,
because pahole tool is missing.
This patch add required support and tested locally.

Signed-off-by: Denys Fedoryshchenko denys.f@collabora.com

[nuclearcat]

There is one concern remains, build of mentioned android package with clang and pahole available generate around 819Mbyte of verbose log. I am adding staging-skip label "just in case", as i am not sure our build system is ready for such surprise.
I will test several other builds and might add dwarves in other packages.

[nuclearcat]

Also spotted one more missing package:

WARN: multiple IDs found for 'pt_regs': 3, 3314001 - using 3
/bin/sh: 1: lz4: not found
make[3]: *** [/root/kernelci-quickstart/kernelci-core/linux-x86/arch/x86/boot/compressed/Makefile:137: arch/x86/boot/compressed/vmlinux.bin.lz4] Error 127

Adding lz4 and retesting

[nuclearcat]

lz4 required in clang packages, but not gcc

[gctucker]

@nuclearcat Maybe the staging-skip label can be dropped now, unless there's more known issues to fix?

[nuclearcat]

Sure, just i kept it over weekend as clang with pahole generate big amount of logs (and i dont want to make issues at weekend).

[nuclearcat]

It looks like except memory consumption issue discussed on IRC (LTO option require around 16-18Gb of RAM during linking, and we have only 4Gb on builders), manual tests are ok, but i am failing (likely due OOM) to build on staging, for example https://bot.staging.kernelci.org/job/kernel-build/64062/console

[nuclearcat]

Thats interesting, job stopped at:

../scripts/link-vmlinux.sh: 51: python: not found

I will add python to list of packages

[gctucker]

Is this working now? I believe you said there was still an issue but I don't remember what it was.

[nuclearcat]

Sorry for delay, not yet, as we have some objects generated and hit another issue, probably size limitation during objects upload.
I will try to run jenkins job and in parallel will do manual build to investigate more

[nuclearcat]

After adding missing tools by this PR we able to compile gki_defconfig now (it is using kernelci/staging-clang-15 ):
https://bot.staging.kernelci.org/job/denysf/job/df-main-kernel-build/12/console

[gctucker]

Why is this needed here?

[gctucker]

It's not clear to me why this is needed in the Clang images and not the GCC ones.

[gctucker]

Thats interesting, job stopped at:

../scripts/link-vmlinux.sh: 51: python: not found

I will add python to list of packages

Which kernel revision were you building? I don't see python mentioned anywhere in link-vmlinux.sh on linux-next.

[nuclearcat]

Kernels from https://storage.kernelci.org/android/android13-5.15/ASB-2022-08-05_13-5.15-8-g5d05a30e879f1/linux-src_android13-5.15.tar.gz
and probably tried another versions.

[nuclearcat]

AS i noticed inside there is:

# Generate a linker script to ensure correct ordering of initcalls.
gen_initcalls()
{
        info GEN .tmp_initcalls.lds
        
        ${PYTHON} ${srctree}/scripts/jobserver-exec             \
        ${PERL} ${srctree}/scripts/generate_initcall_order.pl   \
                ${KBUILD_VMLINUX_OBJS} ${KBUILD_VMLINUX_LIBS}   \
                > .tmp_initcalls.lds
}

[gctucker]

Kernels from https://storage.kernelci.org/android/android13-5.15/ASB-2022-08-05_13-5.15-8-g5d05a30e879f1/linux-src_android13-5.15.tar.gz and probably tried another versions.

If this is only in the Android tree then we're not going to fix it. KernelCI is about testing upstream kernels.

Maybe we should talk to the Android team and set up android.kenelci.org (like it has been suggested for a few years already) to cover specific needs for Android kernels. But it doesn't seem like a good idea to bloat the Docker images used to build upstream kernels with tools for Android downstream branches.

[nuclearcat]

As far as i see such fragment present also in vanilla kernels, just checked scripts/link-vmlinux.sh in 5.15 now.

[gctucker]

AS i noticed inside there is:

# Generate a linker script to ensure correct ordering of initcalls.
gen_initcalls()
{
        info GEN .tmp_initcalls.lds
        
        ${PYTHON} ${srctree}/scripts/jobserver-exec             \
        ${PERL} ${srctree}/scripts/generate_initcall_order.pl   \
                ${KBUILD_VMLINUX_OBJS} ${KBUILD_VMLINUX_LIBS}   \
                > .tmp_initcalls.lds
}

That's not upstream, is it? Please be more precise about where you're quoting things from.

[nuclearcat]

Upstream,
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh?h=v5.15#n45
5.15.x kernel tree

[gctucker]

Upstream, https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh?h=v5.15#n45 5.15.x kernel tree

Right, and that got removed recently (well moved to another file) by 5d45950dfbb1540bba3e3762a3497de8b4a715d3.

[gctucker]

OK so gki_defconfig enables the DEBUG_INFO_BTF config which requires the pahole tool which is part of the dwarves package in Debian. So that explains why we would add dwarves everywhere.

I'm not sure about the lz4 and python / python3 part specific to Clang though.

[nuclearcat]

Mostly because i tested it with clang only, probably i should test with gcc, likely it is required there too. I will do that tomorrow.

[gctucker]

OK thanks. As discussed earlier today, I think we can merge this now and get the extra packages added to the base/host-tools.jinja2 template with the new Docker config to test it there directly.

[gctucker]

@nuclearcat BTW you can't build gki_defconfig with GCC as far as I remember, so just building a kernel with DEBUG_INFO_BTF turned on by hand should be enough to check things aren't completely broken.