Skip to content

docker: Add build deps for perf#1321

Closed
broonie wants to merge 1 commit intokernelci:mainfrom
broonie:perf-docker
Closed

docker: Add build deps for perf#1321
broonie wants to merge 1 commit intokernelci:mainfrom
broonie:perf-docker

Conversation

@broonie
Copy link
Copy Markdown
Member

@broonie broonie commented Jul 26, 2022

The perf tool is included in the kernel source and has some entertainingly
extensive build dependency options. In preparation for building perf and
it's testsuite add these to our images, more work will be needed to enable
both builds but this will be a bit easier if the dependencies are in the
docker images.

There are some duplicate dependencies with kselftest, I've left these in
place as apt will cope with the duplicate adds without trouble and there
is a documenting effect from including them twice.

Signed-off-by: Mark Brown broonie@kernel.org

@nuclearcat
Copy link
Copy Markdown
Member

clang-base fails to build:
E: Unable to locate package libbabeltrace-dev:armel
E: Unable to locate package libcap-dev:armel
E: Unable to locate package libdw-dev:armel
E: Unable to locate package libnuma-dev:armel
E: Unable to locate package libopencsd-dev:armel
E: Unable to locate package libperl-dev:armel
E: Unable to locate package libpython3-dev:armel
E: Unable to locate package libslang2-dev:armel
E: Unable to locate package libssl-dev:armel
E: Unable to locate package libunwind-dev:armel
E: Unable to locate package libzstd-dev:armel
E: Unable to locate package systemtap-sdt-dev:armel
and etc

Copy link
Copy Markdown
Member

@nuclearcat nuclearcat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works fine for me

@gctucker
Copy link
Copy Markdown
Collaborator

gctucker commented Aug 3, 2022

Without these changes and using gcc-10:

Auto-detecting system features:
...                         dwarf: [ OFF ]
...            dwarf_getlocations: [ OFF ]
...                         glibc: [ on  ]
...                        libbfd: [ OFF ]
...                libbfd-buildid: [ OFF ]
...                        libcap: [ on  ]
...                        libelf: [ on  ]
...                       libnuma: [ on  ]
...        numa_num_possible_cpus: [ on  ]
...                       libperl: [ OFF ]
...                     libpython: [ OFF ]
...                     libcrypto: [ on  ]
...                     libunwind: [ OFF ]
...            libdw-dwarf-unwind: [ OFF ]
...                          zlib: [ on  ]
...                          lzma: [ OFF ]
...                     get_cpuid: [ on  ]
...                           bpf: [ on  ]
...                        libaio: [ on  ]
...                       libzstd: [ OFF ]
...        disassembler-four-args: [ OFF ]

Now trying again with a Docker image that has the added packages

@gctucker
Copy link
Copy Markdown
Collaborator

gctucker commented Aug 3, 2022

@broonie Always do apt-get update before apt-get install on the same line in Dockerfiles.

I've fixed it in this PR to test the clang images but will add another commit to fix it everywhere if this works as expected.

@gctucker
Copy link
Copy Markdown
Collaborator

gctucker commented Aug 3, 2022

On clang-14 with the packages from this PR installed:

Auto-detecting system features:
...                         dwarf: [ OFF ]
...            dwarf_getlocations: [ OFF ]
...                         glibc: [ on  ]
...                        libbfd: [ OFF ]
...                libbfd-buildid: [ OFF ]
...                        libcap: [ on  ]
...                        libelf: [ on  ]
...                       libnuma: [ on  ]
...        numa_num_possible_cpus: [ on  ]
...                       libperl: [ OFF ]
...                     libpython: [ OFF ]
...                     libcrypto: [ on  ]
...                     libunwind: [ OFF ]
...            libdw-dwarf-unwind: [ OFF ]
...                          zlib: [ on  ]
...                          lzma: [ OFF ]
...                     get_cpuid: [ on  ]
...                           bpf: [ on  ]
...                        libaio: [ on  ]
...                       libzstd: [ OFF ]
...        disassembler-four-args: [ OFF ]

The diff shows no difference, so I don't think this PR is making any difference at all. Am I missing something?

@gctucker
Copy link
Copy Markdown
Collaborator

gctucker commented Aug 3, 2022

I'll skip this one as well as #1158 for now to complete the migration to the new Docker images, then the changes for perf and gki_defconfig builds can be ported there.

The perf tool is included in the kernel source and has some entertainingly
extensive build dependency options. In preparation for building perf and
it's testsuite add these to our images, more work will be needed to enable
both builds but this will be a bit easier if the dependencies are in the
docker images.

There are some duplicate dependencies with kselftest, I've left these in
place as apt will cope with the duplicate adds without trouble and there
is a documenting effect from including them twice.

Signed-off-by: Mark Brown <broonie@kernel.org>
@gctucker
Copy link
Copy Markdown
Collaborator

@broonie What's the make command to build the perf tests that require these additional packages?

@gctucker
Copy link
Copy Markdown
Collaborator

@broonie What's the make command to build the perf tests that require these additional packages?

As discussed in today's meeting, the make target is build-test.

@gctucker
Copy link
Copy Markdown
Collaborator

Erm I'm getting this now:

In file included from ui/gtk/util.c:3:
ui/gtk/gtk.h:8:10: fatal error: gtk/gtk.h: No such file or directory
    8 | #include <gtk/gtk.h>
      |          ^~~~~~~~~~~

Do we really want to add libgtk-3-dev?

@gctucker
Copy link
Copy Markdown
Collaborator

So I've added libgtk-3-dev, and then libbpf-dev, libbpfcc-dev, and now it's failing to build due to missing libopencsd dev package... Could you please either add the full list of packages needed to build with make -C tools/perf build-test or explain how to build a subset with the packages only included in this PR?

@broonie
Copy link
Copy Markdown
Member Author

broonie commented Aug 18, 2022

OpenCSD needs a backport which is in progress but held up by vacations, the currently packaged version is too old.

@gctucker
Copy link
Copy Markdown
Collaborator

Alright I think we'll have to leave this one stale for a while. It might be worth closing it and opening a new one on top of the new config when all the packages are actually available and we can build the perf build-test in Debian stable (+ backports).

@gctucker gctucker self-assigned this Aug 30, 2022
@gctucker
Copy link
Copy Markdown
Collaborator

While building perf is entirely valid and very good idea, this PR is now lagging behind all the changes in config/docker and there were issues with the list of dependencies. So closing this now, we can create a new PR later to revive this. Also the new pipeline should be able to build perf as an extra build step and a separate Docker image which might simplify things a bit.

@gctucker gctucker closed this Mar 16, 2023
@broonie broonie reopened this Mar 16, 2023
@gctucker
Copy link
Copy Markdown
Collaborator

@broonie How do you propose to resolve this?

@gctucker gctucker added the staging-skip Don't test automatically on staging.kernelci.org label Mar 16, 2023
@broonie
Copy link
Copy Markdown
Member Author

broonie commented Mar 16, 2023

If nothing else it'll be resolved once we switch up to bootworm. In the meantime either build an OpenCSD or configure it out of the builds probably.

@gctucker
Copy link
Copy Markdown
Collaborator

Well please rebase and rewrite it to use the new Jinja2 templates, and address the issue with OpenCSD. I don't think it's going to be helpful to leave this open PR stale until later this year when Debian bookworm is released. Opening a new PR then with the packages available and when the new build job templates are available would seem like a better approach as it'll actually be possible to merge it. We can always have a GitHub issue to keep track of it in the meantime.

@broonie
Copy link
Copy Markdown
Member Author

broonie commented Mar 16, 2023

I'll get round to it at some point.

@gctucker
Copy link
Copy Markdown
Collaborator

Closing now as it has been stale for too long. It would be worth adding a Docker image template for kci docker at some point and add a build step with the new pipeline I think, and there are already some GitHub issues to keep track of it: #498 #679.

@gctucker gctucker closed this Apr 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

staging-skip Don't test automatically on staging.kernelci.org

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants