Kconfig Module

[schultetwin]

Add kconfig integration such that existing projects can use the ".config" file from a kconfig system to conditionally compile in or out features.

[jpakkane]

I have no experience with kconfig, so here are some questions on this.

First of all does kconfig always write its configuration to the source directory? Can it be told to write in a build dir? If not, is it possible to have two outstanding build trees for a single source tree with different configurations?

Would the workflow for this be so that you first use kconfig to set up your configuration and then have Meson read that and build stuff? I'm guessing that eventually people will want us to have full kconfig inside Meson but it is not a place I want to go because that requires an SAT solver and all that which makes things massively complicated. We really need to keep these two clearly separated.

Which project are you using this for or is this more of a "nice to have"?

[msink]

Kconfig (in Linux kernel, Buildroot) supports building out of tree

When using out-of-tree builds, the Buildroot .config and temporary files are also stored in the output directory. This means that you can safely run multiple builds in parallel using the same source tree as long as they use unique output directories.

[schultetwin]

First of all does kconfig always write its configuration to the source directory? Can it be told to write in a build dir? If not, is it possible to have two outstanding build trees for a single source tree with different configurations?

Looking at the kconfig frontend tooling, they can parse a .config file from multiple locations:
http://ymorin.is-a-geek.org/projects/kconfig-frontends
Right now it's passed in via environmental variable (bah), but I'm happy to try and get them to fix that. That being said, the tooling for generating the .config file is separate from using them. So, for example, the project I'm thinking of using meson for (NuttX) has tons of configurations stored as "defconfig" files. So I think yes, it's possible to have two outstanding build trees for a single source tree.

Would the workflow for this be so that you first use kconfig to set up your configuration and then have Meson read that and build stuff? I'm guessing that eventually people will want us to have full kconfig inside Meson but it is not a place I want to go because that requires an SAT solver and all that which makes things massively complicated. We really need to keep these two clearly separated.

Yes my thought it to do the configuration first, and then parse the generated ".config" file in meson. I have no desire to fully support kconfig inside Meson.

Which project are you using this for or is this more of a "nice to have"?

I would like to use this for NuttX. However, this is currently just a "for fun" thing for me. I don't even now if the upstream developer of NuttX would accept changing the build system, but I want to see if even possible before doing so. I'd say my probability of success is low.
http://nuttx.org/

[jpakkane]

Regardless of anything else the test for this should exercise "the full pipe" so to say. That is the source tree should not have a .config file but instead the kconfig defitions. The test would then run kconfig first and only then Meson which would pick things up from the build dir. This requires converting it from a project test into a unit test.

[schultetwin]

Regardless of anything else the test for this should exercise "the full pipe" so to say. That is the source tree should not have a .config file but instead the kconfig defitions. The test would then run kconfig first and only then Meson which would pick things up from the build dir. This requires converting it from a project test into a unit test.

I don't think that quite makes sense. Let me try explaining better how kconfig works. All kconfig does, is generate a configuration file. Once that file has been generated, you normally commit the file and the repo, and you do not need to run kconfig tools again unless you want to change the configuration.

Kconfig consists of 3 parts:

1. Kconfig files
2. defconfig files/.config
3. kconfig tooling

Kconfig files - These files define your configuration options. They are defined in "Kconfig" files across a source tree. These files contain things like "bool DEBUG_OPTIMIZATIONS". But these only define the options, not a configuration.

defconfig files - These files define a configuration. They're very simple, human readable files. Each line contains a setting for a Kconfig option defined in an above Kconfig file. For example: CONFIG_DEBUG_OPTIMIZATIONS=y. These configurations are stored in your source repo as well, such that others can build the same configuration. In the linux kernel, they're stored in "arch//configs/<my_board_config>". The ".config" file is usually youre working copy of a "defconfig" file. So, for example, if you were building for a linux for some arm board, you would:
cp arch/arm/configs/myarm.defconfig .config
In order to tinker with that configuration.

kconfig-tooling - This is the tooling that helps you generate a defconfig file. It normally works on the ".config" file, as described above. It's helpful in that, you choose the selections you want, and it checks to make sure you're not violating any of the rules defined in the Kconfig files. Once you have the configuration the way you want it, you normally:
cp .config arm/arm/configs/myarm.defconfig
And then commit that back into the repo.

So, traditionally, "the full pipe" for a project that's already in progress would be:
cp <path_to_config_file> .config
make

A project that is new using kconfig needs to generate a configuration first. But I think that can (and should) be done outside of the build system.

[jpakkane]

We are currently in release freeze for 0.40.0. We'll get back to this once it is out. Thanks.

[jpakkane]

This message is posted to every outstanding merge request.

We have transitioned from Github wiki to our new documentation system that generates all documentation directly from the Git repo. This means that from now on every merge request must come with corresponding documentation changes.

If your MR requires documentation changes, do the necessary changes to files in docs/markdown.

If your MR adds a new feature, add a description to the release notes in docs/markdown/Release-notes-for-0.41.0.md.

If this commit is just a bugfix with no functional changes, you don't need to do anything.

Thank you

[jpakkane]

A few more questions.

If this were merged, would supporting a full Linux kernel build be possible? Roughly by first doing a config file and then converting the Makefiles into Meson that would just read option values from a kconfig object?

Would the proposed workflow go like the following:

• run kconfig to generate a .config file at source root
• run Meson, which reads the file and exposes settings
• build definitions query properties of the object and build stuff accordingly

If yes, how would we make this support multiple simultaneous build dirs with different configurations? Something like:

• create build dir
• run kconfig and tell it to write its output file to the build dir (maybe requiring patches to kconfig?)
• run Meson, which picks up the configuration file from builddir
• proceed as above

[schultetwin]

The Linux Kernel build system is complicated enough that I doubt just supporting kconfig is the only change needed to support the linux kernel build. That being said, if we limit the discussion to just using the kconfig files, I think it would look something this:

• create build dir
• As part of the initial run of Meson, you select the config file to use. (i.e. https://github.com/torvalds/linux/blob/master/arch/arm/configs/stm32_defconfig), that copies that file into the build dir as ".config"
• If you want to make changes to the configuration, you can run kconfig then (which may requires some modifications to output the .config file to the build directory).

Working on the documentation. Sorry for the delay.

[anatol]

Somewhat related. There is a project that allows to build standalone mconf tool (similar to the one you can find in the Linux kernel tree) https://github.com/TheGeorge/menuconfig

[jpakkane]

⚔️ conflicts. Sorry for the delay, got sidetracked by a bunch of other work...

[schultetwin]

Updated, sorry for the long delay.

[jpakkane]

Thanks. We have now a new mechanism for getting new, possibly unstable modules in Meson. Basically you should rename the module to unstable_kconfig. See unstable_simd for an example.

In addition the test case should not be on its own standalone level but instead under frameworks.

The release note update should go in the snippets directory and not touch the actual md file (as mentioned in the part that this MR removes, which it should not). This is to avoid constant merge problems due to everyone changing the same file.

Thanks.

[jpakkane]

You also need to edit docs/sitemap.txt and Module-reference.md to make the reference page appear on the site.

[krader1961]

It's probably not my place to make this comment but as someone who just opened a PR to fix #3642 it bothers me that there are PRs more than a year old. And the last update of this PR was 10 months ago. A bit of "spring cleaning" might be in order 😄

[nirbheek]

@schultetwin this PR was ready to go in minus some minor changes. Were you able to find some time to do that?

[msink]

Times change, now here is full python version of kconfig: https://github.com/ulfalizer/Kconfiglib
Works everywhere, on Windows too.
So much deeper integration is now possible - interactive interface for meson configure

[jpakkane]

Sadly our coding rules prohibit depending on any Python package that is not in the standard library.

[msink]

It can be just copied as single .py file, if you don't care about bugfixes.
But it depends on curses, so on Windows have one non-standard dependensy.

Funny part is that it really fixes a bug in the standard library:

https://github.com/zephyrproject-rtos/windows-curses

The curses module is in the Python standard library, but is not available on Windows. Trying to import curses gives an import error for _curses, which is provided by Modules/_cursesmodule.c in the CPython source code.

[ulfalizer]

Kconfiglib guy here.

Kconfiglib and the bundled tools only use the Python standard library, so you could always just copy it in if you don't want to install it via pip.

It's pretty common for projects to want to tweak things a bit anyway. Can check out what these guys are doing for example, or check kconfig.py from Zephyr.

@msink
Yeah, it's unfortunate that the curses module doesn't work out-of-the-box on Windows. There's an issue about it. The patches those wheels are based on come from there as well.

Would probably be a lot of work to get the PDCurses integration up to a standard the Python devs would be happy with, even if it already works for fine for most things.

[nirbheek]

Copying the module into meson and updating it with every major release of kconfig makes sense to me.

[jpakkane]

No. Absolutely not. No external dependency embedding. Ever.

[nirbheek]

No. Absolutely not. No external dependency embedding. Ever.

That's a bit strange. If we won't depend on external projects, and we won't copy them into our codebase, then we're doomed to reimplement everything (badly).

[jpakkane]

Yes. This is one of the unfortunate downsides of being a core dependency at the very bottom of the stack. Every dependency we have goes in the list of things that need to be built on every system bootstrap (new architectures, Linux from Scratch, Yocto, etc etc etc).

This is some maintenance pain we take on so our downstreams don't have to.

[nirbheek]

This is some maintenance pain we take on so our downstreams don't have to.

It's strange in the sense that if we don't take on the maintenance burden of copying external projects, we have to unnecessarily reimplement them (more likely, that feature will be entirely missing) which is even more of a burden.

Other core dependencies like glib also copy external libraries into their codebase for exactly this reason.

[hardening]

s/parser/parses/ ?

[hardening]

s/parser/parses/ ?

[bonzini]

I placed a rebased and updated version of this pull request at https://github.com/bonzini/meson/commits/kconfig. The changes are:

• removed all methods but load() from the module. load() now returns a Meson configuration_data object

• added support for file objects in load()

• added @noKwargs annotation

[dcbaker]

Out of curiosity, why a configuration_data instead of a dictionary?

[bonzini]

Out of curiosity, why a configuration_data instead of a dictionary?

Mostly because I didn't know that configuration_data can take a dictionary too, and also I didn't know that configuration_file can take a dictionary. :) If this is considered a better API than the original one, I can change it to a dictionary and open a pull request for my branch.

I'll also modify #5028 to accept a dictionary in addition to a configuration_data, since (at least for my usecase) the two features are linked.

[bonzini]

Obsoleted by #5031?

[dcbaker]

Yes.