Optimal linux kernel configuration (version 6.1 and higher)

[diederikdehaas]

Is your feature request related to a problem? Please describe.

I can see what Debian/Mobian have enabled in their kernel for anbox/waydroid support and on https://wiki.archlinux.org/title/Waydroid#Building_a_kernel I can see different ways one can do it on Arch.

What I haven't found is which modules you should enable and why (and under what circumstance).

Describe the solution you'd like

A clear description of the ideal linux kernel configuration from waydroid's perspective, from the people who should know it best: the waydroid developers.
IOW: Which Kconfig options should be enabled and/or which string value should they have for optimal performance of waydroid. When there is a trade-off involved, I would like to see the pro's and con's of each possible solution and what you'd recommend and why.
That way, distro's can make an informed decision on how to configure their kernel.

It would also be useful to know if things would be different for say amd64 and arm64 or armhf platforms (and how).

Additional context

In https://salsa.debian.org/kernel-team/linux/-/merge_requests/651/diffs?commit_id=e9d7885619fd108fba3889e321afa5b583eb6d4d I wrote the following:

Drop patches:
- debian/android-enable-building-ashmem-and-binder-as-modules.patch
- debian/export-symbols-needed-by-android-drivers.patch
After https://bugs.debian.org/901492 the preceding 2 patches were
created for anbox support. However in kernel 5.18 `ashmem` was removed
from the upstream kernel and since then, anbox has not been working as
reported in https://bugs.debian.org/1014329.
Then in https://bugs.debian.org/1032304, titled "RM: anbox -- ROM;
Upstream discontinued", the anbox package has been removed from the
Debian archive. And on Anbox's GH page one can see the following:
"It's development has however stalled in the past years and it's only
fair to say that now in 2023 it's no longer actively developed."
So it's of no use to continue carrying these patches.

So I wanted (to propose) to remove those patches in a 6.3+ kernel (Bookworm will have 6.1, so after that).
I didn't know if waydroid would need/benefit from them and figured they should file a bug in Debian's BTS, just like happened with anbox.
Then I heard that waydroid likely would benefit from it (and changed the MR status to Draft), but I couldn't find a proper/definitive answer as to which options should be enabled, just some suggestions which one could.
So thus issue.

FTR: A bug report in Debian's BTS (against src:linux) would be useful, especially if a dialog is needed/wanted with the Debian kernel maintainers (which I am not, I'm 'just' a contributor).

[diederikdehaas]

FTR: A bug report in Debian's BTS (against src:linux) would be useful, especially if a dialog is needed/wanted with the Debian kernel maintainers (which I am not, I'm 'just' a contributor).

I went ahead and created one: https://bugs.debian.org/1033176

[aleasto]

Which Kconfig options should be enabled and/or which string value should they have for optimal performance of waydroid

Unfortunately I have no answer to this question. You may try asking yourself the same question: "Which Kconfig options should be enabled to run debian's userspace?". The reality is that Android's userspace is too large for me to know which options are required.

The Android team at Google maintains a base kernel config that android vendors can expand on, documented here, so the android linux-6.1 base config might be what you are looking for.
Downstream instead, postmarketos contributors have identified a subset of configs that have been found sometimes missing in their kernels and you can find the list here (although i can tell you that it's not true at the moment that SQUASHFS is required, nor that ANDROID_BINDERFS must be unset, so take that with a grain of salt)

However, experimentally we find that many GNU/Linux distributions' kernels already include enough options to run Waydroid functionally (I wouldn't know about optimally), debian included.

One crucial Kconfig to run Android's userspace that may not always be found in GNU/Linux kernels, is CONFIG_ANDROID_BINDER_IPC along with its siblings CONFIG_ANDROID_BINDERFS and CONFIG_ANDROID_BINDER_DEVICES. Debian has been compiling this as a module (I see it's a downstream patch in debian that adds support for this), but you may as well compile them in like Fedora is doing: binder is considered mature enough to be a stable linux driver today, and with CONFIG_ANDROID_BINDERFS=y it is only available after mounting a binderfs somewhere.

As for ashmem, Android's userspace has a (currently experimental) translation layer from ashmem library calls to memfd syscalls, which modern linux distributions all support, that we enabled in order to run Waydroid in absence of ashmem. Anbox was unable to do the same, so it stopped working after ashmem was dropped.
It is not perfect, specifically if some userspace (third party android apps) accesses the driver directly instead of through the ashmem standard library they will find that /dev/ashmem is missing. So coming back to your optimality point, having ashmem would be better; but Waydroid and most third party apps can run just fine without.
I would not advise anyone to try to resurrect the ashmem driver after its removal, as it was never considered mature.

TL;DR: Keep building the binder driver, ignore the ashmem driver

[diederikdehaas]

One of the reasons I opened this issue is that it appears to me that everyone follows what is described in the arch wiki, which suggests to either use BINDERFS and empty BINDER_DEVICES or disable BINDERFS and do specify BINDER_DEVICES. And the latter option seems to be used the most.
But I still haven't been able to find out why.

Thanks for your response :-)

[aleasto]

BINDER_DEVICES defines which binder device names should appear in /dev. Different android versions required different devices to be available: the latest android version uses "binder,hwbinder,vndbinder" meaning /dev/binder, /dev/hwbinder, /dev/vndbinder.
At some point Google realized that it's inconvenient to have this change at compile time, so they introduced BINDERFS: you can mount a filesystem of type binder somewhere and use IOCTLs on it to allocate new device names.

So if you use BINDERFS it doesn't matter which value of BINDER_DEVICES you set because you can add (or remove) devices at runtime. In this case BINDER_DEVICES is just the set of initial devices present in the root of a binderfs.

For mainline linux, BINDERFS is heavily recomended. Waydroid will use it to create all the binder devices it needs. You can leave BINDER_DEVICES empty or unchanged or whatever you prefer.

[aleasto]

Feel free to ask more questions here!

Debian removing binder would be quite damaging to this community... Users would have to be instructed to download and compile the binder DKMS fork

[diederikdehaas]

Debian removing binder would be quite damaging to this community... Users would have to be instructed to download and compile the binder DKMS fork

Once I realized that Waydroid could benefit from it, I reverted the above mentioned removal.
It's too late to change things for Bookworm, so it will have BINDER_DEVICES with 'only' binder, but no BINDERFS.

Thanks for your explanation, now I can make an informed case to possibly change that for Trixie.

[aleasto]

Once I realized that Waydroid could benefit from it, I reverted the above mentioned removal. It's too late to change things for Bookworm, so it will have BINDER_DEVICES with 'only' binder, but no BINDERFS.

As in... for the whole lifetime of bookworm?

[diederikdehaas]

Yes. We're now in the hard-freeze of Bookworm, so what can be changed now is very limited.
And this is a change which can/could have a big impact ... with ZERO testing, so that's not going to happen.

[aleasto]

Yes. We're now in the hard-freeze of Bookworm, so what can be changed now is very limited. And this is a change which can/could have a big impact ... with ZERO testing, so that's not going to happen.

It's been tested for the whole time before your change...?

[diederikdehaas]

I have reverted my proposed change, which was for Trixie anyway.

[aleasto]

That caught me by surprise because I knew Waydroid worked on bullseye already, and it has the same CONFIGs.
Then I finally realized... When you compile binder as a module (rather than built-in), when inserting the module you can provide the binder device names you want in the module parameters! So in this case CONFIG_BINDER_DEVICES="binder" is not a limitation.