missing kernel symbol in arm64 5.10.11-v8+ for anbox

[mgrouch]

cat /var/lib/dkms/anbox-ashmem/1/build/make.log
DKMS make.log for anbox-ashmem-1 for kernel 5.10.11-v8+ (aarch64)
Sun 21 Feb 17:03:37 GMT 2021
make -C /lib/modules/5.10.11-v8+/build V=0 M=$PWD
make[1]: Entering directory '/usr/src/linux-headers-5.10.11-v8+'
CC [M] /var/lib/dkms/anbox-ashmem/1/build/deps.o
CC [M] /var/lib/dkms/anbox-ashmem/1/build/ashmem.o
LD [M] /var/lib/dkms/anbox-ashmem/1/build/ashmem_linux.o
MODPOST /var/lib/dkms/anbox-ashmem/1/build/Module.symvers
ERROR: modpost: "kallsyms_lookup_name" [/var/lib/dkms/anbox-ashmem/1/build/ashmem_linux.ko] undefined!
make[2]: *** [scripts/Makefile.modpost:111: /var/lib/dkms/anbox-ashmem/1/build/Module.symvers] Error 1
make[2]: *** Deleting file '/var/lib/dkms/anbox-ashmem/1/build/Module.symvers'
make[1]: *** [Makefile:1712: modules] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-5.10.11-v8+'
make: *** [Makefile:12: all] Error 2

See also anbox/anbox-modules#67

[pelwell]

And also anbox/anbox-modules#49.

I particularly like the comments "What a bunch of incompetent monkeys." and "You do realise this project hasn't been properly maintained in ages right?"

However, this comment suggests that you shouldn't need an out-of-tree module any more. Report back when you've found out what is needed.

[mgrouch]

My understanding they are in Ubuntu kernel but not in raspberry arm64 kernel. just few raspberry kernels back (couple months ago) I was able to follow the steps and load those modules following that procedure.

…

Sent from my iPhone

On Feb 21, 2021, at 12:19 PM, Phil Elwell ***@***.***> wrote:  And also anbox/anbox-modules#49. I particularly like the comments "What a bunch of incompetent monkeys." and "You do realise this project hasn't been properly maintained in ages right?" However, this comment suggests that you shouldn't need an out-of-tree module any more. Report back when you've found out what is needed. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[Dark-Sky]

External modules are not needed in that kernel. They have to be enabled in the kernel config in 2 different places and recompiled though.

# Android
#
CONFIG_ASHMEM=y
# end of Android
...
...
# Android
#
CONFIG_ANDROID=y
CONFIG_ANDROID_BINDER_IPC=y
CONFIG_ANDROID_BINDERFS=y
CONFIG_ANDROID_BINDER_DEVICES="binder,hwbinder,vndbinder"
# CONFIG_ANDROID_BINDER_IPC_SELFTEST is not set
# end of Android

I have been testing anbox after each kernel new kernel compile with aarch64 for quite a while and it works well.

[mgrouch]

Can you please send instructions how raspi os kernel has to be compiled with these 2 modules without compromising/removing what is in the kernel provided by raspberry pi os arm64? Thanks a lot

…

Sent from my iPhone

On Feb 22, 2021, at 11:56 AM, Ray Sherwin ***@***.***> wrote:  External modules are not needed in that kernel. They have to be enabled in the kernel config in 2 different places and recompiled though. # Android # CONFIG_ASHMEM=y # end of Android ... ... # Android # CONFIG_ANDROID=y CONFIG_ANDROID_BINDER_IPC=y CONFIG_ANDROID_BINDERFS=y CONFIG_ANDROID_BINDER_DEVICES="binder,hwbinder,vndbinder" # CONFIG_ANDROID_BINDER_IPC_SELFTEST is not set # end of Android I have been testing anbox after each kernel new kernel compile with aarch64 and it works well. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[Dark-Sky]

This area is for issues with the RPi kernels. You should open up a thread on the raspberry pi forums and provide the info I gave above and ask for help getting the kernel compiled since you evidently know very little about the subject.

Really this issue should be closed as it has nothing to due with an issue with the kernel.

[mgrouch]

Wouldn’t make it easier for all if stock raspberry pi arm64 kernel contained it precompiled already so “incompetent monkeys” like me stoped bothering others with questions? Thanks

…

Sent from my iPhone

On Feb 22, 2021, at 2:17 PM, Ray Sherwin ***@***.***> wrote:  This area is for issues with the RPi kernels. You should open up a thread on the raspberry pi forums and provide the info I gave above and ask for help getting the kernel compiled since you evidently know very little about the subject. Really this issue should be closed as it has nothing to due with an issue with the kernel. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[pelwell]

We will usually consider enabling minor build options on request, but CONFIG_ANDROID does not sound like a minor build option - more like a personality transplant.

[choff]

Hello @pelwell ,

I just wanted to kindly repeat the request to consider enabling CONFIG_ANDROID in Raspberry Pi kernels per default. ashmem and binder (which are required for Anbox) are just about 5000 lines of C code. I don't think that this counts as a "personality transplant" :-) . Anbox used to work well before the upgrade to kernel 5.10 and I would be happy to use it still with newer kernels. Building a custom kernel is very involved and not user-friendly at all - to keep your system secure and up-to-date I'd also need to rebuild this kernel regularly.

If the following kernel options could be enabled, I'd be very happy:

CONFIG_ANDROID=y
CONFIG_ANDROID_BINDER_IPC=y
CONFIG_ANDROID_BINDERFS=y
CONFIG_ASHMEM=y

Unfortunately, it is no longer possible to build anbox and binder as kernel modules - except if I'd consider to apply dirt patches like this one lttng/lttng-modules@a657654 to "anbox-modules", which I'm almost desperate enough to do....

Please consider it again.

Best regards,

Christian

[JamesH65]

Since these options appear to need to be built in to the kernel (not modules), we would need to know the impact that having them has on the below, before we could even start to make a decision ion whether to include them.

1. Kernel Size
2. Kernel memory usage
3. Kernel performance impact.

[choff]

Hello @JamesH65 ,

thanks for your response. I will get back to you as soon as I have an answer to all of your above questions. This might take me a couple of days, as I will need to build a custom kernel with these kernel options enabled. I will also need to review the code of Ashmem and Binder to be able to judge a potential impact on performance and memory usage.

Best regards,

Christian

[JamesH65]

Just to warn you, if any of these things have any noticeable impact, then its very unlikely this option will be added, as it would affect everyone for only a few peoples benefit. (Not withstanding any other objections from our maintainer). Note, figures for Pi Zero would also be relevant here I think, as that show performance hits much more clearly than Pi4's.

[choff]

Hello @JamesH65 and @pelwell ,

sorry for the late reply - during the week I didn't have enough time to look into this topic and so I needed to postpone this Anbox kernel investigation to the weekend. I finally got Anbox to work again with a custom 5.10 kernel! Unfortunately, kernel modifications are required for this. Just enabling the kernel config options for ashmem and binder that I posted above is not sufficient...

This was already an issue with older kernels, where ashmem and binder could still be compiled as modules. Back then the workaround was simply to compile the patched kernel modules out-of-tree. This is now no longer an option as ashmem and binder cannot be compiled as modules anymore. I think I finally need to get in touch with the developers of Binder and Anbox to resolve this issue, so that the required patch to Binder can be mainlined in some way.

I will be on vacation starting next week (without internet access) and so all this will need to wait a while. I will get back to you as soon as I have an update.

Ashmem and Binder should not have a negative impact on performance and memory size. Basically, these are just character devices that are created and if you don't use them, you don't have any memory or performance impact. I reviewed the source code to be sure. But I will get back to you with a more detailed report after the issue with the kernel modifications is solved.

Best regards,

Christian

[mgrouch]

Christian,

Any updates on this? Should this be re-open?

thanks

[choff]

Hello Mikhael, Thanks for the reminder! Unfortunately, I am still on vacation without access to a computer... ☹ I will start working on this in 2 weeks, when I am back again.. I think the first step would be to resolve some issues when building Anbox on ARM, which currently requires code modifications. Afterwards, we can get back to the Raspberry Pi kernel developers and maybe reopen this issue. I will keep you updated on the progress . Best regards,  Christian  Sent from my Samsung Galaxy smartphone. -------- Original message --------From: Mikhail Grushinskiy ***@***.***> Date: 31/03/2021 20:51 (GMT+03:00) To: raspberrypi/linux ***@***.***> Cc: Christian Hoff ***@***.***>, Comment ***@***.***> Subject: Re: [raspberrypi/linux] missing kernel symbol in arm64 5.10.11-v8+ for anbox (#4162) Christian, Any updates on this? Should this be re-open? thanks — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. [ { ***@***.***": "http://schema.org", ***@***.***": "EmailMessage", "potentialAction": { ***@***.***": "ViewAction", "target": "#4162 (comment)", "url": "#4162 (comment)", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { ***@***.***": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

[mgrouch]

Christian,

I hope you had a great vacation!
Any chance this can move forward?

thanks

[mgrouch]

Christian,
Any updates on this?
thanks

[mgrouch]

Christian,
Any chance you get to work on this?
thanks

[chrihoff]

Hello @JamesH65 and @pelwell ,

finally, I have now compiled the information you requested about enabling the kernel config options needed by Anbox:
CONFIG_ANDROID=y
CONFIG_ANDROID_BINDER_IPC=y
CONFIG_ANDROID_BINDERFS=y
CONFIG_ASHMEM=y

I am sorry that this took so long - unfortunately, I could only work on this on weekends and holidays.

The impact on the size of the kernel was the easiest to measure. I found out that the size of "arch/arm/boot/Image" increases by 13 KB when all of the above four kernel options are enabled. The size of the compressed zImage increases by 53 KB for some reason - I would have expected that it doesn't increase by more than 13 KB as well, but anyway... I am not sure if that is a reason for concern as also an increase of 53 KB is still in the moderate range.

To find out more about the performance and memory impact, I reviewed the source code of Binder, Binderfs and Ashmem. Concerning performance, I am sure that there is no impact for users that are not using Anbox. This is for the following reasons:

• Ashmem just creates a device node under /dev/ashmem. If there is no interaction with this device node, Ashmem will do nothing and just allocates some small, minimal data structures.
• Binder will only get active as soon as Binderfs is mounted somewhere. Users that are not using Anbox will not mount Binderfs and Binder will stay dormant.
• Binderfs itself also gets active only as soon as it has been mounted and just registers the binderfs file system type on initialization.
• CONFIG_ANDROID on its own does nothing. It is just a prerequisite to enable Binder and Ashmem.

For the same reason, I am sure that there are also no noticable impact on memory consumption. I reviewed all the code that runs in the initialization phase of Ashmem, Binder and Binderfs. That is the only code that is executed for users that are not interested in the features of Anbox. All this code doesn't allocate large data structures. Bascially, Ashmem only sets up the device node "/dev/ashmem" and Binderfs registers the file system type. That is all.

If you need more information to come to a decision on enabling these Anbox kernel options per default, please let me know.

I have also documented the instructions to get Anbox running on the Raspberry Pi here: https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=312552 . In the future, I also want to make it easier for users to compile Anbox for the Raspberry Pi and so that patches for Anbox are no longer required. But I need to take one step after the other...

Best regards,

Christian

[pelwell]

I am going to give this a try (not this week though), and unless I see a problem it will be enabled for the 64-bit builds.

[pelwell]

Sorry for the wait. A trial kernel shows a static memory loss of about 130kB - not a lot on a system guaranteed to have at least 1GB.

See 9eba214.

[chrihoff]

Thanks a lot, Phil, for looking into this and enabling the required kernel options! This is really appreciated.

[mgrouch]

Hello,
Did it make it into recently released
raspberry pi kernel 20210727?

thanks

[popcornmix]

It's been in (64-bit) kernel builds since 16 June.

[mgrouch]

I have arm64 bit kernel 5.10.52-v8+

There is /dev/ashmem
but no
/dev/binder

What is the correct procedure to enable them?

Thanks

[mgrouch]

@choff
@chrihoff

Christian,

I've built an OS image with 64-bit kernel and armhf user space.
https://bareboat-necessities.github.io/my-bareboat/bareboat-os.html
I'd love to build (at least experimental) support for anbox.
Have you got it running by any chance in this kind of set up?

Thanks a lot,
--MG

[Botspot]

I have arm64 bit kernel 5.10.52-v8+

There is /dev/ashmem but no /dev/binder

What is the correct procedure to enable them?

Thanks

According to here, you can enable /dev/binder with these commands:

sudo mkdir /dev/binderfs
sudo mount -t binder binder /dev/binderfs