Android 12 | [Bug]: OpenJDK is broken due to tagged pointers

[ghost]

If you need OpenJDK in Termux - do not upgrade to Android 12.

Problem description

After upgrading to Android 12, workaround for tagged pointers in package openjdk-17 is not working anymore.

This means we will need to either wait fixes on upstream or fix broken packages manually which may require quite much effort, depending on codebase size.

Other packages known to be affected by issue with tagged pointers:

• swift - swift: some binaries do not work on Android 11 #6261
• st from x11-packages - https://github.com/termux/x11-packages/issues/407

What steps will reproduce the bug?

1. Upgrade to Android 12 or obtain such OS version through installing GSI.
2. Install the latest Termux app (it was 0.117 at time of reporting).
3. pkg upgrade && pkg install openjdk-17
4. Run javac without arguments or attempt to compile something.

System information

termux-info:

Application version:
0.117-gda6174e
Packages CPU architecture:
aarch64
Subscribed repositories:
# sources.list
deb https://termux.astra.in.ua/apt/termux-main stable main
Updatable packages:
All packages up to date
Android version:
12
Kernel build information:
Linux localhost 4.19.191-gc2161d44afae-ab7624114 #1 SMP PREEMPT Mon Aug 9 09:02:05 UTC 2021 aarch64 Android
Device manufacturer:
Google
Device model:
Pixel 5

[agnostic-apollo]

Maybe try adding android:allowNativeHeapPointerTagging="false" to AndroidManifest.xml of termux-app, as per https://source.android.com/devices/tech/debug/tagged-pointers. Although it mentions it only works with targetSdkVersion 30, but don't see any checks in AOSP from a quick look, but at least should work on android 12 with targetSdkVersion 30.

The logic for decision has changed in android 11 r40 vs master as per https://cs.android.com/android/_/android/platform/frameworks/base/+/15cce113f7232e117b091770c220ebc1b55f41d2 even though master is also 11.

https://cs.android.com/android/platform/superproject/+/android-11.0.0_r40:frameworks/base/services/core/java/com/android/server/am/ProcessList.java;l=1672

https://cs.android.com/android/platform/superproject/+/master:frameworks/base/services/core/java/com/android/server/am/ProcessList.java;l=1697

[finagolfin]

I wrote up why this was happening with Swift on the Swift forums in May: the issue is that Swift sets its own tag in the top byte and that collides with Android's system tagging. Presumably something similar is happening with OpenJDK.

[finagolfin]

It appears that hardware support for memory tagging will only roll out with the new ARMv9 CPUs in the next year, but that link does mention "Software support for using MTE is being introduced as part of Android 12."

Maybe Android 12 will require software tagging, even though most hardware that is still ARMv8 doesn't natively support it yet, to get everyone to update any software that uses its own tag in the top byte, causing collisions. In our case, that only appears to be these three packages, so not much?

[finagolfin]

I just upgraded to Android 12 and have no problem with tagged pointers. Maybe this will only be a requirement for new devices that ship with Android 12 from now on.

[Vixeliz]

Seems to work fine for me on my s21 ultra running android 12

[finagolfin]

Seems to work fine for me on my s21 ultra running android 12

That shipped last year with Android 11 though, right? As I noted above, they're not requiring devices that upgrade to Android 12 to enable MTE, though they may be doing so for new devices that ship with Android 12, like the recently released Pixel 6, for all we know.

[Vixeliz]

Oh sorry my bad in that case I hope this gets fixed by time I buy a new phone

[finagolfin]

I asked and was told that the Android team has decided to reserve the entire top byte on pointers in AArch64 devices, android/ndk#1653, which is apparently being slowly rolled out starting with Android 11. I'm working on a patch for Swift, where the runtime has always set the top bit on pointers to determine whether to reference-count them or not, to account for this Android change.

[MonsterDruide1]

Is there any workaround for this at the moment? I'm running Termux on my Galaxy Tab S7, which got updated to Android 12 recently... And I'd really want to run Java on there.

[finagolfin]

@MonsterDruide1, was it working fine with Android 11 on your Tab S7 before?

[MonsterDruide1]

I updated my tablet about half an hour before trying to use Java, so I'm not sure if it would have worked. The update has been published very recently by Samsung and upgrades to OneUI 4.0, but I'm not even sure anymore whether it had Android 11 or 12 before that specific update.

[finagolfin]

OK, was wondering if they switched tagged pointers on with the update... guess we'll never know. It appears that OpenJDK uses its own pointer tags that collides with Android's new memory tagging: I'm unaware of any JDK workaround other than rebuilding Termux yourself with the tagging flag turned off, as mentioned above.

[MonsterDruide1]

Okay, but that definitely already is a workaround I can try. Thanks for the quick response, and I'll report on whether it works later!

[MonsterDruide1]

No, this doesn't seem to work. I've added android::allowNativeHeapPointerTagging="false" in line 38 of the AndroidManifest, and as Android Studio complained about it, afterwards also tried increasing minimum and target SDK to 30 in gradle.properties. No change, it also crashes when just running javac, halfway through the help message.

[finagolfin]

I notice that you pasted two colons in that flag here, while every other flag uses one?

[MonsterDruide1]

Oops, that was just a typo in the comment here. It is properly written with a single colon in Android Studio.

Note, I've also tried upgrading the NDK version. There seem to be more issues with lifting the compile, target and minimum SDK to 31, but it's still crashing with them being at 30.

[MonsterDruide1]

Also, maybe an important thing:
It seems to be luck-based whether javac without options finishes printing the help message or if it gets canceled by Android before. Sometimes it shows all of the help message lines with no issues and no Aborted message, sometimes it gets interrupted halfway through, but always after the line of -profile <profile>.

[agnostic-apollo]

Ummm, just curious, which branch of termux-app are you using to test? Bumping targetSdkVersion in master should result in execution failure for any binary from app data directory unless its a symlink from apk native directory or selinux policy is patched.

[MonsterDruide1]

I'm using master... which one should I test instead?

[agnostic-apollo]

I am assuming that it worked because you upgraded an app from an older targetSdkVersion and android 10 restrictions didn't engage, maybe same reason android:allowNativeHeapPointerTagging didn't work. Uninstall and reinstall app with targetSdkVersion 30.

https://github.com/termux/termux-packages/wiki/Termux-and-Android-10

But if you were to bump version, even login shell would fail, you would need use updated-flavor branch and create bootstrap zip with openjdk-17 in it, since if its download from apt repo, it won't execute. The branch and generate-bootstrap would need some patches, since old now, and --android10 flag needs to be passed too to script.

https://github.com/termux/termux-packages/wiki/Termux-and-Android-10

termux/termux-app#2441

termux/termux-app#1983

[MonsterDruide1]

The libfiles.so should contain mapping of lib names to their real symlink that will be done in $PREFIX. The lib directory in the apk should contain the files, you can extract them.

Ohh, wait a second, there is no libfiles.so! There are only libtermux.so and the bootstrap-library. I think it's something about my gradle, where I just randomly changed things so far until it looked right to me, which is most likely wrong...

What would be the right setup here? I'm now trying to remove all the contents of downloadBootstrap, replacing them with just a call to expandBootstrap (which seems to be responsible for creating the .so above?). Let's see if that works.

Did u reinstall apk? For changes in bootstrap to take place, you need to have a fresh install. Installation is only done once.

Yes, I'm uninstalling after every attempt, and I'm doing a full clean rebuild of gradle after each change to either gradle files or bootstrap zips.

The whole point of updated flavor branch is so that all binaries exist inside the apk, otherwise execution isn't allowed by android. Welcome to the wannabe-security-android-design.

Oh. That's bad. Why would they even want to do that? If the developers of some app want to run malicious code, they can surely do that from within the apk file. No need for external code there, and I have no idea what else they would block with that. But that discussion is probably outside the scope of this issue.

The sh under $PREFIX is for dash shell. You need to call /system/bin/sh and likely set LD_LIBRARY_PATH to /system/lib64. It's execution will be allowed for all cases, but dash only if its inside apk.

I didn't really understand a lot of what you said there, I'm not too experienced with Linux... all I can say is that LD_LIBRARY_PATH is empty (echo $LD_LIBRARY_PATH = nothing). It's not important for me to run the shell, I just noted that javac did not work and tried sh for context... which also didn't work, so it's not a openjdk-specific issue.

I'm currently building for the simplified, updated gradle file, let's see if it works with that one.
That would be

def downloadBootstrap(String arch, String expectedChecksum, String version, boolean isPackagesInApk) {
    def file = new File(projectDir, "src/main/cpp/bootstrap-" + arch + ".zip");
    expandBootstrap(file, expectedChecksum, arch)
    return
}

[agnostic-apollo]

First you need to ensure you are building the updated flavor. Second you need to ensure your custom bootstrap gets added to apk instead of downloaded bootstrap. I am currently on mobile, so can't (harder to) review the code. You need to disable overwrite of custom bootstrap.

https://github.com/termux/termux-app/blob/60f07718880bdd8d241f3c13126343b0b406c818/app/build.gradle#L201

That would be

Yeah, should work.

If the developers of some app want to run malicious code, they can surely do that from within the apk file.

Pretty much, apks built on dev devices or servers already violate W^X. But yeah, off topic.

If you run sh, it will run $PREFIX/bin/sh (dash), not /system/bin/sh. PATH does not have /system/bin in termux by default. Exec restrictions do not apply to system binaries and only to binaries in app data.

FYI, for PATH and LD_LIBRARY_PATH, check https://github.com/agnostic-apollo/tudo#path-and-ld_library_path-priorities

[MonsterDruide1]

Check libfiles.so

Okay, there are now a LOT more .so files in that directory than before (13,319 vs. 2). libfiles.so contains javac:

lib6415.so?opt/openjdk/man/man1/javac.1.gz
lib6639.so?opt/openjdk/bin/javac

However, invoking stat is impossible due to sh: stat: inaccessible or not found.

[MonsterDruide1]

It seems like the encoding of ← did not work properly... any way to fix that?

[JayeshRocks]

Pointer tag for 0x6f43493930 was truncated, see 'https://source.android.com/devices/tech/debug/tagged-pointers'.
Aborted

Receiving this error after upgrading to Android 12 and using OpenJDK 17

[SwimmingTiger]

a new solution

• A complete guide to installing 32-bit java in 64-bit Termux shell

---

the old solution

If you just want to run java and don't care about performance, you only need to get a 32 bit java installation. Tagged pointers are entirely a 64-bit thing, so 32-bit java is completely unaffected.

As a proof of concept, I modified the termux apk, removed the arm64 library and re-signed. After reinstalling the app, I get a pure 32 bit shell. Now I can use java in Android 12.

Here's my modified apk, if you're interested:
termux_arm32_118.apk.zip (You don't need it. You can download termux-app_v0.118.0+github-debug_armeabi-v7a.apk here.)

Note that you need to uninstall the original termux to install it, your data will be lost.

There is obviously a better way to install it, but it's also more complicated (termux's openjdk-17:arm has too many dependencies and conflicts with 64-bit packages). The method mentioned above is the fastest.

[agnostic-apollo]

FYI, termux github builds are also supplied as arch specific so no need to modify apk.

https://github.com/termux/termux-app/releases/tag/v0.118.0

https://github.com/termux/termux-app#installation

[cxyzzz]

have a hack way to use OpenJDK in Android 12,like this but no need install 32bit Termux,just download 32bit OpenJDK deb file from Termux repo and extract deb file resource,then set JAVA_HOME env, using LD_LIBRARY_PATH=$JAVA_HOME/lib $JAVA_HOME/bin/java to run java,you will get some error ,read the error message ,then go to Termux repo download the corresponding 32bit lib file, move it to $JAVA_HOME/lib.

If you just want to run java and don't care about performance, you only need to get a 32 bit java installation. Tagged pointers are entirely a 64-bit thing, so 32-bit java is completely unaffected.

As a proof of concept, I modified the termux apk, removed the arm64 library and re-signed. After reinstalling the app, I get a pure 32 bit shell. Now I can use java in Android 12.

Here's my modified apk, if you're interested: termux_arm32_118.apk.zip

Note that you need to uninstall the original termux to install it, your data will be lost.

There is obviously a better way to install it, but it's also more complicated (termux's openjdk-17 has too many dependencies and conflicts with 64-bit packages). The method mentioned above is the fastest.

[SwimmingTiger]

have a hack way to use OpenJDK in Android 12,like this but no need install 32bit Termux,just download 32bit OpenJDK deb file from Termux repo and extract deb file resource,then set JAVA_HOME env, using LD_LIBRARY_PATH=$JAVA_HOME/lib $JAVA_HOME/bin/java to run java,you will get some error ,read the error message ,then go to Termux repo download the corresponding 32bit lib file, move it to $JAVA_HOME/lib.

A complete guide to installing 32-bit java in 64-bit Termux shell

From: https://hu60.cn/q.php/bbs.topic.102531.html (zhCN)

# View current package architecture
dpkg --print-architecture

# Add 32-bit arm architecture
dpkg --add-architecture arm

# show the added architecture
dpkg --print-foreign-architectures

# update package list
apt update

# Create and enter the deb folder
mkdir ~/deb
cd ~/deb

# Download `openjdk-17:arm`
apt download openjdk-17:arm

# Download all dependencies of `openjdk-17:arm`
LANG=en LANGUAGE=en apt-cache depends --recurse openjdk-17:arm | grep -E '(Depends|PreDepends|Recommends|Suggests):' | awk '{print $2}' | grep -v '<' | xargs apt download

# Create and enter the termux32 folder
mkdir ~/../termux32
cd ~/../termux32

# extract debs
find ~/deb/ -type f | while read f; do dpkg -X "$f" ~/../termux32; done

# Set a PREFIX32 environment variable
PREFIX32="$(realpath ~/../termux32/data/data/com.termux/files/usr)"

# Create a symlink for the deep directory
ln -s "$PREFIX32" ~/../usr32

# Create a wrapper for 32-bit programs

PREFIX32="$(realpath ~/../usr32)"

cat >"$PREFIX/bin/bin32_wrapper" <<EOF
#!$PREFIX/bin/sh

bin="\$(basename "\$0")"

export PREFIX="$PREFIX32"
export JAVA_HOME="\$PREFIX/opt/openjdk"
export PATH="\$PREFIX/bin:\$JAVA_HOME/bin"
export LD_LIBRARY_PATH="\$PREFIX/lib:\$JAVA_HOME/lib"

exec "\$PREFIX/bin/\$bin" "\$@"
EOF

# give execute permission
chmod +x "$PREFIX/bin/bin32_wrapper"

# Create wrapper link for 32-bit java
cd "$PREFIX/bin"
ls ~/../usr32/opt/openjdk/bin | while read f; do echo "$f"; ln -s bin32_wrapper "$f"; done

# Create a test java file
mkdir ~/java
cd ~/java

cat >./HelloWorld.java <<EOF
public class HelloWorld {
     public static void main(String args[]) {
         System.out.println("Hello World!");
     }
}
EOF

# compile it
javac HelloWorld.java

# run it
java HelloWorld

[easyaspi314]

It would be better to wrap the actual $JAVA_HOME/bin/* binaries. Programs like gradlew execute this directly instead of searching $PATH.

[dev-bz]

Disable tagged pointers for Android 12 & 11

[ericwomer]

I don't understand what the problem is, this has been known for over a year and no one has yet decided to create a patch to fix this?

[Grimler91]

I don't understand what the problem is, this has been known for over a year and no one has yet decided to create a patch to fix this?

Feel free to open a PR if you know how to fix it.

[Grimler91]

I don't understand what the problem is, this has been known for over a year and no one has yet decided to create a patch to fix this?

Feel free to open a PR if you know how to fix it.

Seems like this is the fix, #7332 (comment)

Sure, you can follow those instructions as a workaround (easy to do on affected devices), but a fix would preferably fix the root cause as done for swift.

[Leif-W]

One argument for a workaround is that a quick workaround to restore degraded functionality is better than waiting an indefinite amount of time with no functionality for a non-existent proper fix. Is there a strong argument against the workaround? E.g. does it lead to undefined behaviors, cause undue maintenance burden, etc?

[xtkoba]

Maybe I can borrow the idea of #7332 (comment) and write a patch. The problem is that I cannot test it on my own because I do not have access to any affected environment. I won't commit it until sufficiently tested by others.

[xtkoba]

Please test #10347. Thanks.

[licy183]

Seems working on my Android 12 phone (aarch64) running command javac.

[PiprTuff]

Any more update? Is this issue resolved?

[Leif-W]

Seems to be working? Compiled a few small .java files with various standard library imports. Even compiled a trivial Android app to an .apk, installed, and ran. Could use more rigorous testing.

[rocket-pig]

ohhhh... I get it. After 90 hours of thinking I was fixing java wrong, I came to understand that this still doesnt mean gradle wont fail with same error, for same reason.

..or maybe i dont get it. I probably dont get it.