KTOR-7289: Initial support for android native targets #4236

whyoleg · 2024-08-21T18:35:01Z

Fixes https://youtrack.jetbrains.com/issue/KTOR-7289/Support-androidNative-targets

TODO:

needs kotlinx-html release with android native targets for ktor-server-html-builder
ISO_8859_1 charset is not supported - only UTF-8, because iconv is available only from Android API 28 (need hacks in cinterop)
there is a lot of copy-paste from nix mainly, because:
- commonizer doesn't work well in some cases and so it's not possible to use some declarations from common code (f.e in ktor-network
- it's not possible with current auto-module configuration via TargetsConfig class to setup, that in some modules androidNative targets should be like nix, while in others it should be on it's own

Otherwise, it compiles and links :)
Probably it could be tested via an approach shown in Kotlin/kotlinx-datetime#344

To test, run an Android emulator, and then, in the command line, ./gradlew androidNativeArm64TestBinaries && adb push core/build/bin/androidNativeArm64/debugTest/test.kexe /data/local/tmp/ && adb shell /data/local/tmp/test.kexe

Not sure still about the quality of those changes...
Feel free to suggest how to make it better.

But after this, it will be much simpler to depend on ktor in libraries which have all Kotlin/Native targets

Next thing on my list of "small contributions" is to check if it's possible to improve ktor build to be compatible with Configuration Cache.

vbarthel-fr · 2024-09-11T21:28:19Z

I'm not knowledgeable enough to give any constructive feedback on those changes, but since I successfully managed to start a ktor server from an android native binary thanks to this work (whyoleg/android-native), I'm writing a little "thanks a lot" message :)

I just had to modify the publication script so that I can publish androidNativeXXX ktor artifacts locally to make them accessible from my test project.

diff --git a/buildSrc/src/main/kotlin/Publication.kt b/buildSrc/src/main/kotlin/Publication.kt
index 1b62c68b0..6d02ae1f9 100644
--- a/buildSrc/src/main/kotlin/Publication.kt
+++ b/buildSrc/src/main/kotlin/Publication.kt
@@ -47,6 +47,15 @@ fun isAvailableForPublication(publication: Publication): Boolean {
         "macosArm64"
     )
 
+    val androidPublication = setOf(
+        "androidNativeArm32",
+        "androidNativeArm64",
+        "androidNativeX64",
+        "androidNativeX86"
+    )
+
+    result = result || name in androidPublication
+
     result = result || (HOST_NAME == "macos" && name in macPublications)
 
     return result

Thanks again 🙏

whyoleg · 2024-09-24T10:48:30Z

Note: if this PR will be merged, TC configuration for publishing should be updated to include new targets

e5l · 2024-09-24T10:49:36Z

@osipxd, could you check when you have time?

osipxd

Thank you for the contribution!

I would try to reuse nix sources for androidNative targets. I've already tried to do it, but faced with the problem you described in the PR description when some posix functions used in SocketUtilsNix cannot be commonized.
I'm trying to fix this problem by splitting nix to nix32 and nix64. I'll let you know about results of this experiment.

osipxd · 2024-09-26T08:02:06Z

buildSrc/src/main/kotlin/NativeUtils.kt

 */

 import org.gradle.api.*

-fun Project.posixTargets(): List<String> = nixTargets() + windowsTargets()
+fun Project.posixTargets(): List<String> = nixTargets() + windowsTargets() + androidNativeTargets()


Should android native be a part of nixTargets as it is Unix-like OS? If so, we could drop almost all duplicating code.

In theory - it should be, but on practice because of the issues with commonizer there will be a lot of unresolved declarations in nix source-set :(

Probably the hierarchy should be similar to atomicfu: https://github.com/Kotlin/kotlinx-atomicfu/blob/92ff33b4319970b8ab164c61c8ebfb6a69702955/atomicfu/build.gradle#L154-L172

So:

posix (= native)

windows (= mingw)

nix

unix

linux

apple

androidNative

androidNative32

androidNative64

Here is the PR needed for easy hierarchy changes: #4349

osipxd · 2024-09-30T14:10:28Z

@whyoleg, I've managed to get it work without splitting into nix and unix source-sets.
Be careful, I've force-pushed into your branch.

whyoleg

Nice!
I do like how better it is now!
Thanks!

P.S. it's a bit strange to review my own PR :)

whyoleg · 2024-09-30T14:28:02Z

buildSrc/src/main/kotlin/NativeUtils.kt

-fun Project.nixTargets(): List<String> = darwinTargets() + linuxTargets()
+fun Project.nixTargets(): List<String> = darwinTargets() + linuxTargets() + androidNativeTargets()
+
+fun Project.supportsAndroidNative(): Boolean = project.targetIsEnabled("androidNative")


Q: probably the same should be done for watchOS above? In future PR (can do after this is merged)

Yes, it is a good idea, feel free to open a PR

buildSrc/src/main/kotlin/TargetsConfig.kt

whyoleg · 2024-09-30T14:32:51Z

ktor-server/ktor-server-plugins/ktor-server-html-builder/gradle.properties

+# Copyright 2014-2024 JetBrains s.r.o and contributors. Use of this source code is governed by the Apache 2.0 license.
+#
+
+# Needs support in kotlinx-html: https://github.com/Kotlin/kotlinx.html/pull/288


the PR is already merged and new versions need to be released
cc @e5l

...or-serialization/ktor-serialization-kotlinx/ktor-serialization-kotlinx-xml/gradle.properties

osipxd · 2024-10-01T12:41:09Z

Not everything is so smooth 🥲

Task ':ktor-utils:androidNativeArm32MetadataElements' uses this output of task ':ktor-utils:commonizeCInterop' without declaring an explicit or implicit dependency.

I can't reproduce this problem locally (because of gradle/gradle#27576 I suppose).

whyoleg · 2024-10-02T19:45:02Z

I can't reproduce this problem locally (because of gradle/gradle#27576 I suppose).

Same at my side, can't reproduce :(

Interesting, that the issue happens with the same target for which hierarchy API is wrong :)

osipxd · 2024-10-03T08:19:43Z

target for which hierarchy API is wrong

Do you mean withAndroidNative() and withApple() declared in groups containing other groups?

The same task is still failing :(

osipxd · 2024-10-03T08:21:16Z

ktor-utils/build.gradle.kts

@@ -7,7 +7,7 @@ plugins {
 }

 kotlin {
-    createCInterop("threadUtils", nixTargets()) {
+    createCInterop("threadUtils", nixTargets() - androidNativeTargets()) {


The problem exists only for this module where I excluded android native targets from C interop. Maybe it is related

we could try to split this interop per related targets.
E.g create threadUtilsLinux and threadUtilsDarwin with distinct sets of targets (or even different def files) - may be there is some bug in commonizer logic

Hey @osipxd!
I was able to overcome the issue here: bbd3151
Previously failed build is now green :)

whyoleg · 2024-10-03T08:35:49Z

Do you mean withAndroidNative() and withApple() declared in groups containing other groups?

Nope, I'm talking about the issue with withAndroidNativeArm32 - the same target task is failing on CI androidNativeArm32MetadataElements - not sure if it's related though

osipxd

Great job, thank you!
Created a task to update CI configuration: KTOR-7535

e5l self-requested a review August 26, 2024 08:05

whyoleg mentioned this pull request Sep 14, 2024

Support android native targets and wasm-wasi Kotlin/kotlinx.html#288

Merged

whyoleg force-pushed the android-native branch from 9f70a52 to c03a32d Compare September 15, 2024 18:26

whyoleg changed the title ~~Initial support for android native targets~~ KTOR-7289: Initial support for android native targets Sep 21, 2024

e5l requested review from osipxd and removed request for e5l September 24, 2024 10:49

osipxd reviewed Sep 26, 2024

View reviewed changes

osipxd force-pushed the android-native branch from 82f8f69 to 89fc8ee Compare September 30, 2024 14:00

whyoleg commented Sep 30, 2024

View reviewed changes

whyoleg force-pushed the android-native branch from c1781f6 to e650e07 Compare October 2, 2024 19:39

osipxd reviewed Oct 3, 2024

View reviewed changes

whyoleg and others added 6 commits October 5, 2024 22:04

Support for android native targets

6373e68

Update api dump

6b5bfcf

fixup! Support for android native targets

aeb91b0

fixup! Support for android native targets

431723a

Small cleanup

dbe27b6

hack to overcome issue with commonizer

bbd3151

whyoleg force-pushed the android-native branch from 50cdf83 to bbd3151 Compare October 5, 2024 19:05

whyoleg requested a review from osipxd October 5, 2024 19:40

osipxd approved these changes Oct 6, 2024

View reviewed changes

osipxd merged commit a34c260 into ktorio:main Oct 7, 2024
8 of 13 checks passed

whyoleg deleted the android-native branch October 12, 2024 11:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

KTOR-7289: Initial support for android native targets #4236

KTOR-7289: Initial support for android native targets #4236

whyoleg commented Aug 21, 2024 •

edited

Loading

vbarthel-fr commented Sep 11, 2024 •

edited

Loading

whyoleg commented Sep 24, 2024

e5l commented Sep 24, 2024

osipxd left a comment

osipxd Sep 26, 2024

whyoleg Sep 26, 2024

whyoleg Sep 26, 2024 •

edited

Loading

osipxd Sep 27, 2024 •

edited

Loading

osipxd commented Sep 30, 2024

whyoleg left a comment

whyoleg Sep 30, 2024

osipxd Oct 1, 2024

whyoleg Sep 30, 2024 •

edited

Loading

osipxd commented Oct 1, 2024 •

edited

Loading

whyoleg commented Oct 2, 2024

osipxd commented Oct 3, 2024

osipxd Oct 3, 2024 •

edited

Loading

whyoleg Oct 3, 2024

whyoleg Oct 5, 2024

whyoleg commented Oct 3, 2024

osipxd left a comment

KTOR-7289: Initial support for android native targets #4236

KTOR-7289: Initial support for android native targets #4236

Conversation

whyoleg commented Aug 21, 2024 • edited Loading

vbarthel-fr commented Sep 11, 2024 • edited Loading

whyoleg commented Sep 24, 2024

e5l commented Sep 24, 2024

osipxd left a comment

Choose a reason for hiding this comment

osipxd Sep 26, 2024

Choose a reason for hiding this comment

whyoleg Sep 26, 2024

Choose a reason for hiding this comment

whyoleg Sep 26, 2024 • edited Loading

Choose a reason for hiding this comment

osipxd Sep 27, 2024 • edited Loading

Choose a reason for hiding this comment

osipxd commented Sep 30, 2024

whyoleg left a comment

Choose a reason for hiding this comment

whyoleg Sep 30, 2024

Choose a reason for hiding this comment

osipxd Oct 1, 2024

Choose a reason for hiding this comment

whyoleg Sep 30, 2024 • edited Loading

Choose a reason for hiding this comment

osipxd commented Oct 1, 2024 • edited Loading

whyoleg commented Oct 2, 2024

osipxd commented Oct 3, 2024

osipxd Oct 3, 2024 • edited Loading

Choose a reason for hiding this comment

whyoleg Oct 3, 2024

Choose a reason for hiding this comment

whyoleg Oct 5, 2024

Choose a reason for hiding this comment

whyoleg commented Oct 3, 2024

osipxd left a comment

Choose a reason for hiding this comment

whyoleg commented Aug 21, 2024 •

edited

Loading

vbarthel-fr commented Sep 11, 2024 •

edited

Loading

whyoleg Sep 26, 2024 •

edited

Loading

osipxd Sep 27, 2024 •

edited

Loading

whyoleg Sep 30, 2024 •

edited

Loading

osipxd commented Oct 1, 2024 •

edited

Loading

osipxd Oct 3, 2024 •

edited

Loading