Kotlin Multiplatform

[serpro69]

Discussion around adding mutiplatform support to kotlin-faker, which is done in #226

[tamimattafi]

@serpro69 Have you ever tried building the project for macOS aarch64?
I'm getting this error while syncing the project with JDK 11 or 17

Build file '/Users/tamimattafi/Projects/kmm/kotlin-faker/cli-bot/build.gradle.kts' line: 64

Failed to calculate the value of task ':cli-bot:compileJava' property 'javaCompiler'.
> No matching toolchains found for requested specification: {languageVersion=8, vendor=matching('Temurin'), implementation=vendor-specific} for MAC_OS on aarch64.
   > No locally installed toolchains match and toolchain download repositories have not been configured.

Line 64 of cli-bot/build.gradle.kts is

"Class-Path" to project.configurations.compileClasspath.get().joinToString(" ") { it.name },

Commenting this line doesn't hide the issue, it's still present but with line 143 (The end of the file)

Changing IDE's JDK to 1.8 throws another error

A problem occurred configuring project ':cli-bot'.
> Could not resolve all files for configuration ':cli-bot:classpath'.
   > Could not resolve org.graalvm.buildtools:native-gradle-plugin:0.10.1.
     Required by:
         project :cli-bot > org.graalvm.buildtools.native:org.graalvm.buildtools.native.gradle.plugin:0.10.1
      > No matching variant of org.graalvm.buildtools:native-gradle-plugin:0.10.1 was found. The consumer was configured to find a library for use during runtime, compatible with Java 8, packaged as a jar, and its dependencies declared externally, as well as attribute 'org.gradle.plugin.api-version' with value '8.4' but:
          - Variant 'apiElements' capability org.graalvm.buildtools:native-gradle-plugin:0.10.1 declares a library, packaged as a jar, and its dependencies declared externally:
              - Incompatible because this component declares a component for use during compile-time, compatible with Java 11 and the consumer needed a component for use during runtime, compatible with Java 8
              - Other compatible attribute:
                  - Doesn't say anything about org.gradle.plugin.api-version (required '8.4')
          - Variant 'javadocElements' capability org.graalvm.buildtools:native-gradle-plugin:0.10.1 declares a component for use during runtime, and its dependencies declared externally:
              - Incompatible because this component declares documentation and the consumer needed a library
              - Other compatible attributes:
                  - Doesn't say anything about its target Java version (required compatibility with Java 8)
                  - Doesn't say anything about its elements (required them packaged as a jar)
                  - Doesn't say anything about org.gradle.plugin.api-version (required '8.4')
          - Variant 'runtimeElements' capability org.graalvm.buildtools:native-gradle-plugin:0.10.1 declares a library for use during runtime, packaged as a jar, and its dependencies declared externally:
              - Incompatible because this component declares a component, compatible with Java 11 and the consumer needed a component, compatible with Java 8
              - Other compatible attribute:
                  - Doesn't say anything about org.gradle.plugin.api-version (required '8.4')
          - Variant 'sourcesElements' capability org.graalvm.buildtools:native-gradle-plugin:0.10.1 declares a component for use during runtime, and its dependencies declared externally:
              - Incompatible because this component declares documentation and the consumer needed a library
              - Other compatible attributes:
                  - Doesn't say anything about its target Java version (required compatibility with Java 8)
                  - Doesn't say anything about its elements (required them packaged as a jar)
                  - Doesn't say anything about org.gradle.plugin.api-version (required '8.4')

Searching for a solution.

[serpro69]

Here's a git diff for excluding the cli module altogether. I've just tested it myself and I can build everything else with just jdk8.

diff --git a/bom/build.gradle.kts b/bom/build.gradle.kts
index cde7bffe..086d5c12 100644
--- a/bom/build.gradle.kts
+++ b/bom/build.gradle.kts
@@ -26,7 +26,7 @@ val isRelease = provider {
 
 // Exclude subprojects that will never be published so that when configuring this project
 // we don't force their configuration and do unnecessary work
-val excludeFromBom = listOf(":cli-bot", ":docs", ":extension", ":faker", ":test", ":extension:kotest-property-test")
+val excludeFromBom = listOf(/*":cli-bot", */":docs", ":extension", ":faker", ":test", ":extension:kotest-property-test")
 fun projectsFilter(candidateProject: Project) =
     excludeFromBom.none { candidateProject.path == it }
         && candidateProject.name != bom.name
diff --git a/build.gradle.kts b/build.gradle.kts
index 07e444ba..52e20c3d 100644
--- a/build.gradle.kts
+++ b/build.gradle.kts
@@ -172,5 +172,5 @@ tasks.withType<TagTask>().configureEach {
 }
 
 apiValidation {
-    ignoredProjects += listOf("bom", "cli-bot", "docs", "test")
+    ignoredProjects += listOf("bom", /*"cli-bot", */"docs", "test")
 }
diff --git a/settings.gradle.kts b/settings.gradle.kts
index e8b4fb2b..0b8c676d 100644
--- a/settings.gradle.kts
+++ b/settings.gradle.kts
@@ -15,7 +15,7 @@ rootProject.name = "kotlin-faker"
 include(
     ":bom",
     ":core",
-    ":cli-bot",
+    // ":cli-bot",
     ":docs",
 )
 

Try if this works for you, and if so, then I think we can just keep the cli ignored for now.

[tamimattafi]

@serpro69
Yes, I tried to compile from master without any multiplatform setup. I tried to exclude the cli-bot from settings.gradle.kts but it seems like that wasn't enough, I will try it with your instruction.

As for JDK versions, I will take a look on the prerequisites.
I will keep this thread updated.

[serpro69]

In short, the problem with cli module is that it uses a graal gradle plugin that seems to require jdk version >= 11 (there's no mention of that in the docs that I could see, but I couldn't get it to work with java8)
Graal-CE distribution itself starts from version 17. But the gradle process needs to be running with java 11 because that's maximum supported version for a plugin in the docs module. So it's a small mess because of these hard dependencies on different java versions....

I'm now starting to look into refactoring docs and getting rid of the old unsupported framework that is currently used to build documentation. But that will only solve half of the problem and CLI would still need to be tackled sooner or later. I'm not sure yet what to do with it, so I'll have to take a look into it once I have some time. But since it's not part of the library itself, and isn't that critical, I think it should be OK to just ignore it for now so it doesn't become a blocker. If need be, I can just keep compiling it locally for awhile, in worst case.

What version of java do you have locally btw? And what vendor? I think this could also be causing some issues if you have a different vendor locally:

kotlin-faker/cli-bot/build.gradle.kts

Line 54 in d9cd70a

vendor.set(JvmVendorSpec.matching("Temurin"))

(I've just removed this part, so it should't be tied to a specific vendor anymore. Have no idea why I added that... and seems like that's the only place where vendor was specified explicitly)

But anyways, just try to exclude cli completely, and if that works - let's keep it excluded for now.

[tamimattafi]

@serpro69 Having JDK installed as mentioned in the prerequisites fixed the problem, and even tho there's an issue on my machine with graalvm-17, the sync completed successfully. Thank you for the tips.

I will study the build config and start adding the multiplatform plugin this week, hopefully on the weekend I will have something to show.

[serpro69]

Awesome! Feel free to post more questions here if/when you need further help.

[tamimattafi]

@serpro69 Hello!
Unfortunately, I didn't make any significant progress this week due to tight time at work. However, I studied the build config and made some observations, which might be important for further steps.

1. The current config uses maven-publish to upload artifacts to mavenCentral, however, this might not work seamlessly for Kotlin Multiplatform projects. I suggest using com.vanniktech.maven.publish plugin since it handles all artifact types. Here's an example of how to set it up: here
2. It might be worth it to move from buildSrc to includeBuild and break plugins into separate modules, for example: here. This will make build logic cleaner and more flexible
3. Not sure about the use-case differences between faker-lib and faker-ext conventions, what is each of them used for?
4. What is faker-provider conventions used for?

The first step will require implementing kotlin multiplatform plugin with basic targets, and move all the current logic from the main source set to jvmMain, and test to jvmTest (Basically rename the directories). This will lay the foundation for creating new source sets such as commonMain and move shared logic eventually. However, this should be done and merged quickly to avoid potential conflicts.

[tamimattafi]

@serpro69 It seems like implementing the multiplatform plugin is not as easy as I thought, I tried doing simple changes, and I found my self constantly breaking stuff in other build scripts and plugins 😂

Maybe you will have better luck since you know the config better? At the moment, all you need is to do these changes while keeping everything working as supposed:

1. Change kotlin("jvm") to kotlin("multiplatform") basically everywhere
2. The java { } block will be available since the multiplatform plugin provides JavaPluginExtension, therefore you might not need java-library anymore
3. Add the multiplatform configuration with jvm target only (For now)

plugins {
    kotlin("multiplatform")
}

kotlin {
    applyDefaultHierarchyTemplate()
    jvmToolchain(8) // Mabye consider using a newer version
    
    // Targets
    jvm()
}

4. If you have src/main/java (I only found this in the :core module) then you might consider renaming it to src/main/kotlin since it won't be available as a source set with the new plugin (Maybe you can add it manually using srcDir)
5. Your main source set will work just fine since we only have a jvm target, but ideally you should rename it to jvmMain and use the new dependencies block as shown below:

plugins {
    id("faker-lib-conventions")
}

kotlin {
    sourceSets {
        jvmMain.dependencies { 
            implementation("...")
            api("...")
        }
     }
}

6. As in the previous point, the same is correct for the test source set, but ideally it should become jvmTest and use the new dependencies block jvmTest.dependencies. But If you need more control over a source set, you can do something like this:

sourceSets {
    val jvmTest by getting {
        dependsOn(jvmMain)
        dependencies {
            // ...
        }
    }
}

That's basically it, after that you should consider creating commonMain source set and start moving pure Kotlin logic step by step, and later on, you might add more targets and use expect / actual for platform specific logic.

I recently moved a simple library to multiplatform, which can be taken as an example. We might have different approaches when it comes to build logic, but I'm sure you will find the diff useful. Here's the pull request (You can check the second commit only since it has the most important changes) mr0xf00/easycrop#12

As for publishing, in this step you probably won't need to change anything since having only jvm target doesn't change publication artifacts. But if you have spare time, consider moving to the plugin that I mentioned earlier, especially if you are considering migration to the new Central Portal with minimal efforts.

I would be happy to answer your questions if you have any

[serpro69]

Thanks for the inputs and details @tamimattafi . Yeah, I can definitely try :)
In fact, I've started doing something similar in another lib which has a simpler setup, just to learn multiplatform stuff - serpro69/semver.kt#31 .

[tamimattafi]

@serpro69, I can take a look at the Pull Request and give some thoughts when you are done, if you don't mind :D

[serpro69]

@tamimattafi not at all, thanks! I'll post it here or mention you in the PR once it's ready, but don't expect anything very soon - a bit busy at work at the moment :D

[tamimattafi]

a bit busy at work at the moment :D

I couldn't relate more 🥲