Add iOS targets for kotlin mpp project

[Munzey]

Note: the ios() target is a shortcut to create targets for iosArm64(since ios11 all apps must be 64 bit) and iosX64(simulator) that will have any code in commonMain be depended on for code residing in iosMain.
See https://kotlinlang.org/docs/reference/building-mpp-with-gradle.html#target-shortcuts for details.

Note regarding publishing: the iOS artifacts must be compiled and deployed/published on a machine running macOS with full Xcode installed. see https://kotlinlang.org/docs/reference/building-mpp-with-gradle.html#using-kotlinnative-targets

[Munzey]

also just occurred to me, @michaelbull do you think it would be handy to add a section to the readme advertising that the lib is mpp-friendly?

[michaelbull]

the iOS artifacts must be compiled and deployed/published on a machine running macOS with full Xcode installed.

This is going to be problematic. Right now the published artifacts are built on my local PC (Fedora 31). The CI itself doesn't publish the packages, as they are signed with my GPG key. Not sure how to get around this.

[Munzey]

oof yep that's gonna make this tricky.
I would volunteer (macbook pro) but not sure how big a pain that would be to try to give me access rights to publish it.
I guess other option is to borrow one, or find a used mac mini for sale or something 😅

[Munzey]

ok there is one more option - rent a macos server: https://checkout.macincloud.com

looks like they do a prepaid 30$ for 30 hours plan, or 20$ to rent for a month 🤷

[Munzey]

could also be handy to get it added to this list: https://github.com/AAkira/Kotlin-Multiplatform-Libraries

[william-reed]

any updates on working something out for this? this is a blocker for me right now since I can't use it in my multiplatform project

[michaelbull]

No viable solutions have been proposed yet so unfortunately not

[yshrsmz]

About CI and deployment, GitHub Actions offers macOS environment.

[Munzey]

About CI and deployment, GitHub Actions offers macOS environment.

that's pretty cool! so It would be free for this repository: https://docs.github.com/en/free-pro-team@latest/github/setting-up-and-managing-billing-and-payments-on-github/about-billing-for-github-actions#about-billing-for-github-actions

interestingly for macOS though the time is multiplied by ten - which sounds more than enough if we set up a plan only for the master branch for publishing - 200 mins a month sounds plenty.

[Munzey]

ah actually maybe this wont work - if we need to install xcode in the environment, that would immediately exceed the limit for a free account (approx 12gbs):

The storage used by a repository is the total storage used by GitHub Actions artifacts and GitHub Packages

I think we'd need to run an action like: https://github.com/marketplace/actions/setup-xcode
so this would stop us going this direction. EDIT: not true, see latest comment.

On a side note, the original link I posted regarding needing full xcode installed is no longer there. It seems a lot of the documentation around the multiplatform plugin has been shuffled around.

If we only needed the command line tools (approx 120mbs), then this would be viable

[Munzey]

so turns out xcode comes preinstalled on the github action macos images 🤦
https://github.com/actions/virtual-environments/blob/main/images/macos/macos-11.0-Readme.md

so looks like this should be a good option!

Ill update this pr once #37 and #36 are merged

[michaelbull]

Can we somehow get the artifacts off of the machine that builds them? Ultimately they need signing before going onto maven central

[william-reed]

I believe github actions would just "attach" it's artifacts to the commit. So you could always download them and sign them if you would like, then upload to maven

…

On Fri, Nov 20, 2020, 07:58 Michael Bull ***@***.***> wrote: Can we somehow get the artifacts off of the machine that builds them? Ultimately they need signing before going onto maven central — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#23 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAT6ND5TDL4465QRJL7OSLLSQZRZBANCNFSM4OPZPCIA> .

[Munzey]

yep looks like there is this handy action: https://github.com/marketplace/actions/upload-a-build-artifact

they have a page on general stuff about artifacts too: https://docs.github.com/en/free-pro-team@latest/actions/guides/storing-workflow-data-as-artifacts

[msfjarvis]

yep looks like there is this handy action: https://github.com/marketplace/actions/upload-a-build-artifact

There's an official action here.

[msfjarvis]

Usage is pretty simple as well, you can see a live example here which is used to upload JUnit results when CI fails.

[yshrsmz]

I can't paste a link now but iirc sqldelight does signing and deployment with GitHub Actions

[william-reed]

I can't paste a link now but iirc sqldelight does signing and deployment with GitHub Actions

I believe @michaelbull needs to sign them manually since he uses his GPG key.

also if you can get this working on multiplatform, I believe this will be the first library to support it. Even the experimental kotlin.Result has bugs on multiplatform

[michaelbull]

I believe @michaelbull needs to sign them manually since he uses his GPG key.

Yeah so fundamentally right I am building releases on my local PC and using a GPG key (yubikey) to sign them. I would like to continue to do this because it removes any possibility of somebody hijacking my GitHub account and pushing a malicious release to maven central, which is a possibility if I put my GPG key somewhere in the repo itself and achieve CD.

[Munzey]

this is ready to merge again

[mayankmkh]

Any updates on this? I really want to use this library in KMM project.

[michaelbull]

Hi @mayankmkh. I think somebody suggested using github actions for building the artifacts on iOS, which was a potential solution, but no progress has been made on that yet. This is a prerequisite for them to be signed and uploaded to maven central, so as soon as that part of the puzzle is solved we should be good to go I think? Others in this thread may remember something I've forgotten though.

[mayankmkh]

Hi @michaelbull. I looked into Github Actions and as suggested in the comments above it should work. GPG signature file can be added as Github Secret and used inside the Github Action Workflow. There is an option to manually trigger the Workflow which accepts input and can be triggered from within Github or from an API call. For additional security, instead of saving the GPG key password in Secrets, the password can be passed as an input.
I don't have much experience with Github Actions so maybe others can give some more insight.

Edit: https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow

[michaelbull]

I've implemented the necessary GitHub action to automate deployments going forward. As you can see it's successfully publishing the JS artifacts now, so the missing piece going forward is for us to configure this job in the workflow to also run on macOS as well as ubuntu, at which point we can compile these artifacts and they will be automatically signed and published.

If somebody can finish that last piece off we can close this issue down and be a true MPP.

[Munzey]

@michaelbull nice work!
Ive pushed what I believe is the small required change - from reading the github action doc and into the actions in use, it seems the only necessary step was to move the publish job to a macos environment. Other than that, I believe this should mean we are good to go! 🚀

[mayankmkh]

Great work! 🚀 @michaelbull @Munzey
I also looked into some open-source multiplatform libraries, and all of them are using macOS image to publish them with exception of Linux and Windows artifacts. Some examples

• Korge
• Splitties

@Munzey I don't know if runs-on is case sensitive or not but Github suggests using macos-latest

[Munzey]

I actually copied that casing from the splitties project 😅

[michaelbull]

e: /Users/runner/work/kotlin-result/kotlin-result/kotlin-result-coroutines/src/commonTest/kotlin/com.github.michaelbull.result.coroutines/RunBlockingTest.kt: (11, 12): Expected function 'runBlockingTest' has no actual declaration in module <com.michael-bull.kotlin-result:kotlin-result-coroutines_test> for Native
> Task :kotlin-result-coroutines:compileTestKotlinIosX64 FAILED

[michaelbull]

@Munzey the iOS tests are failing because of a missing runBlockingTest declaration. Also I guess people are going to want windows artifacts as soon as we get the mac ones working, so maybe we should bite the bullet ahead of time and get that working in the same PR. We should probably be conditionally publishing the macOS artifacts when building on mac, and the windows ones on windows, etc.

This article on the subject looks useful.

[Munzey]

sorry yes will try take a look at this over next couple days 😓

[Munzey]

@michaelbull should be working now.

Wanted to mention that I'm not sure as a library whether we should be offering the multithreaded branch of coroutines to support multi threaded coroutines for iOS due to:

The current version of kotlinx.coroutines, which can be used for iOS, supports usage only in a single thread. You cannot send work to other threads by changing a dispatcher.

To switch the dependency we would just add-native-mt as listed here:
https://kotlinlang.org/docs/mobile/concurrency-and-coroutines.html#multithreaded-coroutines

However I'm not sure if that would be problematic downstream.

Also had a look at that doc for using matrix with build jobs - without showing a full script I found it kinda confusing how to get the yaml to not complain. I found splitties workflows to be a lot easier to follow: https://github.com/LouisCAD/Splitties/tree/main/.github/workflows

Either way, I think that's a larger change outside the scope of this pr (also I'd rather not commit that as I don't have access to the workflow to test it).

[mayankmkh]

@Munzey I think the non-multithreaded version should be used in the library as mentioned in the link provided by you.

Because the main version of kotlinx.coroutines is a single-threaded one, libraries will almost certainly rely on this version.

Also, user can simply override the dependency (mentioned in the link) as

implementation ("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.4.2-native-mt"){
    version {
        strictly("1.4.2-native-mt")
    }
}

How about adding this to README?

@michaelbull As many use cases for multiplatform targets mobile-only, maybe we can have a separate PR for other platforms?

[michaelbull]

Looks like it's working

There are two bits of follow on work before I want to consider making a release for this. We need to use a matrix of build targets, because right now the macOS build is taking 8 minutes just to run ./gradlew check, which is rather unacceptable.

Ideally we would:

• Run the cross-platform tests (i.e. ./gradlew check) on ubuntu
• Run only the iOS tests on macOS
• Publish the JVM artifacts on ubuntu
• Publish only the iOS artifacts on macOS

[Munzey]

@mayankmkh regarding the multi threading I think then maybe best to just leave as is, wait and see if someone encounters a problem with async behaviour on ios before attempting to change anything or document this. At least we are aware of the potential issue.

Other than that, any volunteers to cleanup the github actions script so we can get a release going?

[michaelbull]

I've got the cross platform tests running on ubuntu now and the iosX64Test task running on macOS, so that should help speed it up. If people want other platforms, they can adapt the workflow and added the necessary build targets in a separate PR.

[avently]

@Munzey @michaelbull could you, please, support LinuxX64, MingwX64 and MacosX64 as well?
Adapting native target and a build error guided me here

[michaelbull]

@avently we're open to pull requests to add other platforms - there are links in this issue's comments to help anyone that wants to pick that bit of work up