Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Native: Support Kotlin/Native in MockK #58

Open
olonho opened this issue Apr 8, 2018 · 32 comments

Comments

@olonho
Copy link

commented Apr 8, 2018

For multiplatform projects it would be really nice if MockK would support Kotlin/Native as well. If some missing functionality on K/N side is needed, please let us know, and we will add it.

@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Apr 8, 2018

I wonder how that can be done technically.

Does KN support any kind of instrumentation?

We need at least somehow inject a check of boolean in each method that defines if the function should be intercepted or something alike. This would be compile-time instrumentation/compiler plugin, of course, the runtime is even better. I've heard Kotlin way is to write compiler plugins

@olonho how is it done?

@olonho

This comment has been minimized.

Copy link
Author

commented Apr 8, 2018

Runtime instrumentation is indeed likely out of the question. Unless we will do something like https://en.wikipedia.org/wiki/DTrace with lightweight probes.

Compile-time wise, there are options. Most obvious one is a source transformation, but it isn't really nice. Another one is to implement compiler plugins API, and do something like https://github.com/JetBrains/kotlin-native/blob/master/backend.native/compiler/ir/backend.native/src/org/jetbrains/kotlin/backend/konan/lower/TestProcessor.kt on IR.

I think plugin compiler API is most sensible option, we can try to implement an extension point which runs an additional pass, if certain plugin is available. Do you find acceptable option to implement simple IR transformer?

@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Apr 8, 2018

Yes, IR transformer seems to be the way to go.

One more possible solution is to dynamically intercept functions after code generation i.e. embedding some assembly code on the fly by using additional information that compiler provides and dispatching only needed functions. But this seems to be hard. (I believe it is similar to DTrace, just another level)

Many details and design decision to be discussed if IR transformer is selected.

I'll sketch just a few:

  • mocking can be done to any class in runtime. should we instrument all functions? what can be done better?
  • need instrumentation that can call original functions and return result instead of it. how to do that?
  • how to achieve minimum performance impact?
  • should compiler build transformed version only for testing? is there such possibility?
  • should be supported other mocking frameworks? sooner or later such interference may appear.
@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Apr 8, 2018

In general, I can admit that most of the functions of mockk itself are transferred to common modules. Besides implementing the interface for function interception there are no hard tasks to perform for porting it to any platform.

Additionally, I struggle with following problems:

  • JS support is experimental because it works quite badly with default parameters which are not supported by JS Proxy class. It would be nice to have solution for multiplatform mocking at the end.
  • some things users would like to have extra, alike mocking inline functions
  • Ideally I would like to fully support Kotlin for all platforms(Java, JS, Native, Android device testing) and possible features without any Java agents or Proxies, just via the processor, and only for legacy use additional Java agents and Proxies.
@olonho

This comment has been minimized.

Copy link
Author

commented Apr 9, 2018

mocking can be done to any class in runtime. should we instrument all functions? what can be done better?
Probably, for Native we could compile with a special option, and insert call to some uniform listener with callee information. Then, actual behavior can be overiden in the runtime (slow, but will work).
Another option is limit mockability scope to only those classes seen by the compiler.

need instrumentation that can call original functions and return result instead of it. how to do that?

This shall be straightforward (as in test runner above) - you can create arbitrary complex IR instead of the original function.

how to achieve minimum performance impact?

Without dtrace-like mechanisms, I presume it will slow down execution anyway, but smartness may come from the way of calling mocked function (smth like dynamic table of invocation targets, where either original implementations lie, or we could replace it with custom pointer when mocking).

With that design, when compiled with the dynamic targets table, we could create following additional structures:

  • map of function name to an index in tables below (i.e. list of (const char*, index) pairs) - name table (NT)
  • original function pointers table (OFP)
  • actual function pointers table (AFP)

On call site of every function we call indirectly via the actual function table. Mockers can just modify this table, using elements of the original table to figure out where to call if need original implementation. This would kill performance (every call is indirect, and no inlining), but could allow rather flexible mocking without too deep integration into compiler. WDYT?

@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Apr 9, 2018

Performance is a difficult thing to reason as many things might be optimized on CPU level e.t.c. Without performance testing and optimization I don't think we can prove anything here in discussion.

Dynamic tables is kind of classics but require I believe more compiler/runtime rewrite.

Better do just interception via similar approach i.e. have a table that is checked at the beginning of every function and if it is filled-in then special call to mocking framework is performed:

  • call sites don't need to know anything about it
  • can be implemented at a higher level IR (in case you have such). best IMHO if such approch would be portable between platforms.
  • as dynamic call can be two level: class lookup and then function. good performance because do just one check at the begning and table of classes can be cached, this assumes most classes are not mocked.
  • can support mocking inline functions
  • it should translate to few additional asm instructions with guaranteed branch prediction if done carefully. (hope so)
@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Apr 10, 2018

Although one thing that is better with dynamic dispatch is that it allows calling original methods, which is required by spies. See no way now to do it with interceptors without overcomplication of it.

oleksiyp added a commit that referenced this issue Apr 10, 2018
@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Apr 10, 2018

@olonho I have following error here:

* Where:
Build file '/home/oleksiyp/workspace/mockk/dsl/native/build.gradle' line: 24

* What went wrong:
A problem occurred evaluating project ':mockk-dsl-native'.
> tried to access method org.jetbrains.kotlin.gradle.plugin.KotlinPlatformImplementationPluginBase$Companion.whenEvaluated(Lorg/gradle/api/Project;Lkotlin/jvm/functions/Function1;)V from class org.jetbrains.kotlin.gradle.plugin.KotlinNativePlatformPlugin

idea.log

@olonho

This comment has been minimized.

Copy link
Author

commented Apr 11, 2018

We're looking on this issue.

@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Apr 11, 2018

@olonho I use following kotlin plugin:

buildscript {
    ext.kotlin_version = '1.2.21'
    repositories {
        mavenCentral()
        jcenter()
    }
    dependencies {
        classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
    }
}
@vvlevchenko

This comment has been minimized.

Copy link

commented Apr 11, 2018

workaround for you issue could be following:

  • switch to kotlin-plugin 1.2.20 (because kotlin-native plugin depends on this version)
  • add kotlin-native plugin dependency to main build.gradle
@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Apr 11, 2018

Thanks. Will try this evening

oleksiyp added a commit that referenced this issue Apr 11, 2018
@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Apr 11, 2018

As in my project DSL is extracted to specific module I need to port it first. Added dsl/CMakeLists.txt with common part + InternalPlatform borrowed from JS. Now it fails with following error:

exception: java.lang.AssertionError: Built-in class kotlin.Function23 is not found
	at org.jetbrains.kotlin.builtins.KotlinBuiltIns.getBuiltInClassByName(KotlinBuiltIns.java:431)
	at org.jetbrains.kotlin.builtins.KotlinBuiltIns.access$700(KotlinBuiltIns.java:57)
	at org.jetbrains.kotlin.builtins.KotlinBuiltIns$4.invoke(KotlinBuiltIns.java:141)
	at org.jetbrains.kotlin.builtins.KotlinBuiltIns$4.invoke(KotlinBuiltIns.java:138)
	at org.jetbrains.kotlin.storage.LockBasedStorageManager$MapBasedMemoizedFunction.invoke(LockBasedStorageManager.java:408)
	at org.jetbrains.kotlin.storage.LockBasedStorageManager$MapBasedMemoizedFunctionToNotNull.invoke(LockBasedStorageManager.java:483)
	at org.jetbrains.kotlin.builtins.KotlinBuiltIns.getBuiltInClassByName(KotlinBuiltIns.java:424)
	at org.jetbrains.kotlin.builtins.KotlinBuiltIns.getBuiltInClassByName(KotlinBuiltIns.java:466)
	at org.jetbrains.kotlin.builtins.KotlinBuiltIns.getFunction(KotlinBuiltIns.java:561)
	at org.jetbrains.kotlin.builtins.FunctionTypesKt.createFunctionType(functionTypes.kt:199)
	at org.jetbrains.kotlin.builtins.FunctionTypesKt.createFunctionType$default(functionTypes.kt:194)
	at org.jetbrains.kotlin.builtins.SuspendFunctionTypesKt.transformSuspendFunctionToRuntimeFunctionType(suspendFunctionTypes.kt:57)
	at org.jetbrains.kotlin.serialization.KonanDescriptorSerializer.type(KonanDescriptorSerializer.kt:501)
	at org.jetbrains.kotlin.serialization.KonanDescriptorSerializer.typeId(KonanDescriptorSerializer.kt:475)
	at org.jetbrains.kotlin.serialization.KonanDescriptorSerializer.typeParameter(KonanDescriptorSerializer.kt:465)
	at org.jetbrains.kotlin.serialization.KonanDescriptorSerializer.functionProto(KonanDescriptorSerializer.kt:281)
	at org.jetbrains.kotlin.serialization.KonanDescriptorSerializer.classProto(KonanDescriptorSerializer.kt:111)
	at org.jetbrains.kotlin.backend.konan.serialization.KonanSerializationUtil.serializeClass(KonanSerializationUtil.kt:173)
	at org.jetbrains.kotlin.backend.konan.serialization.KonanSerializationUtil.serializeClasses(KonanSerializationUtil.kt:193)
	at org.jetbrains.kotlin.backend.konan.serialization.KonanSerializationUtil.serializePackage(KonanSerializationUtil.kt:219)
	at org.jetbrains.kotlin.backend.konan.serialization.KonanSerializationUtil.serializeModule$backend_native_compiler(KonanSerializationUtil.kt:262)
	at org.jetbrains.kotlin.backend.konan.KonanDriverKt$runTopLevelPhases$3.invoke(KonanDriver.kt:88)
	at org.jetbrains.kotlin.backend.konan.KonanDriverKt$runTopLevelPhases$3.invoke(KonanDriver.kt)
	at org.jetbrains.kotlin.backend.konan.PhaseManager$phase$$inlined$with$lambda$1.invoke(KonanPhases.kt:140)
	at org.jetbrains.kotlin.backend.konan.PhaseManager$phase$$inlined$with$lambda$1.invoke(KonanPhases.kt:119)
	at org.jetbrains.kotlin.backend.konan.util.UtilKt.profileIf(util.kt:34)
	at org.jetbrains.kotlin.backend.konan.PhaseManager.phase$backend_native_compiler(KonanPhases.kt:139)
	at org.jetbrains.kotlin.backend.konan.KonanDriverKt.runTopLevelPhases(KonanDriver.kt:84)
	at org.jetbrains.kotlin.cli.bc.K2Native.doExecute(K2Native.kt:61)
	at org.jetbrains.kotlin.cli.bc.K2Native.doExecute(K2Native.kt:39)
	at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.java:108)
	at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.java:52)
	at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:92)
	at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:70)
	at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:36)
	at org.jetbrains.kotlin.cli.common.CLITool$Companion.doMainNoExit(CLITool.kt:157)
	at org.jetbrains.kotlin.cli.common.CLITool$Companion.doMain(CLITool.kt:148)
	at org.jetbrains.kotlin.cli.bc.K2Native$Companion$main$1.invoke(K2Native.kt:183)
	at org.jetbrains.kotlin.cli.bc.K2Native$Companion$main$1.invoke(K2Native.kt:174)
	at org.jetbrains.kotlin.backend.konan.util.UtilKt.profileIf(util.kt:34)
	at org.jetbrains.kotlin.backend.konan.util.UtilKt.profile(util.kt:29)
	at org.jetbrains.kotlin.cli.bc.K2Native$Companion.main(K2Native.kt:176)
	at org.jetbrains.kotlin.cli.bc.K2NativeKt.main(K2Native.kt:188)
	at org.jetbrains.kotlin.cli.utilities.MainKt.main(main.kt:27)

I have functions of 22 args + continatuion arg in API.kt:

inline fun <reified T : suspend (A1, A2, A3, A4, A5, A6, A7, A8, A9, A10, A11, A12, A13, A14, A15, A16, A17, A18, A19, A20, A21, A22) -> R, R, A1, A2, A3, A4, A5, A6, A7, A8, A9, A10, A11, A12, A13, A14, A15, A16, A17, A18, A19, A20, A21, A22> CapturingSlot<T>.coInvoke(

Is it supported?

oleksiyp added a commit that referenced this issue Apr 11, 2018
@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Apr 11, 2018

Wow! After fixing this error DSL compiled in CLion and there is a possibility to run it till the moment actual implementation is needed. Optimistically tomorrow evening will port main implementation. After that only method interception is left.

@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Apr 12, 2018

@olonho @vvlevchenko what to do with following message?

error: the feature "multi platform projects" is experimental and should be enabled explicitly

Is it possible to turn it on in CMakeLists.txt?

@olonho

This comment has been minimized.

Copy link
Author

commented Apr 12, 2018

To enable MPP please use -Xmulti-platform compiler option. But for build we'd recommend to use Gradle, not CMake.

@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Apr 12, 2018

Then how I'll get IDE support if I use gradle?

@olonho

This comment has been minimized.

Copy link
Author

commented Apr 12, 2018

Currently (for short period of time, we hope) it is somewhat problematic if you want IDE for Native-specific components, for common parts IDEA will work, and could be used in non-intelligent mode to edit Native-specific files as well. Longer term we will go Gralde way, so I just suggest to take that into account.

@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Apr 12, 2018

And what about "suspend" 22 arg lambda? Should I raise ticket somewhere?

@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Apr 12, 2018

I ported boilerplate implementation code. output

Following things are marked as TODO() and required to be provided by platform/implementation:

  • proxy for function interception (this will be replaced by dynamic dispatch)
  • instantiation empty object of an arbitrary class
  • random
  • identity hash code
  • copying all fields of from one object to another (used in spies)

As well to mock private properties / fun I use reflective calls. For coroutines I need runBlocking, don't know if it is supported.

Additionally, my CLion is not indexing any content except I already visited, even simple serach is not working.

oleksiyp added a commit that referenced this issue Apr 12, 2018
@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Apr 12, 2018

Besides that I added println('hi') statement to SuspendFunctionsLowering and compiled mockk using new compiler. It is working!

I can say that without knowledge of IR it is quite difficult to understand what each lowering does. I tried to figure out for Autoboxing because it is small and seems even that is not so easy

@olonho

This comment has been minimized.

Copy link
Author

commented Apr 13, 2018

Regarding 23-parameters functions - guess this is will be fixed generally with https://youtrack.jetbrains.com/issue/KT-13764 in all Kotlin flavours.

Regarding IR transformation, likely what you may need is somewhat a mix of https://github.com/JetBrains/kotlin-native/blob/master/backend.native/compiler/ir/backend.native/src/org/jetbrains/kotlin/backend/konan/lower/BridgesBuilding.kt
and
https://github.com/JetBrains/kotlin-native/blob/master/backend.native/compiler/ir/backend.native/src/org/jetbrains/kotlin/backend/konan/lower/TestProcessor.kt
Autoboxing is not exactly what you need, as it modifies callsites, while bridge builder processes callees.

oleksiyp added a commit that referenced this issue Apr 15, 2018

@oleksiyp oleksiyp changed the title Support Kotlin/Native in MockK Native: Support Kotlin/Native in MockK May 16, 2018

@oleksiyp oleksiyp added this to To do in MockK features Dec 25, 2018

@ScottPierce

This comment has been minimized.

Copy link

commented Feb 13, 2019

I've been hesitant to let my team use mockk, despite hearing good things, since it's unclear if this library will ultimately support kotlin/native. Is this still something that's planned for mockk?

@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Feb 14, 2019

@ScottPierce I wouldn't rely on that. Both Native and JS need better support for reflection first. Don't know if it is planned somehow. And even together with that it is big challenge

@charleskorn

This comment has been minimized.

Copy link

commented Feb 14, 2019

The Kotlin/Native team have said a couple of times they won't support reflection like what is possible on the JVM. Would it be possible to support MockK with a plugin for the Kotlin compiler?

(reference: https://discuss.kotlinlang.org/t/reflection/4054/12)

@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Feb 14, 2019

I dont want to say impossible, but it is near that

@sdeleuze

This comment has been minimized.

Copy link

commented Feb 14, 2019

Mockk team could maybe propose some ideas to Kotlin/Native team based on how other native technologies handle mocking? For example FakeIt seems to provide Mockk like features for C++11. See also this comparison of Rust mocking frameworks.

@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Feb 14, 2019

@sdeleuze thanks for links.

For me learning deepest details about C++/Asm/Rust is not easy. Although I am not counting myself totally newbie in this topic.

I had a hope that somebody will join MockK community to lead this direction. As well I made an attempt to write compiler plugin in oleksiyp/kotlin-native#1, but stuck at some hard error. Besides that Kotlin/Native was supposed to be refactored that time. Don't know if it is finished.

So to sum up I am gathering courage/motiviation to do second attempt.

@oleksiyp

This comment has been minimized.

Copy link
Collaborator

commented Feb 15, 2019

Rust impress with mocking library fragmentation
screenshot_20190215-084118_chrome

@kpgalligan

This comment has been minimized.

Copy link

commented May 22, 2019

Bringing this back up. Will read through this again, but wondering if there are any more recent thoughts/efforts. I don't really know mockk well, but was thinking I'd start diving into compiler plugins on native.

@stale

This comment has been minimized.

Copy link

commented Jul 23, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. If you are sure that this issue is important and should not be marked as stale just put an important tag.

@stale stale bot added the stale label Jul 23, 2019

@JanStoltman

This comment has been minimized.

Copy link

commented Jul 24, 2019

@oleksiyp Could you perhaps add important tag to this issue, as it definitely is an important one.

@stale stale bot removed the stale label Jul 24, 2019

@oleksiyp oleksiyp added the important label Jul 24, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
8 participants
You can’t perform that action at this time.