TurboModules (NativeModules Re-architecture)

[fkgozali]

Introduction

This is a place for discussions around the upcoming "TurboModule" feature.

• Full proposal here: https://github.com/react-native-community/discussions-and-proposals/blob/master/proposals/0002-Turbo-Modules.md
• Previous discussions: #11

Terminology

• Fabric: just the UI layer re-architecture, to take full advantage of concurrent React architecture. (dedicated issue)
• JSI: JavaScript Interface, it's a unified lightweight general purpose API for (theoretically) any JavaScript virtual machine. It enables every other piece of the rearchitecture. (dedicated issue)
• CodeGen: a tool to "automate" the compatibility between JS and native side. (dedicated issue)

TL;DR

From @axe-fb's blogpost, here's a temporary description of TurboModules (please consider that this is not yet finalized, it may change in the future)

The JSI system can also be used to call leverage device capabilities like bluetooth or other sensors by exposing functions that JS can call. This is similar to how browsers expose functions like navigator.geolocation.getCurrentPosition that, when invoked in JavaScript, trigger the respective C++ call in the browser.
In the current system, a table with information about module names and methods is created. When JS calls a specific native module, the indices of the module and methods are passed to Java/ObjC, which then invoke the specific methods. The arguments and return values are also converted between JavaScript and JNI/ObjC objects.
[...] Now that we have a JSI object for "SampleTurboModule", can invoke methods on this JSI object from JavaScript. During the calls, we also need to convert JSI Values to JNI for argument parameters, and the reverse when sending back results.
Like in the current architecture, most types including boolean, strings, Maps, Arrays, Callbacks and Promises are supported.

Available Materials

At ReactConf 2018 @axe-fb did a talk about React Native's New Architecture, which also explains the 3 concepts above: JSI, Fabric, TurboModule.

IN Q1 2019, @kelset wrote a more high-level explanation in a blogpost: https://formidable.com/blog/2019/fabric-turbomodules-part-3/ and did a talk about the whole rearchitecture in April 2019 at React Edinburgh.

@kelset also did a more in-depth talk at React Advanced London in Oct 2019: youtube recording & slides.

Q&A

This is also a place for questions related to this effort and its direction.

[fkgozali]

cc @kelset - wanna label this one with "Transparency"? I don't have access looks like.

cc @axe-fb

[TheSavior]

@empyrical, this isn’t related to turbomodules, but rather one of the other projects I’m working on. This won’t need to be lazy because requireNativeComponent won’t exist and the only work that will be done is returning the string “RCTView”. There will be no round trips to native so it won’t provide value to make this async.

[tomduncalf]

Does this proposal (and related parts such as JSI) mean C++ native modules will become a first class, documented citizen of the RN universe?

That would be awesome, I’m doing a lot of work in this area and most of it had to be figured out by reverse engineering/trial and error.

[fkgozali]

Does this proposal (and related parts such as JSI) mean C++ native modules will become a first class, documented citizen of the RN universe?

Depends on what you meant by first class citizen. You can already build C++ class that talks directly to the VM via JSI's HostObject - all JSI code is already in master: https://github.com/facebook/react-native/blob/bf2500e38ec06d2de501c5a3737e396fe43d1fae/ReactCommon/jsi/jsi.h#L80

An example consumer is https://github.com/facebook/react-native/blob/94d49e544d10d0039a1178dc97533e96a4354198/ReactCommon/fabric/uimanager/UIManagerBinding.h

We plan to build a thin C++ wrapper on top of this direct access to HostObject to make the binding slightly easier in the future, but the first focus will be ObjC/Java modules. Documentation will come when they are all ready and stable. So yes, C++ modules will become first class in this angle.

[tomduncalf]

This is really interesting, thanks for the pointers. So far we've been working with CxxModule, looks like this could be a cleaner alternative!

Will have a proper look into it when time allows, we are doing some pretty interesting stuff with C++/React Native integration which pushes it fairly hard and could make a good test case for some of this stuff, so if you'd be interested in discussing in more detail please feel free to get in touch (details in Github bio)!

[an-kumar]

@fkgozali Do you mean in the current version of react native we can already build and register C++ classes using jsi's HostObject to the VM?

Are there any examples of this? I'm particularly unclear on where the registration would occur -- from outside of RN core.

[tomduncalf]

Yeah same question here - would be nice to know the recommended way to register a C++ JSI module

[fkgozali]

Do you mean in the current version of react native we can already build and register C++ classes using jsi's HostObject to the VM?

Yes. We're working on a cleaner abstraction, but at the moment you can try something like this at your own:

• iOS
  • provide RCTCxxBridgeDelegate for the bridge in your app
  • #import <React/RCTBridge+Private.h>
  • in the protocol impl for - (std::unique_ptr<facebook::react::JSExecutorFactory>)jsExecutorFactoryForBridge:(RCTBridge *)bridge
    • return nullptr;
    • before returning, get the jsi::Runtime off the bridge: jsi::Runtime *runtime = (jsi::Runtime *)bridge.runtime;
    • once you get the runtime, you can do whatever you want to install your jsi::HostObject to JS. Example: https://github.com/facebook/react-native/blob/acc2ed2488761cd3d63839d56cbf95079c8cd551/ReactCommon/turbomodule/core/TurboModuleBinding.cpp#L25-L38
• Android
  • Same deal as iOS, but get the runtime off catalystInstance.getJavaScriptContextHolder(), which you can pass down to JNI, then cast it to jsi::Runtime * like in iOS.

would be nice to know the recommended way to register a C++ JSI module

Expect more documentation when TurboModule is ready to use. We moved a bunch of code for iOS already to github, Android coming soon, but sample code will be provided at later time.

[tomduncalf]

Thanks very much - I was looking at this today but it would have taken me a while to figure that out :)

I noticed the Turbomodules stuff in github - seems to be adding a new layer of abstraction for this stuff? Would you say it’s better to develop against what is in master than what is in the RC branch if we want to have more of a chance of keeping close to the final API?

[fkgozali]

I noticed the Turbomodules stuff in github - seems to be adding a new layer of abstraction for this stuff?

That's correct. It's abstraction for crossing C++-->ObjC and C++-->Java. You don't need it if you want to stay in pure C++ via jsi::HostObject.

Would you say it’s better to develop against what is in master than what is in the RC branch if we want to have more of a chance of keeping close to the final API?

Nothing in master/RC is ready to use, but you can get the idea about where we're headed. If you for some reason needs to play with jsi::HostObject sooner, by all means go for it. If you're in no rush, I'd wait for more official guidelines/docs on how to use TurboModule.

[tomduncalf]

@fkgozali I wonder if you'd be able to advise me a little further on the Android side of this? I was able to get it up and running on iOS without too much trouble and it seems great, but on Android I'm having issues and as I'm pretty new to the Android platform, not sure quite where I am going wrong.

You said:

Android
Same deal as iOS, but get the runtime off catalystInstance.getJavaScriptContextHolder(), which you can pass down to JNI, then cast it to jsi::Runtime * like in iOS.

So, the only way I could find to get access to the JavaScriptContextHolder was to create a new Native Module, and get hold of this in the initialise method like: long context = getReactApplicationContext().getJavaScriptContextHolder().get(); (in iOS, I was able to do this without a module, in my AppDelegate by just getting .runtime from the RCTCxxBridge instance, but I couldn't work out the equivalent in Android).

I'm then passing the context over JNI to C++, so my native module code looks like:

package com.reactnativejsi;

import com.facebook.react.bridge.NativeModule;
import com.facebook.react.bridge.ReactApplicationContext;
import com.facebook.react.bridge.ReactContext;
import com.facebook.react.bridge.ReactContextBaseJavaModule;
import com.facebook.react.bridge.ReactMethod;

public class JSIModule extends ReactContextBaseJavaModule {
    static {
        System.loadLibrary("test_module_jni");
    }

    public JSIModule(ReactApplicationContext reactContext) {
        super(reactContext);
    }

    @Override
    public void initialize() {
        super.initialize();

        long context = getReactApplicationContext().getJavaScriptContextHolder().get();
        install(context);
    }

    @Override
    public String getName() {
        return "JSIModule";
    }

    public native void install(long runtime);
}

Then in my TestModule.cpp, I'm installing the module by calling runtime.global().setProperty(runtime, testModuleName, std::move(object)); - the code is roughly:

std::shared_ptr<TestModuleBinding> testModuleBinding;

extern "C" {
  JNIEXPORT void JNICALL
  Java_com_reactnativejsi_JSIModule_install(JNIEnv* env, jobject thiz, jlong runtimePtr) {
    testModuleBinding = std::make_shared<TestModuleBinding>();
    jsi::Runtime* runtime = (jsi::Runtime*) runtimePtr;

    auto testModuleName = "testModule";
    auto object = jsi::Object::createFromHostObject(*runtime, testModuleBinding);
    runtime->global().setProperty(*runtime, testModuleName, std::move(object));
  }
}

But this causes it to crash with a useless stack trace and no errors that I can see in logcat:

Indeed, either of the lines auto object = jsi::Object::createFromHostObject(*runtime, testModuleBinding); or runtime->global().setProperty(*runtime, testModuleName, std::move(object)); on their own is enough to cause the crash, or even just trying to access runtime->global() and assign it to something... so I am guessing there is something bad about my runtime instance, but I'm a bit of a loss how to debug further!

Like I say, new to all this so I may be making some stupid mistake – if you have any pointers (heh...) or examples of doing this on Android, I'd be really grateful!

Thanks,
Tom

[rhysforyou]

Will TuboModules make it easier to write native modules in Swift? Right now we still have to write Objective-C bridging code which is far from ideal.

[rhdeck]

In case it helps, I wrote a module to automate away the creation of bridging code: NPM: https://www.npmjs.com/package/react-native-swift-bridge GitHub: https://github.com/rhdeck/reactnative-swift-bridge And eagerly await TM to find out how the bridge changes to similarly make swift code easier for us native devs.

…

On Thu, Feb 14, 2019 at 8:54 PM Rhys Powell ***@***.***> wrote: Will TuboModules make it easier to write native modules in Swift? Right now we still have to write Objective-C bridging code which is far from ideal. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#40 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALov_2APjndVX1WTDOSW5PI8psWUrowHks5vNhNkgaJpZM4XaH5o> .

[TheSavior]

@rhdeck that’s awesome and a great idea. We are actually planning on building this into the system for TurboModules. ;-) Have you seen this discussion? #92

[fkgozali]

Will TuboModules make it easier to write native modules in Swift? Right now we still have to write Objective-C bridging code which is far from ideal.

TurboModule foundation is just JSI + platform specific binding. JSI is C++, and for iOS, we need a binding to convert c++ calls into objc++ class methods.

To support swift, such c++ to swift binding needs to be built (we won’t be building this as we’re focused on objc support right now).

Then with codegen, swift code per module can be generated to some degree.

[sunnylqm]

@fkgozali Can you introduce more about the jsi/jsc refactor? Some native modules may need the jsContextRef which is now gone facebook/react-native@c49d365#diff-e15318f48b6447f2d9936c5e047d882fL395

[fkgozali]

Some native modules may need the jsContextRef which is now gone

See the comment above for now: #40 (comment)

Why would you need direct access to the VM though?

[sunnylqm]

@fkgozali the webgl context needs this https://github.com/react-native-community/react-native-webgl/blob/master/cpp/RNWebGL.cpp#L205
I am not familar with c++, is there anyway to access the vm in the legacy native module?

[fkgozali]

I am not familar with c++, is there anyway to access the vm in the legacy native module?

The existing nativemodules system was not designed to provide you access to the VM directly since that can be dangerous (e.g. multithreaded access, since JS is single threaded, etc). The jsContextRef just happened to be added for a different purpose in the past.

With JSI/TurboModule, you can achieve the same thing as https://github.com/react-native-community/react-native-webgl/blob/master/cpp/RNWebGL.cpp#L205 without getting access to the VM. This is done by subclassing JSI HostObject: https://github.com/facebook/react-native/blob/0d7faf6f73b942126e1f45016cde8fd480fd0164/ReactCommon/jsi/jsi.h#L98 -- In fact, TurboModule is just a special subclass of HostObject.

So if your webgl stuff registers itself as TurboModule compatible and you provide your own HostObject subclass, then you don't need any access to the VM directly. This process is work in progress and will be documented when they're ready. For now, you can see the comment above as I previously pasted. #40 (comment)

[sunnylqm]

@fkgozali Thanks for your detailed explanation! I'll give it a try.

[mrousavy]

@DomiR let's hope they create good guides for TurboModules, because let's be honest, I didn't get much information from the current NativeModules guides on reactnative.dev. My only source of information were 6 month old medium articles.

I've thought about contributing to the docs and rewriting the Native Modules sections to be more detailled, but I'm afraid once TurboModules launch all of this will just be wasted time since NativeModules will be "deprecated" sooner or later anyways.

[kelset]

hey folks - there's no updated at this point in time around TurboModules. I am aware that FB is cooking up something for the next 0.64 release that will be related to that.

I don't want to give you false promises or misinformations, but what I know is that TurboModules before end of 2020 is still the plan on their side.

[Enigma10]

@kelset Sorry for asking again. But i was checking 0.64 release issue and there is no mention of turbomodule or fiber.

[terrysahaidak]

The Turbomodules, as well as JSI, is pretty much working and you can use it even right now – a great example of this is the new version of Reanimated which uses both JSI and Turbomodules to provide the best possible performance.

But all the JSI bindings have been written by hand which is hard. That's why Codegen is that important in order to get to the point where we use Turbomodules for all the libraries and it's stable. But as you can see in the commit history there are lots of commits related to add support for Codegen for android. The iOS part seems to be ready.

Also, there are constant commits of Fabric related things as well.

There is no ETA but everything is in active development AFAIK.

[kelset]

+1 on what @terrysahaidak is saying (thanks for answer that, I was on holiday last week).

Just a correction on my last comment, the

I am aware that FB is cooking up something for the next 0.64 release that will be related to that.

Was scraped so I don't think in 0.64 there's going to be any TurboModule specific feature drop.

[a-eid]

@terrysahaidak @kelset I'm wondering how will the Animated Api when Turbomodules & jsi are fully out,
and how would it compare to reanimated 1 and reanimated 2 ?

[terrysahaidak]

@terrysahaidak @kelset I'm wondering how will the Animated Api when Turbomodules & jsi are fully out,
and how would it compare to reanimated 1 and reanimated 2 ?

Nothing really changes for Animated API. Right now (alpha.7 version) you can see in Reanimated 2 installation instruction how to enable TurboModules. After that all the React Native modules (at least on iOS) will use TurboModules and JSI.

But nothing changes because Reanimated 1 and 2 will allow you to write custom animation logic that will be executed on the UI thread directly, just like useNativeDriver does.

[a-eid]

@terrysahaidak thank you,
afaik, JSI & TurboModules will eliminate using the RN bridge, which is the main limitation for the Animated Api, any why SWM had to create their awesome RNGH & Reanimated.

will animated still be inefficient to be used compared to reaniamted 1 or 2 ?

[terrysahaidak]

@terrysahaidak thank you,
afaik, JSI & TurboModules will eliminate using the RN bridge, which is the main limitation for the Animated Api, any why SWM had to create their awesome RNGH & Reanimated.

will animated still be inefficient to be used compared to reaniamted 1 or 2 ?

Not really. The main benefit of Reanimated - code is executed on UI Thread. Even with sync access to style updating from JS Thread, you still can just block the thread by some expensive business logic or for example when you render a new screen. The same thing may happen on web - that's why css3 transitions is used for such things.

[a-eid]

@terrysahaidak thanks, but still jsi & turbomodules would bring a lot of benefits to the Animated Api.

[kelset]

I'm wondering how will the Animated Api when Turbomodules & jsi are fully out, and how would it compare to reanimated 1 and reanimated 2 ?

Too early to say.

afaik, JSI & TurboModules will eliminate using the RN bridge,

The removal of the bridge will happen at a later stage. The bridge will be around for a while even after JSI + TurboModules + Fabric will be fully rolled out to ensure that everyone can smoothly transition.

[axemclion]

@kelset When the re architecture is completed, do you think that the performance gap with flutter will be filled?

Could you elaborate on the performance gap you refer to, in your question ? Are you talking about startup, TTI, or simply calling native calls ? FWIW, you can use RN plugins with Flutter - http://blog.nparashuram.com/2018/12/using-react-natives-plugins-with.html, and the calling mechanism is similar.

[axemclion]

I'm wondering how will the Animated Api when Turbomodules & jsi are fully out, and how would it compare to reanimated 1 and reanimated 2 ?

Reanimated is actually an excellent example of how JSI is used. At a high level, jumping back and forth between Native to JS would be much simpler. I am also hoping that we are able to use JSI to make interruptable or JS Controllable animated APIs.

[EhsanSarshar]

@axemclion reanimated not only benefit the jsi but it also run the javascript part of the animation on the UI thread

[axemclion]

@axemclion reanimated not only benefit the jsi but it also run the javascript part of the animation on the UI thread

Are you saying that JSC or Hermes would run on the UI thread ? Could you elaborate what you mean by running JS on the UI thread ? If I understand TurboModules correctly, you would basically "interrupt" UI thread, and wait for a response to the curve function from JS, and then resume the UI thread. Similar to how android animations work, where you have a separate function that helps you calculate the curve.

[axemclion]

@axemclion for example the use of cpu.

It would help to define what you mean by CPU here. Are you refering to the various threads ? For example, if you run JS with Hermes, you would still use a separate thread which will still run the Hermes bytecode. This would be no different than what happens today.

[EhsanSarshar]

@axemclion Reanimated 2 will spawn a secondary JS context on the UI thread that then is able to run JavaScript functions.
have a look on worklet in reanimated 2 doc

[kelset]

@lorenzoangelini it looks like you are not backing your claims with data and without those it's really hard to understand the actual differences you seem to imply there are 🤷‍♂️

---

EDIT: for future memory, this conversation looks like it doesn't make much sense because the original comment from the author quoted above disappeared (maybe he canceled it).

[aanah0]

Does it have any chance to support remote debugging in Chrome like it currently working? I really like reanimate 2 library, it awesome. but large debugging is required for large projects

[halaei]

In case you are working on more data-type supports can you consider the followings as well?

1. Int64 and UInt64 and other primitive data-types (https://developer.mozilla.org/en-US/docs/Mozilla/js-ctypes/js-ctypes_reference/Int64).
2. Typed arrays (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Typed_arrays).
3. Blob objects with actual data possibly in native side (https://developer.mozilla.org/en-US/docs/Web/API/Blob), with networking support.
4. Null character in strings (facebook/react-native#24129)

[Luckygirlllll]

When approximately these TurboModules will be ready?

[mrousavy]

To support swift, such c++ to swift binding needs to be built (we won’t be building this as we’re focused on objc support right now).

Since Swift is already the mainstream language for iOS development (and is easier to understand than ObjC, for JS developers at least) it would make sense to provide first class support for creating TurboModules in Swift.

I've seen a lot of people refusing to edit anything natively since Objective-C land is a "frightening place". That's why using Swift results in a lot more people getting ready to jump into native development and increasing the overall react-native community engagement (see #253)

While it is theoretically possible to use Swift for TurboModule development, it would be a real pain to interop with the JSI layer since that's written in C++. Correct me if I'm wrong, but afaik stuff like calling host functions, interacting with host objects, etc require a separate bridging layer that's built in one of two ways:

• Swift -> Objective-C -> Objective-C++
• Swift -> C -> C++

Either way you're very limited with C++ language features and can't use C++ types or templates in the second approach since Swift <-> C++ interop isn't fully available yet (see "How developed is C++ interoperability?")

I think that it would make a lot of sense to make use of Swift's Property Wrappers (see Properties, scroll down to "Property Wrappers") for stuff like exporting functions, properties, TurboModules etc, which can then be transformed using codegen internally.

Pseudo code on how this would look for turbo modules:

@ReactModule
class AddTurboModule {

  @ReactMethod
  func add(a: Int, b: Int) -> Int {
    return a + b
  }
}

Pros:

• Makes react native module development a lot easier (and therefore more beginner friendly, increasing community engagement)
• Essentially provides more performance since Swift can be faster than Objective-C in some cases. And don't we all want our apps to be as fast as possible?
• Less boilerplate
• Support for APIs written in Swift (there are a lot of those already)
• As an added bonus, this also makes writing a native module for iOS very similar to writing a native module for Android, since Attributes such as @ReactMethod are used in Android as well.

Note that a good developer experience almost always results in a good product.

Then with codegen, swift code per module can be generated to some degree.

Do you have any info to share about that part? @fkgozali

[fkgozali]

@mrousavy

Do you have any info to share about that part?

The long term plan is to allow adding custom generators that consume the schema produced by react-native-codegen. Today, we're just focusing on ObjC and Java generators because that's what React Native core supports directly.

Right now both generators live directly in the package, but later this year, we want to explore providing API to install your own custom generators outside of that package. When then happens, then more generators for other languages can be built and installed as "plugins" to your apps. This allows us to properly support Kotlin, Swift, pure C++, C# at least, like how ObjC and Java are supported.

In summary: we have plans to allow this, but we won't be focusing directly on Swift support in core anytime soon. We see the react-native-codegen project to be the path towards providing more language support in the future.