Yet another performance benchmark.

[psionic12]

Our team found that the performance is actually got worse after switching to the new architecture, so I benchmarked and dived into the code for a while, and here's all the information I've investigated so far.

My test starts from runApplication get called, and ends to the first onDraw get called, which means all views are ready to render. The device being tested is a Pixel4, the tested app is rn-tester and here's the result:

JSC:

	Total cost	Total layout cost
Fabric	544ms	147ms
Non Fabric	377ms	85ms

Hermes

	Total cost	Total layout cost
Fabric	464ms	158ms
Non Fabric	324ms	88ms

We can tell that no matter what engine it is on, the performance with fabric on is worse. And although not for all, the layout procedure takes a big amount of time. This is strange since the new arch use the "flatten view" technology. I check this, and it indeed reduces the complicity of the view tree (9 level in old arch vs 6 level with new arch).

What I'm gonna do next is to cut the layout procedure in pieces and hope if I can tell the difference between the old arch and the new one.

What's more, I didn't test this on iOS, because I didn't find a function similar to the function onDraw on Android, the function -drawRect seems not what I want. If anyone knows a proper function, I can test the performance of iOS and shared here.

P.S. The layout costing time I calculated is to add all the duration of running YGNodeCalculateLayoutWithContext, since YGNodeCalculateLayoutWithContext is the entry that both old arch and new arch will call.

[troZee]

Hey 👋
Could you provide a methodology of those experiments ? I would like to see the code and how it has been measured. Moreover could you also test the app using https://github.com/doomsower/react-native-startup-time#readme and provide the same tables for both old and new arch ?

[psionic12]

Hi,
Honestly speaking, taking runApplication as the start point is not very accurate since it involves the startup thing, although my point is that the layout should be faster, but the fact is not. I'm going to find another start point which only have things to do with rendering, because our programmer complained to me about the rendering speed in his views, not the whole app's startup costing.

Meanwhile, I'll investigate the app you mentioned and pasted the result here, and after I found another proper start point, I will fork and create a branch, so others can do the same benchmark.

[troZee]

I'll investigate the app you mentioned

It is not an app. It is just a library, which will help you to measure startup time. All what you need to do is just to add this view to Home Screen and that's it.

[psionic12]

I update a new profile to exclude the cost of startup, now the start point of calculation is the render() get called in JS, not runApplication, here's the result:
JSC

	Total cost	Total layout cost
Fabric	451ms	139ms
Non Fabric	300ms	95ms

Hermes

	Total cost	Total layout cost
Fabric	376ms	149ms
Non Fabric	266ms	76ms

[cortinico]

Hey 👋
Could you provide a methodology of those experiments ? I would like to see the code and how it has been measured.

+1 to what @troZee mentioned. We need to have the code and the full methodology of how those benchmarks have been executed to start looking into them

[psionic12]

@cortinico I uploaded my code to my forked base, it will print logs at runapplication in JS and dispatchDraw in Java, and the total time cost executing YGNodeCalculateLayoutWithContext.

The log may look like this (by typing adb logcat |grep my_fabric_profile):

10-31 15:55:07.529 30289 31537 I ReactNativeJS: my_fabric_profile: runapplication
10-31 15:55:07.532 30289 31586 W yoga    : my_fabric_profile YGNodeCalculateLayoutWithContext: duration: 19000, total: 58692000
10-31 15:55:07.587 30289 31586 W yoga    : my_fabric_profile YGNodeCalculateLayoutWithContext: duration: 485000, total: 59177000
10-31 15:55:07.619 30289 30289 D my_fabric_profile: dispatchDraw
10-31 15:55:07.630 30289 31586 W yoga    : my_fabric_profile YGNodeCalculateLayoutWithContext: duration: 1181000, total: 60358000
10-31 15:55:07.705 30289 31586 W yoga    : my_fabric_profile YGNodeCalculateLayoutWithContext: duration: 1763000, total: 62121000
10-31 15:55:07.814 30289 31586 W yoga    : my_fabric_profile YGNodeCalculateLayoutWithContext: duration: 2523000, total: 64644000

The total duration is the time from runapplication to the first dispatchDraw. And the time of layout is the duration part of the log YGNodeCalculateLayoutWithContext: duration: 2523000 (in nanosecond), and the total part can be ignored.

This code is ugly, but it is one of our item of profiling monitoring (not using this ugly code, though).

According to this log, the time from runapplication to dispatchDraw and the total layout time of fabric is higher than non-fabirc.

[troZee]

Thank you @psionic12 for providing a measure method. For any kind of measurements, we should not use console.log or Logd since it does not guarantee, that timestamp is correct (Logging has some additional overhead and it is not deterministic, especially when you call function from different thread). What I would suggest here is to assign this time to any variable and read those timestamp after init. You can read that time using eg Button component.

For android please use

long ms = System.currentTimeMillis();

For JS please use

Date.now()

[psionic12]

@troZee partially done. In this case, you button suggestion need to communicate between Java and JS, which is a little difficult, and as I measured, logging will have very little compact to the whole result, so I'll come back later.

[psionic12]

I found that the total cost of measuring(ReactTextShadowNode::measure vs ReactTextViewManager::measure) gets a significant difference, about 57ms vs 94ms (old arch vs new arch), and it seems that operations on native maps takes some time.

I don't know it this is the root case, but there seem to be too much JNI calls to import keys and values from a native map, I did a calculation and there are 4391 calls of JNI on importing native maps, even though ReadableNativeMap has a caching mechanism.

This may explain why fabric on iOS works well, since there's no extra JNI data copy on iOS.

P.S. Another test, a view with nearly zero texts vs full of texts:

	zero text	lots of texts
fabric	183.5ms	250.7ms
non-fabric	172.5ms	213.8ms

[troZee]

FYI: Shopify/flash-list#550 (comment)

[kelset]

this seems to be consistent with @Almouro's benchmarks on the text component: #85

[psionic12]

I found this on the doc:

It is important to recognize that the renderer use case for Android still incurs the cost of JNI for two primary use cases:

1. Layout calculation of complex views (e.g. Text, TextInput, etc.) requires sending props over JNI.
2. The mount phase requires sending mutation operations over JNI.

The team is exploring replacing ReadableMap with a new mechanism to serialize data using ByteBuffer to reduce overhead of JNI. Our goal is to reduce overhead of JNI by 35–50%.

Seems that my guessing is right.

[lindboe]

@cortinico Just wanted to ask to confirm, since the above quote from the docs is about a year old (Dec 2021). Is it still the case that the team is exploring that ReadableMap replacement?

[cortinico]

@lindboe sorry I missed this comment, the most up to date write up on ReadableMap and replacements (MapBuffers) is this #32

[sammy-SC]

Hello @psionic12,

we spent some time analysing performance and summarised our findings in #123.

Could you please try to run your benchmarks on the 0.72.0-RC.1 and see what the performance is like? We've made some improvements.

[psionic12]

@sammy-SC I'll do the benchmark and let you know ASAP.

[psionic12]

Hi @sammy-SC, I found that I can't disable fabric for rn_tester as before. In 0.68, there is a build config for that, but it seems to be deleted in 0.72. Do you know how to do this? Blow is the test result, compared with 0.68 with fabric off:

	Time to show	Time to layout
Fabric(0.72)	375ms	61ms
Fabric(0.68)	451ms	139ms
Non Fabric(0.68)	300ms	95ms

The result shows that the total layout time cost is even better compared to no fabric, but the time from app start to the view is actually shown is still slow, compared to no fabric, but faster than before, compared to old version with fabric on.

[sammy-SC]

There should be newArchEnabled flag in android/gradle.properties of your project. Is that not a case? It is documented here: https://reactnative.dev/docs/next/the-new-architecture/use-app-template?platform=android#enable-the-new-architecture

this is interesting, if you isolate out app startup and measure just pure render time, what do you see?
I wonder if it isn't coming from extra classes being loaded when the New Architecture is enabled.