Let's begin to understand performance metrics of using JavaScript universally for rich platform development.
The NativeScript TSC joined forces with the "React Native Avengers", Margelo, to begin a deep dive into all aspects of taking advantage of JavaScript for rich development practices. Together we are committed to continually evolving, optimizing and enriching the expansive JavaScript ecosystem.
Various benchmarks to explore in the future
JavaScript has become a universally popular programming language and thus has naturally grown to be used for all sorts of diverse platform developments such as backend server infrastructure as well as desktop and mobile client applications.
All apps involve accessing platform APIs in one form or fashion (e.g. draw the UI, track gestures, access the camera, and more). Most platform APIs (outside the web platform) are not typically written in JavaScript (e.g. most iOS APIs are written in Objective-C and Swift). Therefore to continue using JavaScript with your platform target of choice, there are some metrics to understand when interfacing between the languages.
Exactly how costly this is in practice deserves a deep and thorough analysis to properly understand. The lack of clear and reproducible test cases with accompanying benchmarks leads to the hesitation in using JavaScript to build apps for native platforms. Thus, we on the NativeScript Technical Steering Committee were interested to quantify the overhead of using JavaScript for native app development over a number of performance metrics to address the need to understand.
A few up front considerations:
We hold the sharing of benchmarks in the highest regard due to how fundamental they can become to a decision maker's understanding thus we find it prudent to reiterate:
Sharing performance benchmarks 101:
To expand our field of vision, we decided to profile a NativeScript and React Native app alongside a purely native Objective C app against precise test cases in each of our benchmarks focusing only on iOS in Part 1 here.
We used up-to-date versions of both, using the recommended JavaScript runtimes and optimal approaches for calling native APIs:
| Name | Libraries | Runtime | Approach |
|---|---|---|---|
| NativeScript | @nativescript/[email protected] |
V8 | core |
| React Native | [email protected] |
Hermes | JSI-based native module |
The iOS Objective C app was benchmarked using Xcode 13.4.1.
The performance metrics represent a test of the “marshalling” speed at which each approach can communicate with the target platform through their JavaScript engine. The word "marshalling" is used when you're crossing some sort of boundary. In the case of JavaScript on different devices, the boundary is language specific meaning JavaScript communicating with the natural platform language of the target platform (for example, Swift or Objective C on iOS).
The test cases utilize a single Objective C class, TestFixtures, with 3 different methods represented as follows:
TestFixtures.m:#import "TestFixtures.h"
@implementation TestFixtures
- (int32_t)methodWithX:(int32_t)x Y:(int32_t)y Z:(int32_t)z {
return x + y + z;
}
- (NSString*)methodWithString:(NSString*)aString {
return aString;
}
- (UIImage *)methodWithBigData:(NSArray *)array {
uint8_t *bytes = malloc(sizeof(*bytes) * array.count);
size_t i = 0;
for (id value in array) {
bytes[i++] = [value intValue];
}
NSData *imageData = [NSData dataWithBytesNoCopy:bytes length:array.count freeWhenDone:YES];
UIImage *image = [UIImage imageWithData:imageData];
return image;
}
@end
(int32_t)methodWithX:(int32_t)x Y:(int32_t)y Z:(int32_t)z: Exercises Primitive data types such as numbers
(NSString*)methodWithString:(NSString*)aString: Exercises Strings
(UIImage *)methodWithBigData:(NSArray *)array: Exercises Big Data handling such as binary images
Each benchmark will use identical measuring exercised via a JavaScript loop to determine which approach can communicate from JavaScript to the natural host platform API the fastest. The loop will be implemented as follows:
The measure function:
function measure(name: string, action: () => void) {
const start = performance.now();
action();
const stop = performance.now();
console.log(`${stop - start} ms (${name})`);
}
The measure targets:
measure("Primitives", function () {
for (var i = 0; i < 1e6; i++) {
// Marshall JavaScript to Platform API
}
});
measure("Strings", () => {
const strings = [];
for (var i = 0; i < 100; i++) {
strings.push("abcdefghijklmnopqrstuvwxyz" + i);
}
for (var i = 0; i < 100000; i++) {
// Marshall JavaScript to Platform API
}
});
measure("Big data marshalling", () => {
const array = [];
for (var i = 0; i < 1 << 16; i++) {
array.push(i);
}
for (var i = 0; i < 200; i++) {
// Marshall JavaScript to Platform API
}
});
To utilize TestFixtures.{h,m} in React Native, we included the .h and .m files in the project and then wrote the JSI to expose each method for usage from JavaScript.
The JSI for these benchmarks are setup as follows:
- (void)install:(jsi::Runtime &)runtime {
// initialize the test class
testFixtures = [[TestFixtures alloc] init];
// implement the JSI and expose `methodWithXYZ` to JavaScript
auto marshalMethodWithXYZHostFunction = [] (jsi::Runtime& _runtime, const jsi::Value& thisValue, const jsi::Value* arguments, size_t count) -> jsi::Value {
int32_t result = [testFixtures methodWithX:(int32_t)arguments[0].asNumber() Y:(int32_t)arguments[1].asNumber() Z:(int32_t)arguments[2].asNumber()];
return jsi::Value(result);
};
auto marshalMethodWithXYZJsiFunction = jsi::Function::createFromHostFunction(runtime, jsi::PropNameID::forUtf8(runtime, "methodWithXYZ"), 3, marshalMethodWithXYZHostFunction);
runtime.global().setProperty(runtime, "methodWithXYZ", std::move(marshalMethodWithXYZJsiFunction));
// implement the JSI and expose `methodWithString` to JavaScript
auto marshalMethodWithStringHostFunction = [] (jsi::Runtime& _runtime, const jsi::Value& thisValue, const jsi::Value* arguments, size_t count) -> jsi::Value {
NSString* result = [testFixtures methodWithString: [NSString stringWithUTF8String: arguments[0].asString(_runtime).utf8(_runtime).c_str()]];
return jsi::String::createFromUtf8(_runtime, [result UTF8String]);
};
auto marshalMethodWithStringJsiFunction = jsi::Function::createFromHostFunction(runtime, jsi::PropNameID::forUtf8(runtime, "methodWithString"), 1, marshalMethodWithStringHostFunction);
runtime.global().setProperty(runtime, "methodWithString", std::move(marshalMethodWithStringJsiFunction));
// implement the JSI and expose `methodWithBigData` to JavaScript
auto marshalMethodWithBigDataHostFunction = [] (jsi::Runtime& _runtime, const jsi::Value& thisValue, const jsi::Value* arguments, size_t count) -> jsi::Value {
auto array = arguments[0].asObject(_runtime).asArray(_runtime);
NSMutableArray* nsArray = [NSMutableArray new];
size_t size = array.size(_runtime);
for (int i = 0; i < size; i++) {
[nsArray addObject: [NSNumber numberWithInt: (int32_t)array.getValueAtIndex(_runtime, i).asNumber()]];
}
[testFixtures methodWithBigData:nsArray];
return jsi::Value::undefined();
};
auto marshalMethodWithBigDataJsiFunction = jsi::Function::createFromHostFunction(runtime, jsi::PropNameID::forUtf8(runtime, "methodWithBigData"), 1, marshalMethodWithBigDataHostFunction);
runtime.global().setProperty(runtime, "methodWithBigData", std::move(marshalMethodWithBigDataJsiFunction));
}
We can now use those exposed methods in our React code:
measure("Primitives", function () {
for (var i = 0; i < 1e6; i++) {
methodWithXYZ(i, i, i);
}
});
measure("Strings", () => {
const strings = [];
for (var i = 0; i < 100; i++) {
strings.push("abcdefghijklmnopqrstuvwxyz" + i);
}
for (var i = 0; i < 100000; i++) {
methodWithString(strings[i % strings.length]);
}
});
measure("Big data marshalling", () => {
const array = [];
for (var i = 0; i < 1 << 16; i++) {
array.push(i);
}
for (var i = 0; i < 200; i++) {
methodWithBigData(array);
}
});
To utilize TestFixtures.{h,m} in NativeScript, we placed the .h and .m files within App_Resources/iOS/src where the NativeScript CLI builds them into our Xcode project and auto generates the metadata whereby we can utilize directly.
We can now just use TestFixtures in our TypeScript:
const testFixtures = TestFixtures.alloc().init();
measure("Primitives", () => {
for (var i = 0; i < 1e6; i++) {
testFixtures.methodWithXYZ(i, i, i);
}
});
measure("Strings", () => {
const strings = [];
for (var i = 0; i < 100; i++) {
strings.push("abcdefghijklmnopqrstuvwxyz" + i);
}
for (var i = 0; i < 100000; i++) {
testFixtures.methodWithString(strings[i % strings.length]);
}
});
measure("Big data marshalling", () => {
const array = [];
for (var i = 0; i < 1 << 16; i++) {
array.push(i);
}
for (var i = 0; i < 200; i++) {
testFixtures.methodWithBigData(array);
}
});
The measure function:
- (void)measurePerf:(NSString *)name action:(void (^)(void))action {
NSDate* startDate = [NSDate date];
action();
NSTimeInterval elapsedSeconds = -[startDate timeIntervalSinceNow];
NSLog(@"%@: %fms", name, elapsedSeconds * 1000);
}
To utilize TestFixtures.{h,m} in Xcode, we added the .h and .m files to the Xcode project.
We can now just use TestFixtures in our Xcode project:
id instance = [[TestFixtures alloc] init];
[self measurePerf:@"Primitives" action: ^void() {
for (int32_t i = 0; i < 1e6; i++) {
[instance methodWithX:i Y:i Z:i];
}
}];
[self measurePerf:@"Strings" action: ^void() {
NSMutableArray* strings = [NSMutableArray array];
for (int32_t i = 0; i < 100; i++) {
[strings addObject:[NSString stringWithFormat:@"abcdefghijklmnopqrstuvwxyz%d", i]];
}
for (int32_t i = 0; i < 100000; i++) {
[instance methodWithString:strings[i % strings.count]];
}
}];
[self measurePerf:@"Big data marshalling" action: ^void() {
@autoreleasepool {
NSMutableArray* array = [NSMutableArray array];
for (int32_t i = 0; i < (1 << 16); i++) {
[array addObject:@(i)];
}
for (int32_t i = 0; i < 200; i++) {
[instance methodWithBigData:array];
}
}
}];
| Primitives | Strings | Big data marshalling | |
|---|---|---|---|
| Run 1 | 258ms | 49ms | 1010ms |
| Run 2 | 262ms | 52ms | 1014ms |
| Run 3 | 261ms | 54ms | 1012ms |
| Primitives | Strings | Big data marshalling | |
|---|---|---|---|
| Run 1 | 1359ms | 186ms | 812ms |
| Run 2 | 1362ms | 189ms | 824ms |
| Run 3 | 1359ms | 188ms | 815ms |
| Primitives | Strings | Big data marshalling | |
|---|---|---|---|
| Run 1 | 289ms | 57ms | 1052ms |
| Run 2 | 298ms | 58ms | 1048ms |
| Run 3 | 304ms | 59ms | 1061ms |
You can run these benchmarks by doing the following:
git clone https://github.com/NativeScript/perf-metrics-universal-javascript.git
git checkout part-1
cd perf-metrics-universal-javascript/NativeScript
ns clean
ns run ios
| Primitives | Strings | Big data marshalling | |
|---|---|---|---|
| Run 1 | 1361ms | 215ms | 1427ms |
| Run 2 | 1372ms | 211ms | 1421ms |
| Run 3 | 1387ms | 212ms | 1429ms |
You can run these benchmarks by doing the following:
git clone https://github.com/NativeScript/perf-metrics-universal-javascript.git
git checkout part-1
cd perf-metrics-universal-javascript/ReactNative
yarn
yarn ios
| Primitives | Strings | Big data marshalling | |
|---|---|---|---|
| Run 1 | 8ms | 14ms | 107ms |
| Run 2 | 7ms | 17ms | 116ms |
| Run 3 | 9ms | 16ms | 110ms |
You can run these benchmarks by doing the following:
git clone https://github.com/NativeScript/perf-metrics-universal-javascript.git
git checkout part-1
cd perf-metrics-universal-javascript/NativeiOS
open NativeiOS.xcodeproj
You can now run in Xcode.
To summarize the best numbers from each for just String handling:
As we can see JavaScript is quite performant with platform API interaction.
Both React Native and NativeScript can achieve great performance results across rich platform development which is why we are committed to evolving both together.
NativeScript 8.3 specifically brings optimizations close to pure platform API interaction speed and is continuing to optimize further.
The recent NativeScript 8.3 release brought ~30% performance optimization gains. This can be measured using the same test cases above, for example:
| Primitives | Strings | Big data marshalling | |
|---|---|---|---|
| Run 1 | 831ms | 121ms | 1240ms |
| Primitives | Strings | Big data marshalling | |
|---|---|---|---|
| Run 1 | 258ms | 49ms | 1010ms |
Primitives and Strings make up some of the most commonly marshalled data types throughout apps and the 8.3 optimizations improved benchmarks throughout all three categories.
During the 8.3 optimization pass we found other areas that can also be improved for further gains in subsequent releases and are continually hyper focused on performance.
The nuanced understanding in how these results apply to your case comes from what you're doing and what you're after. Marshalling speed benefits cases where you are doing a lot of platform API interaction between your JavaScript code and your target platform.
It's important to understand that NativeScript can be used with React Native. We plan to share more on this in the future.
We hope these benchmarks begin to enlighten you when talking benchmarks among your team and why we continue to enjoy the results.
[^1]: React Native JSI (Javascript Interface) is a layer that helps in communication between Javascript and Native Platforms easier and faster. It is the core element in re-architecture of React Native with Fabric UI Layer and Turbo Modules. - React Native JSI: Part 1 - Getting Started