Performance Benchmarks using JavaScript for Universal Platform Development

Let's begin to understand performance metrics of using JavaScript universally for rich platform development.

The NativeScript TSC joined forces with the "React Native Avengers", Margelo, to begin a deep dive into all aspects of taking advantage of JavaScript for rich development practices. Together we are committed to continually evolving, optimizing and enriching the expansive JavaScript ecosystem.

• → iOS Marshalling

Various benchmarks to explore in the future

• iOS Startup time, App size, navigation, view rendering
• Android Benchmarks for Marshalling
• Android Startup time, App size, navigation, view rendering
• A look at how many js <-> native calls are even made in typical apps
• Multi-threading understanding alongside performance benchmarks

JavaScript has become a universally popular programming language and thus has naturally grown to be used for all sorts of diverse platform developments such as backend server infrastructure as well as desktop and mobile client applications.

All apps involve accessing platform APIs in one form or fashion (e.g. draw the UI, track gestures, access the camera, and more). Most platform APIs (outside the web platform) are not typically written in JavaScript (e.g. most iOS APIs are written in Objective-C and Swift). Therefore to continue using JavaScript with your platform target of choice, there are some metrics to understand when interfacing between the languages.

Exactly how costly this is in practice deserves a deep and thorough analysis to properly understand. The lack of clear and reproducible test cases with accompanying benchmarks leads to the hesitation in using JavaScript to build apps for native platforms. Thus, we on the NativeScript Technical Steering Committee were interested to quantify the overhead of using JavaScript for native app development over a number of performance metrics to address the need to understand.

A few up front considerations:

• There are multiple different JS runtimes
• Each with multiple possible approaches for accessing native platform APIs
• Some likely may be more optimal than others

We hold the sharing of benchmarks in the highest regard due to how fundamental they can become to a decision maker's understanding thus we find it prudent to reiterate:

Sharing performance benchmarks 101:

• Never trust a benchmark you cannot easily run yourself
• Never trust a benchmark which does not clearly describe the setup and implementation of the benchmark

To expand our field of vision, we decided to profile a NativeScript and React Native app alongside a purely native Objective C app against precise test cases in each of our benchmarks focusing only on iOS in Part 1 here.

We used up-to-date versions of both, using the recommended JavaScript runtimes and optimal approaches for calling native APIs:

Name	Libraries	Runtime	Approach
NativeScript	@nativescript/[email protected]	V8	core
React Native	[email protected]	Hermes	JSI-based native module

The iOS Objective C app was benchmarked using Xcode 13.4.1.

Benchmark Setup

The performance metrics represent a test of the “marshalling” speed at which each approach can communicate with the target platform through their JavaScript engine. The word "marshalling" is used when you're crossing some sort of boundary. In the case of JavaScript on different devices, the boundary is language specific meaning JavaScript communicating with the natural platform language of the target platform (for example, Swift or Objective C on iOS).

The test cases utilize a single Objective C class, TestFixtures, with 3 different methods represented as follows:

• TestFixtures.m:

#import "TestFixtures.h"

@implementation TestFixtures
- (int32_t)methodWithX:(int32_t)x Y:(int32_t)y Z:(int32_t)z {
    return x + y + z;
}

- (NSString*)methodWithString:(NSString*)aString {
    return aString;
}

- (UIImage *)methodWithBigData:(NSArray *)array {
    uint8_t *bytes = malloc(sizeof(*bytes) * array.count);

    size_t i = 0;
    for (id value in array) {
        bytes[i++] = [value intValue];
    }

    NSData *imageData = [NSData dataWithBytesNoCopy:bytes length:array.count freeWhenDone:YES];
    UIImage *image = [UIImage imageWithData:imageData];
    return image;
}
@end

• (int32_t)methodWithX:(int32_t)x Y:(int32_t)y Z:(int32_t)z: Exercises Primitive data types such as numbers

• (NSString*)methodWithString:(NSString*)aString: Exercises Strings

• (UIImage *)methodWithBigData:(NSArray *)array: Exercises Big Data handling such as binary images

Each benchmark will use identical measuring exercised via a JavaScript loop to determine which approach can communicate from JavaScript to the natural host platform API the fastest. The loop will be implemented as follows:

The measure function:

function measure(name: string, action: () => void) {
  const start = performance.now();

  action();

  const stop = performance.now();
  console.log(`${stop - start} ms (${name})`);
}

The measure targets:

measure("Primitives", function () {
  for (var i = 0; i < 1e6; i++) {
    // Marshall JavaScript to Platform API
  }
});


measure("Strings",  () => {
  const strings = [];

  for (var i = 0; i < 100; i++) {
    strings.push("abcdefghijklmnopqrstuvwxyz" + i);
  }

  for (var i = 0; i < 100000; i++) {
    // Marshall JavaScript to Platform API
  }
});

measure("Big data marshalling", () => {
  const array = [];

  for (var i = 0; i < 1 << 16; i++) {
    array.push(i);
  }

  for (var i = 0; i < 200; i++) {
    // Marshall JavaScript to Platform API
  }
});

Benchmark Implementation in React Native

To utilize TestFixtures.{h,m} in React Native, we included the .h and .m files in the project and then wrote the JSI to expose each method for usage from JavaScript.

The JSI for these benchmarks are setup as follows:

- (void)install:(jsi::Runtime &)runtime {
  
  // initialize the test class
  testFixtures = [[TestFixtures alloc] init];
  
  // implement the JSI and expose `methodWithXYZ` to JavaScript
  auto marshalMethodWithXYZHostFunction = [] (jsi::Runtime& _runtime, const jsi::Value& thisValue, const jsi::Value* arguments, size_t count) -> jsi::Value { 
    int32_t result = [testFixtures methodWithX:(int32_t)arguments[0].asNumber() Y:(int32_t)arguments[1].asNumber() Z:(int32_t)arguments[2].asNumber()];
    return jsi::Value(result);
  };
  auto marshalMethodWithXYZJsiFunction = jsi::Function::createFromHostFunction(runtime, jsi::PropNameID::forUtf8(runtime, "methodWithXYZ"), 3, marshalMethodWithXYZHostFunction);
  runtime.global().setProperty(runtime, "methodWithXYZ", std::move(marshalMethodWithXYZJsiFunction));
  
  // implement the JSI and expose `methodWithString` to JavaScript
  auto marshalMethodWithStringHostFunction = [] (jsi::Runtime& _runtime, const jsi::Value& thisValue, const jsi::Value* arguments, size_t count) -> jsi::Value {
    NSString* result = [testFixtures methodWithString: [NSString stringWithUTF8String: arguments[0].asString(_runtime).utf8(_runtime).c_str()]];
      return jsi::String::createFromUtf8(_runtime, [result UTF8String]);
  };
  auto marshalMethodWithStringJsiFunction = jsi::Function::createFromHostFunction(runtime, jsi::PropNameID::forUtf8(runtime, "methodWithString"), 1, marshalMethodWithStringHostFunction);
  runtime.global().setProperty(runtime, "methodWithString", std::move(marshalMethodWithStringJsiFunction));
  
  // implement the JSI and expose `methodWithBigData` to JavaScript
  auto marshalMethodWithBigDataHostFunction = [] (jsi::Runtime& _runtime, const jsi::Value& thisValue, const jsi::Value* arguments, size_t count) -> jsi::Value {
    
    auto array = arguments[0].asObject(_runtime).asArray(_runtime);
    NSMutableArray* nsArray = [NSMutableArray new];
    size_t size = array.size(_runtime);
    for (int i = 0; i < size; i++) {
      [nsArray addObject: [NSNumber numberWithInt: (int32_t)array.getValueAtIndex(_runtime, i).asNumber()]];
    }
    [testFixtures methodWithBigData:nsArray];
    return jsi::Value::undefined();
  };
  auto marshalMethodWithBigDataJsiFunction = jsi::Function::createFromHostFunction(runtime, jsi::PropNameID::forUtf8(runtime, "methodWithBigData"), 1, marshalMethodWithBigDataHostFunction);
  runtime.global().setProperty(runtime, "methodWithBigData", std::move(marshalMethodWithBigDataJsiFunction));
}

We can now use those exposed methods in our React code:

measure("Primitives", function () {
  for (var i = 0; i < 1e6; i++) {
    methodWithXYZ(i, i, i);
  }
});


measure("Strings",  () => {
  const strings = [];

  for (var i = 0; i < 100; i++) {
    strings.push("abcdefghijklmnopqrstuvwxyz" + i);
  }

  for (var i = 0; i < 100000; i++) {
    methodWithString(strings[i % strings.length]);
  }
});

measure("Big data marshalling", () => {
  const array = [];

  for (var i = 0; i < 1 << 16; i++) {
    array.push(i);
  }

  for (var i = 0; i < 200; i++) {
    methodWithBigData(array);
  }
});

Benchmark Implementation in NativeScript

To utilize TestFixtures.{h,m} in NativeScript, we placed the .h and .m files within App_Resources/iOS/src where the NativeScript CLI builds them into our Xcode project and auto generates the metadata whereby we can utilize directly.

We can now just use TestFixtures in our TypeScript:

const testFixtures = TestFixtures.alloc().init();

measure("Primitives",  () => {
  for (var i = 0; i < 1e6; i++) {
    testFixtures.methodWithXYZ(i, i, i);
  }
});

measure("Strings",  () => {
  const strings = [];

  for (var i = 0; i < 100; i++) {
    strings.push("abcdefghijklmnopqrstuvwxyz" + i);
  }

  for (var i = 0; i < 100000; i++) {
    testFixtures.methodWithString(strings[i % strings.length]);
  }
});

measure("Big data marshalling", () => {
  const array = [];

  for (var i = 0; i < 1 << 16; i++) {
    array.push(i);
  }

  for (var i = 0; i < 200; i++) {
    testFixtures.methodWithBigData(array);
  }
});

Benchmark Implementation in Objective C

The measure function:

- (void)measurePerf:(NSString *)name action:(void (^)(void))action {
    NSDate* startDate = [NSDate date];

    action();

    NSTimeInterval elapsedSeconds = -[startDate timeIntervalSinceNow];
    NSLog(@"%@: %fms", name, elapsedSeconds * 1000);
}

To utilize TestFixtures.{h,m} in Xcode, we added the .h and .m files to the Xcode project.

We can now just use TestFixtures in our Xcode project:

id instance = [[TestFixtures alloc] init];

[self measurePerf:@"Primitives" action: ^void() {
    for (int32_t i = 0; i < 1e6; i++) {
        [instance methodWithX:i Y:i Z:i];
    }
}];

[self measurePerf:@"Strings" action: ^void() {
    NSMutableArray* strings = [NSMutableArray array];

    for (int32_t i = 0; i < 100; i++) {
        [strings addObject:[NSString stringWithFormat:@"abcdefghijklmnopqrstuvwxyz%d", i]];
    }

    for (int32_t i = 0; i < 100000; i++) {
        [instance methodWithString:strings[i % strings.count]];
    }
}];

[self measurePerf:@"Big data marshalling" action: ^void() {
    @autoreleasepool {
        NSMutableArray* array = [NSMutableArray array];

        for (int32_t i = 0; i < (1 << 16); i++) {
            [array addObject:@(i)];
        }

        for (int32_t i = 0; i < 200; i++) {
            [instance methodWithBigData:array];
        }
    }
}];

Benchmark Results

Release Mode (Production)

NativeScript 8.3 Metrics

• iOS 15.5 iPhone 13 Pro Device

	Primitives	Strings	Big data marshalling
Run 1	258ms	49ms	1010ms
Run 2	262ms	52ms	1014ms
Run 3	261ms	54ms	1012ms

React Native 0.69.3 metrics

• iOS 15.5 iPhone 13 Pro Device

	Primitives	Strings	Big data marshalling
Run 1	1359ms	186ms	812ms
Run 2	1362ms	189ms	824ms
Run 3	1359ms	188ms	815ms

Simulators (Debug)

NativeScript 8.3 Metrics

• Xcode 13.4, iOS 15.5 iPhone 13 Pro Simulator
• Run on a Mac M1 Max (macOS 12.3.1) with 64 GB Memory

	Primitives	Strings	Big data marshalling
Run 1	289ms	57ms	1052ms
Run 2	298ms	58ms	1048ms
Run 3	304ms	59ms	1061ms

Run NativeScript benchmarks yourself

You can run these benchmarks by doing the following:

git clone https://github.com/NativeScript/perf-metrics-universal-javascript.git
git checkout part-1
cd perf-metrics-universal-javascript/NativeScript

ns clean
ns run ios

React Native 0.69.3 metrics

• Xcode 13.4, iOS 15.5 iPhone 13 Pro Simulator
• Run on a Mac M1 Max (macOS 12.3.1) with 64 GB Memory

	Primitives	Strings	Big data marshalling
Run 1	1361ms	215ms	1427ms
Run 2	1372ms	211ms	1421ms
Run 3	1387ms	212ms	1429ms

Run React Native benchmarks yourself

You can run these benchmarks by doing the following:

git clone https://github.com/NativeScript/perf-metrics-universal-javascript.git
git checkout part-1
cd perf-metrics-universal-javascript/ReactNative

yarn
yarn ios

Objective C Metrics

• Xcode 13.4, iOS 15.5 iPhone 13 Pro Simulator
• Run on a Mac M1 Max (macOS 12.3.1) with 64 GB Memory

	Primitives	Strings	Big data marshalling
Run 1	8ms	14ms	107ms
Run 2	7ms	17ms	116ms
Run 3	9ms	16ms	110ms

Run Objective C benchmarks yourself

You can run these benchmarks by doing the following:

git clone https://github.com/NativeScript/perf-metrics-universal-javascript.git
git checkout part-1
cd perf-metrics-universal-javascript/NativeiOS

open NativeiOS.xcodeproj

You can now run in Xcode.

Initial Impressions

To summarize the best numbers from each for just String handling:

• NativeScript: 49ms
• React Native: 186ms
• Objective C: 14ms

As we can see JavaScript is quite performant with platform API interaction.

Both React Native and NativeScript can achieve great performance results across rich platform development which is why we are committed to evolving both together.

NativeScript 8.3 specifically brings optimizations close to pure platform API interaction speed and is continuing to optimize further.

NativeScript Obsession with Performance

The recent NativeScript 8.3 release brought ~30% performance optimization gains. This can be measured using the same test cases above, for example:

8.2 benchmarks

	Primitives	Strings	Big data marshalling
Run 1	831ms	121ms	1240ms

8.3 benchmarks

	Primitives	Strings	Big data marshalling
Run 1	258ms	49ms	1010ms

Primitives and Strings make up some of the most commonly marshalled data types throughout apps and the 8.3 optimizations improved benchmarks throughout all three categories.

During the 8.3 optimization pass we found other areas that can also be improved for further gains in subsequent releases and are continually hyper focused on performance.

Observations and Nuanced Understandings

The nuanced understanding in how these results apply to your case comes from what you're doing and what you're after. Marshalling speed benefits cases where you are doing a lot of platform API interaction between your JavaScript code and your target platform.

It's important to understand that NativeScript can be used with React Native. We plan to share more on this in the future.

We hope these benchmarks begin to enlighten you when talking benchmarks among your team and why we continue to enjoy the results.

[^1]: React Native JSI (Javascript Interface) is a layer that helps in communication between Javascript and Native Platforms easier and faster. It is the core element in re-architecture of React Native with Fabric UI Layer and Turbo Modules. - React Native JSI: Part 1 - Getting Started