Skip to content
This repository has been archived by the owner on Jun 13, 2024. It is now read-only.

test #275

Open
wants to merge 17 commits into
base: release
Choose a base branch
from
Open

test #275

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ Other options can be found in [Build Napa.js](https://github.com/Microsoft/napaj

## Quick Start
```js
var napa = require('napajs');
var zone1 = napa.zone.create('zone1', { workers: 4 });
const napa = require('napajs');
const zone1 = napa.zone.create('zone1', { workers: 4 });

// Broadcast code to all 4 workers in 'zone1'.
zone1.broadcast('console.log("hello world");');
Expand All @@ -37,11 +37,11 @@ More examples:
* [Synchronized loading](./examples/tutorial/synchronized-loading)

## Features
- Multi-threaded JavaScript runtime
- Node.js compatible module architecture with NPM support
- API for object transportation, object sharing and synchronization across JavaScript threads
- API for pluggable logging, metric and memory allocator
- Distributed as a Node.js module, as well as supporting embed scenarios
- Multi-threaded JavaScript runtime.
- Node.js compatible module architecture with NPM support.
- API for object transportation, object sharing and synchronization across JavaScript threads.
- API for pluggable logging, metric and memory allocator.
- Distributed as a Node.js module, as well as supporting embed scenarios.

## Documentation
- [Napa.js Home](https://github.com/Microsoft/napajs/wiki)
Expand All @@ -56,7 +56,7 @@ You can contribute to Napa.js in following ways:
* Contribute to core module compatibility with Node.
* Contribute bug fixes.

This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact opencode@microsoft.com with any additional questions or comments.
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).<br> For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact opencode@microsoft.com with any additional questions or comments.

# License
Copyright (c) Microsoft Corporation. All rights reserved.
Expand Down
24 changes: 12 additions & 12 deletions benchmark/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ We got this report on environment below:

| Name | Value |
|-------------------|---------------------------------------------------------------------------------------|
|**Processor** |Intel(R) Xeon(R) CPU L5640 @ 2.27GHz, 8 virtual procesors |
|**Processor** |Intel(R) Xeon(R) CPU L5640 @ 2.27GHz, 8 virtual processors |
|**System Type** |x64-based PC |
|**Physical Memory**|16.0 GB |
|**OS version** |Microsoft Windows Server 2012 R2 |
Expand All @@ -36,12 +36,12 @@ Please refer to [execute-scalability.ts](./execute-scalability.ts) for test deta

## Execute overhead
The overhead of `zone.execute` includes
1. marshalling cost of arguments in caller thread.
2. queuing time before a worker can execute.
3. unmarshalling cost of arguments in target worker.
4. marshalling cost of return value from target worker.
5. queuing time before caller callback is notified.
6. unmarshalling cost of return value in caller thread.
1. Marshalling cost of arguments in caller thread.
2. Queuing time before a worker can execute.
3. Unmarshalling cost of arguments in target worker.
4. Marshalling cost of return value from target worker.
5. Queuing time before caller callback is notified.
6. Unmarshalling cost of return value in caller thread.

In this section we will examine #2 and #5. So we use empty function with no arguments and no return value.

Expand Down Expand Up @@ -90,7 +90,7 @@ Average overhead is around 0.06ms to 0.12ms for `zone.execute`.
## Transport overhead

The overhead of `transport.marshall` includes
1. overhead of needing replacer callback during JSON.stringify. (even empty callback will slowdown JSON.stringfiy significantly)
1. overhead of needing replacer callback during JSON.stringify. (even an empty callback will slow down JSON.stringify significantly)
2. traverse every value during JSON.stringify, to check value type and get `cid` to put into payload.
- a. If value doesn't need special care.
- b. If value is a transportable object that needs special care.
Expand Down Expand Up @@ -132,12 +132,12 @@ Please refer to [transport-overhead.ts](./transport-overhead.ts) for test detail
## Store access overhead

The overhead of `store.set` includes
1. overhead of calling `transport.marshall` on value.
2. overhead of put marshalled data and transport context into C++ map (with exclusive_lock).
1. Overhead of calling `transport.marshall` on value.
2. Overhead of put marshalled data and transport context into C++ map (with exclusive_lock).

The overhead of `store.get` includes
1. overhead of getting marshalled data and transport context from C++ map (with shared_lock).
2. overhead of calling `transport.unmarshall` on marshalled data.
1. Overhead of getting marshalled data and transport context from C++ map (with shared_lock).
2. Overhead of calling `transport.unmarshall` on marshalled data.

For `store.set`, numbers below indicates the cost beyond marshall is around 0.07~0.4ms varies per payload size. (10B to 18KB). `store.get` takes a bit more: 0.06~0.9ms with the same payload size variance. If the value in store is not updated frequently, it's always good to cache it in JavaScript world.

Expand Down
12 changes: 6 additions & 6 deletions docs/api/log.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
## <a name="intro"></a> Introduction
Logging is a basic requirement for building services. `napajs` logging API enables developers to integrate their own logging capabilities in both JavaScript and C++ (addon) world.

A log row may contain following information:
A log row may contain the following information:
- (Optional) Section: Useful field to filter log rows. Treatment is defined by logging providers.
- (Optional) Trace ID: Useful field to join logs in the same transaction or request.
- (Required) Message: Log message.
Expand Down Expand Up @@ -54,7 +54,7 @@ void MyFunction() {
### <a name="log"></a> log(message: string): void
It logs a message. Using info level.

*`log` is a shortcut for `log.info`.*
* A `log` is a shortcut for `log.info`.*

Example:
```js
Expand All @@ -78,16 +78,16 @@ Example:
napa.log('request', 'A1B2C3D4', 'request received');
```
### <a name="log-err"></a> log.err(...)
It logs an error message. Three variation of arguments are the same with `log`.
It logs an error message. Three variations of arguments are the same with the `log`.

### <a name="log-warn"></a> log.warn(...)
It logs a warning message. Three variation of arguments are the same with `log`.
It logs a warning message. Three variations of arguments are the same with the `log`.

### <a name="log-info"></a>log.info(...)
It logs an info message. Three variation of arguments are the same with `log`.
It logs an info message. Three variations of arguments are the same with the `log`.

### <a name="log-debug"></a> log.debug(...)
It logs a debug message. Three combinations of arguments are the same with `log`.
It logs a debug message. Three combinations of arguments are the same with the `log`.

## <a name="use-custom-providers"></a> Using custom logging providers
Developers can hook up custom logging provider by calling the following before creation of any zones:
Expand Down
2 changes: 1 addition & 1 deletion docs/api/memory.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Memory allocation in C++ addon is tricky. A common pitfall is to allocate memory

There are also advanced scenarios that user want to customize memory allocation. Napa.js provides APIs for customizing memory allocator as well.

### Recommended way of allocate memory.
### Recommended way of allocate memory
TBD

### Customize memory allocation
Expand Down
6 changes: 3 additions & 3 deletions docs/api/metric.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ A metric has its identity containing following information:
- **Section**: The group or category of the metric.
- **Name**: Name of the metric. Section/Name combination should be unique in the system.
- **Type**: Type of the metric, which can be
- *Number*: A absolute number, e.g: PrivateBytes.
- *Number*: An absolute number, e.g: PrivateBytes.
- *Rate*: A flowing volume in number, e.g: QueryPerSecond.
- *Percentile*: A absolute number that needs to be sampled by percentiles, e.g: SuccessLatency.
- *Percentile*: An absolute number that needs to be sampled by percentiles, e.g: SuccessLatency.
- **Dimensions**: A metric can have multiple dimensions, each dimension can bind with a string value at runtime. e.g: IncomingRequestRate can have 2 dimensions: ['client-id', 'request-type'].

Metrics are process-wise objects, which can be used across [zones](./zone.md#intro).
Expand All @@ -40,7 +40,7 @@ Metrics are process-wise objects, which can be used across [zones](./zone.md#int
Percentile,
};

/// <summary> Interface to represents a multi-dimensional metric with a maximum dimensionality of 64. </summary>
/// <summary> Interface to represent a multi-dimensional metric with a maximum dimensionality of 64. </summary>
class Metric {
public:

Expand Down
2 changes: 1 addition & 1 deletion docs/api/module.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Napa.js follows [Node.js' convention](https://nodejs.org/api/modules.html) to su
4) API of creating C++ modules (addons) are similar. Napa.JS introduced macros that the same source code can be compiled to produce both Napa.js addon and Node.js addon.

But there are also differences:
1) C++ module that is designed/implemented for Napa.js can run on Node.JS (need different compile flags to produce '.napa' and '.node'). But not vice versal.
1) C++ module that is designed/implemented for Napa.js can run on Node.JS (need different compile flags to produce '.napa' and '.node'). But not vice versa.
2) Napa.js doesn't support all Node.js API. Node API are supported [incrementally](./node-api.md) on the motivation of adding Node.js built-ins and core modules that are needed for computation heavy tasks. You can access full capabilities of Node exposed via [Node zone](./zone.md#node-zone).
3) Napa.js doesn't provide `uv` functionalities, thus built-ins and core modules have its own implementation. To write async function in addon, methods `DoAsyncWork`/`PostAsyncWork` are introduced to work for both Napa.js and Node.js.
4) Napa.js supports embed mode. C++ modules need separate compilation between Node mode and embed mode.
Expand Down
2 changes: 1 addition & 1 deletion docs/api/store.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Though very convenient, it's not recommended to use store to pass values within
Following APIs are exposed to create, get and operate upon stores.

### <a name="create"></a> create(id: string): Store
It creates a store by a string identifer that can be used to get the store later. When all references to the store from all JavaScript VMs are cleared, the store will be destroyed. Thus always keep a reference at global or module scope is usually a good practice using `Store`. Error will be thrown if the id already exists.
It creates a store by a string identifier that can be used to get the store later. When all references to the store from all JavaScript VMs are cleared, the store will be destroyed. Thus always keep a reference at global or module scope is usually a good practice using `Store`. Error will be thrown if the id already exists.

Example:
```js
Expand Down
30 changes: 15 additions & 15 deletions docs/api/transport.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Transportable types are:
- JavaScript primitive types: undefined, null, boolean, number, string
- Object (TypeScript class) that implement [`Transportable`](#transportable) interface
- Function without referencing closures.
- <a name="built-in-whitelist"></a> JavaScript standard built-In objects in this whitelist.
- <a name="built-in-whitelist"></a> JavaScript standard built-in objects in this whitelist.
* ArrayBuffer
* Float32Array
* Float64Array
Expand All @@ -54,23 +54,23 @@ Transportable types are:
- Array or plain JavaScript object that is composite pattern of above.

### <a name="constructor-id"></a> Constructor ID (cid)
For user classes that implement [`Transportable`](#transportable) interface, Napa uses Constructor ID (`cid`) to lookup constructors for creating a right object from a string payload. `cid` is marshalled as a part of the payload. During unmarshalling, transport layer will extract the `cid`, create an object instance using the constructor associated with it, and then call unmarshall on the object.
For user classes that implement the [`Transportable`](#transportable) interface, Napa uses Constructor ID (`cid`) to lookup constructors for creating a right object from a string payload. `cid` is marshalled as a part of the payload. During unmarshalling, the transport layer will extract the `cid`, create an object instance using the constructor associated with it, and then call unmarshall on the object.

It's class developer's responsibility to choose the right `cid` for your class. To avoid conflict, we suggest to use the combination of module.id and class name as `cid`. Developer can use class decorator [`cid`](#decorator-cid) to register a user Transportable class automatically, when using TypeScript with decorator feature enabled. Or call [`transport.register`](#register) manually during module initialization.
It's the class developer's responsibility to choose the right `cid` for your class. To avoid conflict, we suggest to use the combination of module.id and class name as `cid`. Developer can use class decorator [`cid`](#decorator-cid) to register a user Transportable class automatically, when using TypeScript with decorator feature enabled. Or call [`transport.register`](#register) manually during module initialization.

### <a name="transport-context"></a> Transport context
There are states that cannot be saved or loaded in serialized form (like std::shared_ptr), or it's very inefficient to serialize (like JavaScript function). Transport context is introduced to help in these scenarios. TransportContext objects can be passed from one JavaScript VM to another, or stored in native world, so lifecycle of shared native objects extended by using TransportContext. An example of `Transportable` implementation using TransportContext is [`ShareableWrap`](./../../inc/napa/module/shareable-wrap.h).
There are states that cannot be saved or loaded in serialized form (like std::shared_ptr), or it's very inefficient to serialize (like JavaScript function). Transport context is introduced to help in these scenarios. TransportContext objects can be passed from one JavaScript VM to another, or stored in the native world, so lifecycle of shared native objects extended by using TransportContext. An example of the `Transportable` implementation using TransportContext is [`ShareableWrap`](./../../inc/napa/module/shareable-wrap.h).

### <a name="transporting-functions"></a> Transporting functions
JavaScript function is a special transportable type, through marshalling its definition into a [store](./store.md#intro), and generate a new function from its definition on target thread.

Highlights on transporting functions are:
- For the same function, marshall/unmarshall is an one-time cost on each JavaScript thread. Once a function is transported for the first time, later transportation of the same function to previous JavaScript thread can be regarded as free.
- Closure cannot be transported, but you won't get error when transporting a function. Instead, you will get runtime error complaining a variable (from closure) is undefined when you can the function later.
- `__dirname` / `__filename` can be accessed in transported function, which is determined by `origin` property of function. By default `origin` property is set to current working directory.
- Closure cannot be transported, but you won't get an error when transporting a function. Instead, you will get runtime error complaining a variable (from closure) is undefined when you can the function later.
- `__dirname` / `__filename` can be accessed in transported function, which is determined by `origin` property of the function. By default, `origin` property is set to the current working directory.

### <a name="transporting-built-in"></a> Transporting JavaScript built-in objects
JavaScript standard built-In objects in [the whitelist](#built-in-whitelist) can be transported among napa workers transparently. JavaScript Objects with properties in these types are also able to be transported. Please refer to [unit tests](./../../test/transport-test.ts) for detail.
JavaScript standard built-in objects in [the whitelist](#built-in-whitelist) can be transported among napa workers transparently. JavaScript Objects with properties in these types are also able to be transported. Please refer to [unit tests](./../../test/transport-test.ts) for detail.

An example [Parallel Quick Sort](./../../examples/tutorial/parallel-quick-sort) demonstrated transporting TypedArray (created from SharedArrayBuffer) among multiple Napa workers for efficient data sharing.

Expand Down Expand Up @@ -105,7 +105,7 @@ class B {
assert(!transport.isTransportable(new B()));
```
### <a name="register"></a> register(transportableClass: new(...args: any[]) => any): void
Register a `Transportable` class before transport layer can marshall/unmarshall its instances.
Register a `Transportable` class before the transport layer can marshall/unmarshall its instances.
User can also use class decorator [`@cid`](#cid-decorator) for class registration.

Example:
Expand All @@ -121,7 +121,7 @@ class A extends transport.AutoTransportable {
transport.register(A);
```
### <a name="marshall"></a> marshall(jsValue: any, context: TransportContext): string
Marshall a [transportable](#transportable-types) JavaScript value into a JSON payload with a [`TransportContext`](#transport-context). Error will be thrown if the value is not transportable.
Marshall a [transportable](#transportable-types) JavaScript value into a JSON payload with a [`TransportContext`](#transport-context).An Error will be thrown if the value is not transportable.

Example:
```js
Expand All @@ -132,7 +132,7 @@ var jsonPayload = transport.marshall(
console.log(jsonPayload);
```
### <a name="unmarshall"></a> unmarshall(json: string, context: TransportContext): any
Unmarshall an [transportable](#transportable-types) JavaScript value from a JSON payload with a [`TransportContext`](#transport-context). Error will be thrown if `cid` property is found and not registered with transport layer.
Unmarshall a [transportable](#transportable-types) JavaScript value from a JSON payload with a [`TransportContext`](#transport-context).An Error will be thrown if `cid` property is found and not registered with the transport layer.

Example:
```js
Expand All @@ -148,18 +148,18 @@ Save a shareable object in context.
Load a shareable object from handle.

### <a name="transportcontext-sharedcount"></a> context.sharedCount: number
Count of shareable objects saved in current context.
Count of shareable objects saved in the current context.

## <a name="transportable"></a> Interface `Transportable`
Interface for Transportable object.
Interface for the Transportable object.
### <a name="transportable-cid"></a> transportable.cid: string
Get accessor for [Constructor ID](#constructor-id). It is used to lookup constructor for payload of current class.
Get accessor for [Constructor ID](#constructor-id). It is used to lookup constructor for the payload of the current class.

### <a name="transportable-marshall"></a> transportable.marshall(context: TransportContext): object
Marshall transform this object into a plain JavaScript object with the help of [TransportContext](#transport-context).
Marshall transforms this object into a plain JavaScript object with the help of [TransportContext](#transport-context).

### <a name="transportable.unmarshall"></a> transportable.unmarshall(payload: object, context: TransportContext): void
Unmarshall transform marshalled payload into current object.
Unmarshall transforms marshalled payload into current object.

## <a name="transportableobject"></a> Abstract class `TransportableObject`
TBD
Expand Down
Loading