💎 @ton.js/json-parser

A customizable JSON parser that is 100% compatible with native implementation (JSON.parse()), can be used as a polyfill, adds TC39 source access proposal and additional features, have better security like secure-json-parse and is… 25 times slower…

Rationale

The native implementation of JSON parser in JavaScript (i.e. JSON.parse()) doesn't allow to fully customize the parsing behavior. The JSON specification allows documents to include numbers of arbitrary and unlimited size. However, EcmaScript is using IEEE 754 standard to represent numbers internally and doesn't support numbers of the unlimited size. This leads to the data loss when JSON.parse() is attempting to read big numbers from the JSON documents.

Modern versions of JavaScript support a special BigInt data type specifically designed to represent integer numbers of unlimited size, but there is no way to tell JSON.parse() to use BigInt for parsing numbers.

Web 3.0 community tends to use very big numbers to represent cryptocurrency values, like the number of nanocoins in transaction. This library was designed as a workaround that allows to read big integer numbers from the JSON documents. However, considering that it's a full-fledged customized JSON parser you can use it for other cases as well.

Features

modern cross-platform multi-format package that can be used in any JavaScript environment,
written in pure super-strict TypeScript with 100% type coverage,
minimum size package with zero dependencies,
robust security practices,
100% compatible with the JSON standard and native JSON.parse() implementation, can be used as a polyfill,
future-compatible by implementing the Stage-3 TC39 source access proposal and additional features,
adds special handling for __proto__ and constructor.prototype object properties to implement better security,
extensively covered by unit tests and tested out on multiple real-life JSON samples,
parses 1 MB of nested JSON in ~40 ms. (25 times slower than native implementation).

Normal usage (ponyfill)

Ponyfill is a side effects free package that provides an alternative implementation and doesn't affect the native behavior of the system.

Install the package first:

npm install --save @ton.js/json-parser

Please see the examples project for the complete examples.

Simple parsing

import { parseJson } from '@ton.js/json-parser';

interface DocumentType {
 // …
}

const object = parseJson<DocumentType>('{ … }');
// object type will be: DocumentType

Using native reviver

interface DocumentType {
  birthDate: Date;
}

const content = '{ "birthDate": "1989-08-16T10:20:30.123Z" }';

const object = parseJson<DocumentType>(content, (key, value) => (
  (key.endsWith('Date') ? new Date(value) : value)
));

assert(object.birthDate instanceof Date);

Using reviver with source text access

interface DocumentType {
  valueBN: bigint;
}

const content = '{ "valueBN": 12345678901234567890 }';

const object = parseJson<DocumentType>(content, (key, value, context) => (
  (key.endsWith('BN') ? BigInt(context.source) : value)
));

assert.equal(typeof object.valueBN, 'bigint');

Using key path

interface DocumentType {
  foo: {
    bar: {
      value: bigint;
    };
  };
}

const content = '{ "foo": { "bar": { "value": 12345678901234567890 } } }';

const object = parseJson<DocumentType>(content, (key, value, context) => (
  (context.keys.join('.') === 'foo.bar.value' ? BigInt(context.source) : value)
));

assert.equal(typeof object.foo.bar.value, 'bigint');

Throwing on prototypes

In order to prevent parsed object prototype override the JSON parser will automatically skip __proto__ and constructor.prototype properties. However, you can use the throwOnProto: true option to make this behavior more explicit — the parser will throw an error instead:

const content = '{ "foo": true, "__proto__": {} }';

try {
  parseJson(content, undefined, {
    throwOnProto: true,
  });

} catch (error: any) {
  // Forbidden object property name: "__proto__"
  console.log(error);

}

Polyfill usage

The polyfill implements the Stage-3 TC39 source access proposal and adds some additional useful features.

Polyfill is a package that when globally imported, overrides the behavior of the native JSON.parse() method. The polyfill will detect the number of arguments that your reviver function has and will use custom JSON parser implementation only when it has three parameters (i.e. context), in other case the native implementation will be used instead.

Please see the polyfill examples project for the complete examples.

Be advised, that polyfill can't detect if you are using arguments[2] to access the context, so make sure to use a normal function parameter.

Prerequisites

1). Install the polyfill package:

npm install --save @ton.js/json-parse-polyfill

2). Import the package only once as close to the beginning of your program as possible:

import '@ton.js/json-parse-polyfill';

(function main() {
  // …
})();

Simple parsing

interface DocumentType {
 // …
}

// Native implementation will be used
const object = <DocumentType> JSON.parse('{ … }');

Using native reviver

interface DocumentType {
  birthDate: Date;
}

const content = '{ "birthDate": "1989-08-16T10:20:30.123Z" }';

// Native implementation will be used
const object = <DocumentType> JSON.parse(content, (key, value) => (
  (key.endsWith('Date') ? new Date(value) : value)
));

assert(object.birthDate instanceof Date);

Using reviver with source text access

import type { ReviverFunc } from '@ton.js/json-parse-polyfill';

interface DocumentType {
  valueBN: bigint;
}

const content = '{ "valueBN": 12345678901234567890 }';

// Custom implementation will be used
const reviver: ReviverFunc = (key, value, context) => (
  (key.endsWith('BN') ? BigInt(context.source) : value)
);

const object = <DocumentType> (
  JSON.parse(content, reviver as any)
);

assert.equal(typeof object.valueBN, 'bigint');

Using key path

import type { ReviverFunc } from '@ton.js/json-parse-polyfill';

interface DocumentType {
  foo: {
    bar: {
      value: bigint;
    };
  };
}

const content = '{ "foo": { "bar": { "value": 12345678901234567890 } } }';

const reviver: ReviverFunc = (key, value, context) => (
  ((context.keys.join('.') === 'foo.bar.value')
    ? BigInt(context.source)
    : value
  )
);

// Custom implementation will be used
const object = <DocumentType> (
  JSON.parse(content, reviver as any)
);

assert.equal(typeof object.foo.bar.value, 'bigint');

Parsing by rules

A dedicated package is provided to support JSON deserialization using a special rules syntax. This simplifies writing custom revivers for your data types.

Prerequisites

This package requires a JSON.parse implementation that supports TC39 source access proposal and additional features. You can either install the polyfill or specify the parser function directly via the options.

Install the package:

npm install --save @ton.js/json-parser-rules

Example

Please see the parse by rules example project for the complete examples.

import '@ton.js/json-parse-polyfill';

import { strict as assert } from 'node:assert';

import type { Reviver } from '@ton.js/json-parser-rules';
import { parseJsonSchema } from '@ton.js/json-parser-rules';


//===============//
// JSON DOCUMENT //
//===============//

const content = (`
{
  "foo": {
    "hex": {
      "value": "416C6C20796F75206E656564206973206C6F7665"
    },
    "bar": {
      "baz": {
        "valueBig": 11145678901234567890,
        "hex": "466f72204169757221"
      }
    },
    "feeBig": 22245678901234567890,
    "myArray": [0, 1, "2023-01-20T19:30:45.904Z", 3, {
      "hex": {
        "value": "616c6c20796f75722062617365206172652062656c6f6e6720746f207573"
      }
    }]
  }
}
`);


//==================//
// RESULT INTERFACE //
//==================//

interface ParseResult {
  foo: {
    hex: HexBag;
    bar: {
      baz: {
        valueBig: bigint;
        hex: string;
      };
    };
    feeBig: bigint;
    myArray: [number, number, Date, number, {
      hex: HexBag;
    }];
  };
}

interface HexBag {
  value: string;
}


//==========//
// REVIVERS //
//==========//

const bigIntReviver: Reviver = (
  context => BigInt(context.source)
);

const dateReviver: Reviver = (
  context => new Date(context.value)
);

const hexReviver: Reviver = (
  context => Buffer.from(context.value, 'hex').toString()
);


//==================//
// PARSING BY RULES //
//==================//

const object = parseJsonByRules<ParseResult>(content, {
  rules: [{
    pattern: '**.*Big',
    reviver: bigIntReviver,
  }, {
    pattern: [
      '**.hex.value',
      'foo.bar.baz.hex',
    ],
    reviver: hexReviver,
  }, {
    pattern: 'foo.myArray.2',
    reviver: dateReviver,
  }],
});


//=================//
// TESTING RESULTS //
//=================//

assert.equal(typeof object.foo.bar.baz.valueBig, 'bigint');
assert.equal(object.foo.bar.baz.valueBig, 11145678901234567890n);

assert.equal(typeof object.foo.feeBig, 'bigint');
assert.equal(object.foo.feeBig, 22245678901234567890n);

assert.equal(
  object.foo.hex.value,
  'All you need is love'
);

assert.equal(
  object.foo.bar.baz.hex,
  'For Aiur!'
);

assert.equal(
  object.foo.myArray[4].hex.value,
  'all your base are belong to us'
);

assert.equal(
  object.foo.myArray[2].toISOString(),
  '2023-01-20T19:30:45.904Z'
);

If you don't want to use the polyfill you can do this instead:

import { parseJson } from '@ton.js/json-parser';
import { parseJsonByRules } from '@ton.js/json-parser-rules';

const result = parseJsonByRules('{ … }', {
  rules: [],
  parser: parseJson,
});

API

parseJsonByRules<Type>(source, options): Type;

Parses the specified JSON document according to the provided deserialization rules.

Property	Type	Description
`* source`	`string`	The JSON document to parse.
`* options`	`Options`	An options object.
`* options.rules`	`ReviverRule[]`	A list of parsing rules.
`options.parser`	`ParserFunc`	An optional `JSON.parse` compatible implementation.

ReviverRule

Specifies the reviver function that needs to be applied for the specified patterns.

Property	Type	Description
`* pattern`	`string` or `string[]`	A pattern or a list of patterns to match against parsing object keys.
`* reviver`	`Reviver`	A reviver function to apply when the pattern is getting matched.

Reviver = (context: ReviverContext) => any

Reviver is a callback function that is getting called to deserialize specific key-value pairs from the JSON document. It must return the deserialized value.

ReviverContext

Reviver function will return a context object with the following information that can be used to deserialize the document data.

Property	Type	Description
`value`	`any`	A parsed value.
`source`	`string`	An original source text of the value.
`path`	`string`	A path of keys delimited by dots.

Rule format

The following rule formats are supported:

Pattern	Description
`foo.bar.baz`	A direct key path that is getting matched exactly.
`foo.*.bar`	The `*` part will match anything except the dot, e.g.: `foo.baz.bar`, `foo.qux.bar`
`foo.date*`	This will match any keys under the `foo` that start with `date`, e.g. `foo.dateCreated`
`foo.*Date`	This will match any keys under the `foo` that end with `Date`, e.g. `foo.expirationDate`
`**`	This will apply to all the keys
`**.foo`	This will match all the `foo` keys nested in the object, e.g. `my.deeply.nested.foo` or `foo`
`foo.bar.**`	This will match every key under the `foo.bar` path, e.g. `foo.bar.qux.quux`
`foo..bar..Date`	This will match all the keys ending with `Date` under the `foo.X.bar`, where `X` could be anything

Benchmarks

Normal dataset

name	ops	margin	percentSlower
native	12535	0.37	0
parse-json	460	0.66	96.33

Big dataset

name	ops	margin	percentSlower
native	29	1.23	0
parse-json	3	1.28	89.66

Security

Please see our security policy.

Contributing

Want to help? Please see the contributing guide.

Support

If you have any questions regarding this library or TON development in general — feel free to join our official TON development Telegram group.

The MIT License (MIT)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Name		Name	Last commit message	Last commit date
Latest commit History 16 Commits
.github/workflows		.github/workflows
benchmark		benchmark
bin		bin
examples		examples
packages		packages
readme		readme
.editorconfig		.editorconfig
.gitattributes		.gitattributes
.gitignore		.gitignore
CONTRIBUTING.md		CONTRIBUTING.md
README.md		README.md
SECURITY.md		SECURITY.md
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml
tsconfig.common.json		tsconfig.common.json

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

💎 @ton.js/json-parser

Rationale

Contents

Features

Normal usage (ponyfill)

Simple parsing

Using native reviver

Using reviver with source text access

Using key path

Throwing on prototypes

Polyfill usage

Prerequisites

Simple parsing

Using native reviver

Using reviver with source text access

Using key path

Parsing by rules

Prerequisites

Example

API

Rule format

Benchmarks

Normal dataset

Big dataset

Security

Contributing

Support

The MIT License (MIT)

About

Releases

Packages

Languages

ton-js/json-parser

Folders and files

Latest commit

History

Repository files navigation

💎 @ton.js/json-parser

Rationale

Contents

Features

Normal usage (ponyfill)

Simple parsing

Using native reviver

Using reviver with source text access

Using key path

Throwing on prototypes

Polyfill usage

Prerequisites

Simple parsing

Using native reviver

Using reviver with source text access

Using key path

Parsing by rules

Prerequisites

Example

API

Rule format

Benchmarks

Normal dataset

Big dataset

Security

Contributing

Support

The MIT License (MIT)

About

Resources

Security policy

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages