Skip to content

Commit

Permalink
Merge dc83050 into a0b4447
Browse files Browse the repository at this point in the history
  • Loading branch information
nikolai-shabalin committed Dec 4, 2020
2 parents a0b4447 + dc83050 commit ec4be27
Show file tree
Hide file tree
Showing 4 changed files with 10 additions and 10 deletions.
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,11 @@ npm install @discoveryjs/json-ext

Works the same as [`JSON.parse()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse) but takes `chunkEmitter` instead of string and returns [Promise](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise).

> NOTE: `reviver` paramenter is not supported yet, but will be added in next releases.
> NOTE: `reviver` parameter is not supported yet, but will be added in next releases.
When to use:
- Huge JSON needs to be parsed (e.g. >500MB on Node.js)
- Needed to reduce memory pressure. `JSON.parse()` needs to recieve the entire JSON before parsing it. With `parseChunked()` you may parse JSON as first bytes of it comes. This approach helps to avoid storing a huge string in the memory at a single time point and following GC.
- Needed to reduce memory pressure. `JSON.parse()` needs to receive the entire JSON before parsing it. With `parseChunked()` you may parse JSON as first bytes of it comes. This approach helps to avoid storing a huge string in the memory at a single time point and following GC.

[Benchmark](https://github.com/discoveryjs/json-ext/tree/master/benchmarks#parse-chunked)

Expand Down Expand Up @@ -90,7 +90,7 @@ parseChunked(() => ['{ "hello":', ' "world"}'])
Using with [fetch()](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API):

```js
async function loadData(url)
async function loadData(url) {
const response = await fetch(url);
const reader = response.body.getReader();

Expand Down Expand Up @@ -183,14 +183,14 @@ Result is an object:

##### async

Type: `Boolean`
Type: `Boolean`
Default: `false`

Collect async values (promises and streams) or not.

##### continueOnCircular

Type: `Boolean`
Type: `Boolean`
Default: `false`

Stop collecting info for a value or not whenever circular reference is found. Setting option to `true` allows to find all circular references.
Expand Down
6 changes: 3 additions & 3 deletions benchmarks/benchmark-utils.js
Original file line number Diff line number Diff line change
Expand Up @@ -242,17 +242,17 @@ async function collectGarbage() {
global.gc();
} else if (!exposeGcShowed) {
exposeGcShowed = true;
console.warn(chalk.magenta('Looks like script is forcing GC to collect garbarge, but coresponding API is not enabled'));
console.warn(chalk.magenta('Looks like script is forcing GC to collect garbage, but corresponding API is not enabled'));
console.warn(chalk.magenta('Run node with --expose-gc flag to enable API and get precise measurements'));
}
}

function captureStdio(stream, buffer) {
const oldWrite = stream.write;

stream.write = (chunk, encondig, fd) => {
stream.write = (chunk, encoding, fd) => {
buffer.push(chunk);
return oldWrite.call(stream, chunk, encondig, fd);
return oldWrite.call(stream, chunk, encoding, fd);
};

return () => stream.write = oldWrite;
Expand Down
2 changes: 1 addition & 1 deletion src/parse-chunked.js
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ class ChunkParser {

// In case Buffer/Uint8Array, an input is encoded in UTF8
// Seek for parts of uncompleted UTF8 symbol on the ending
// This makes sence only if we expect more chunks and last char is not multi-bytes
// This makes sense only if we expect more chunks and last char is not multi-bytes
if (!last && chunk[chunk.length - 1] > 127) {
for (let seqLength = 0; seqLength < chunk.length; seqLength++) {
const byte = chunk[chunk.length - 1 - seqLength];
Expand Down
2 changes: 1 addition & 1 deletion src/stringify-info.js
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ function stringLength(str) {
continue;
} else if (isTrailingSurrogate(code)) {
len = prevLeadingSurrogate
? len - 2 // surrogate pair (4 bytes), since we calulate prev leading surrogate as 6 bytes, substruct 2 bytes
? len - 2 // surrogate pair (4 bytes), since we calculate prev leading surrogate as 6 bytes, substruct 2 bytes
: len + 6; // \uXXXX
} else {
len += 3; // code >= 2048 is 3 bytes length for UTF8
Expand Down

0 comments on commit ec4be27

Please sign in to comment.