Skip to content

Releases: thkprado/jlens

Release list

jlens 0.19.0

jlens 0.19.0 Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 17 Oct 02:49
v0.19.0
e43b63b

jlens gets faster where jsonatapy, the Python package of jsonata-core, was ahead: a result is written straight into Python and JavaScript values, a predicate of literals is evaluated once, an object constructor over one item skips grouping, $formatNumber takes half the time, and the Python package reads dict fields with interned keys and allocates with mimalloc. With Python values in, jlens is now the fastest on ten of the twelve cases of benches/compare/python.py. The CLI is still not on npm (see 0.14.0).

Added

  • Results written straight into another representation (ADR 0017): implement output::Builder, and evaluate with Expression::evaluate_source_to or evaluate_document_to, whose closure writes the result with it, within the size limit of a result. evaluate and the other entry points write their serde_json result through the same walk, which makes the object constructor case about 6% faster natively.
  • input::Sources, with Sink::array_of and Sink::object_of: a source can write the arrays and objects of a level as members of one value, so that each costs one allocation instead of two (ADR 0010, amended).

Changed

  • A predicate built from literals alone, such as [[0..19]], is evaluated once instead of once per item: the number formatting case of the benchmarks takes less than half the time.
  • An object constructor over a single item, as in items.{...}, builds its object without grouping: the object constructor case of the benchmarks takes 22% less time (25% against a Document), and 20% in Python (24% against a Document).
  • Python: evaluate and evaluate_document build their result as Python objects directly, without a serde_json value in between, and create each key once per result. The object constructor case of benches/compare/python.py takes 27% less time (31% against a Document), the transform 14% (24%).
  • WebAssembly: evaluate and evaluateDocument build their result as JavaScript values directly (ADR 0017), through one call to a JavaScript helper that decodes each key once, instead of printing JSON text for JSON.parse. Against a Document, the object constructor case of benches/compare/compare.mjs takes 35% less time and the transform 24%; returning the whole input, $, 70%. evaluateJson still writes JSON text through serde_json values.
  • Python: the arrays and objects of a level read from Python values cost one allocation each instead of two: $count(items) over 200 dicts takes 19% less time, names over items 6%.
  • $formatNumber parses its picture once per evaluation, rounds without heap strings and detects ties from the bits of the number: the number formatting case of the benchmarks takes 51% less time (58% against a Document), and 41% in Python (53% against a Document). $round with decimals is about three times faster.
  • Python: the options of evaluate, evaluate_json, evaluate_document and evaluate_with_audit are keyword-only parameters instead of **options, so that a call builds no dict for them: two options cost 0.04 µs per call instead of 0.19 µs, and path of benches/compare/python.py takes 0.9 µs instead of 1.1. An unknown keyword is still a TypeError, whose message now names the method.
  • Python: each Expression keeps an interned str for every field name it reads, made on its first evaluation, so that a step into a dict neither builds nor hashes a str. In benches/compare/python.py, names over items takes 18% less time, the object constructor, group-by, filter + aggregate and strings 11% to 15% less, and the shipping rule 20% less.
  • Python: the package uses mimalloc as its allocator, as the jlens command does (ADR 0014, amended). In benches/compare/python.py on Linux, ten cases take 4% to 19% less time, recursion 7% more and the smallest evaluations 0.02 to 0.07 µs more; a wheel is 60 to 70 KB larger.

Fixed

  • $formatNumber with a zero-digit just below U+D800, whose digits would be lone surrogates, panicked; those digits are now U+FFFD, as a lone surrogate in a string literal is.
  • $fromMillis fails with D1001 for NaN, which $toMillis returns for a date that does not exist ($fromMillis($toMillis('2024-13-01'))), as for a date beyond JavaScript's range. It wrote the epoch, "1970-01-01T00:00:00.000Z"; jsonata-js writes "NaN" in every component.
  • An order-by whose keys are NaN ($^(0 / 0)) orders its items as jsonata-js does, which reads NaN as neither equal to nor less than another key and swaps both ways; jlens read it as equal and kept the order.

Install

cargo install jlens-cli                 # the jlens command, including jlens lsp
brew install thkprado/jlens/jlens       # the same, with Homebrew (macOS, Linux)
scoop bucket add jlens https://github.com/thkprado/scoop-jlens
scoop install jlens/jlens               # the same, with Scoop (Windows)
docker run --rm -i ghcr.io/thkprado/jlens --version   # the same, as a container
code --install-extension thkprado.jlens # the VS Code extension, with the same binary
cargo add jlens-core jlens-schema       # the Rust crates
npm install jlens-wasm                  # the browser ES module
pip install jlens                       # the Python package
Assets 8
Loading

jlens 0.18.0

jlens 0.18.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 16 Oct 17:31
v0.18.0
c9aa459

jlens tells an application which fields an expression reads, worked out from the expression alone: in Rust, JavaScript, Python and on the command line, for one rule or the union of several. The same answer lets it parse JSON text without the fields no evaluation reads, which evaluateJson, evaluate_json and jlens eval now do on their own. The CLI is still not on npm (see 0.14.0).

Added

  • The fields an expression reads (ADR 0016): Expression::reads returns a Reads, what the input and each binding need, as a tree of fields whose leaves are read whole, with the items of an array sharing a node. Where the analysis cannot follow (*, **, $keys, $string, $eval, a value given to a host function, a transform, an object in the result), it reads the value whole, so the answer is an upper bound: every case of the suite gives the same result or error on its input cut down to it. Reads::union combines the reads of several expressions, Read::cut cuts a value down to a read, and both have a JSON form.
  • HostFunction::without_input marks a host function that does not read the context of its call, and Expression::reads_with takes the bindings into account, so that a call to a marked function reads only its arguments.
  • Expression.reads and unionReads in the WebAssembly package, Expression.reads and union_reads in the Python package, and jlens reads, given an expression or several files (a result for each, or their union with --union). JavaScript and Python functions cannot read the context of their call, so both packages mark every one.
  • Document::parse_cut parses JSON text without the fields a read leaves out: a 1 MB document cut to two fields of each record parses in a quarter of the time. Document.fromJson in the WebAssembly package and Document.from_json in the Python package take a read too, so that one document serves the union of several expressions.

Changed

  • evaluateJson, evaluate_json and jlens eval parse their input without the fields the expression does not read: jlens eval '$sum(items.price)' on a 10 MB document runs in 27 ms instead of 76, and --ndjson in about half the time. The fields skipped are still checked for syntax, but a number too large for a float or a lone surrogate escape ("\ud800") in one of them no longer fails the parse. --explain, --audit and --trace-log still parse the whole input, since they show the values.

Install

cargo install jlens-cli                 # the jlens command, including jlens lsp
brew install thkprado/jlens/jlens       # the same, with Homebrew (macOS, Linux)
scoop bucket add jlens https://github.com/thkprado/scoop-jlens
scoop install jlens/jlens               # the same, with Scoop (Windows)
docker run --rm -i ghcr.io/thkprado/jlens --version   # the same, as a container
code --install-extension thkprado.jlens # the VS Code extension, with the same binary
cargo add jlens-core jlens-schema       # the Rust crates
npm install jlens-wasm                  # the browser ES module
pip install jlens                       # the Python package
Loading

jlens 0.17.0

jlens 0.17.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 16 Oct 08:11
v0.17.0
eb9bf13

jlens comes to Python: the package jlens on PyPI evaluates expressions against Python values, reading only what the expression reaches. A path step now reads its one field of an input object instead of the whole object, and values nested thousands of levels deep no longer overflow the stack. The CLI is still not on npm (see 0.14.0).

Added

  • The Python package jlens on PyPI (ADR 0014): Expression compiles an expression and evaluates it against Python values, read as the evaluation goes (evaluate), JSON text (evaluate_json) or a Document; bindings, Python functions as host functions, options and limits, audits (evaluate_with_audit), and JlensError with the JSONata code, position and token. abi3 wheels for Python 3.10 and later and abi3t wheels for free-threaded Python 3.15, on the seven release targets, and a source distribution. A guide in the book.
  • input::Source::read_field: a source can write one field of an object, by its key, which a path step reads without the rest of the object.

Changed

  • A path step reads its one field of an input object not read yet, instead of the object's every field (ADR 0010, amended): evaluate on a serde_json::Value takes 18.5 µs on the benchmark's names over items, against 152 µs, and 151 µs on the shipping rule, against 216 µs. Python's evaluate reads a dict the same way. Arrays and objects read so are kept and read once; a scalar is read again each time.

Fixed

  • Values nested thousands of levels deep, read from the input or built by a tail-recursive function, no longer overflow the stack. Returning, comparing, serializing, cloning or passing one to a host function fails with U1001 beyond 1,000 levels; dropping one never fails.
  • evaluateWithAudit in the Wasm package converts to UTF-16 only the spans of the audit's nodes: a captured value with a span or range field ({start, end}) was rewritten too when the expression held non-ASCII text.

Install

cargo install jlens-cli                 # the jlens command, including jlens lsp
brew install thkprado/jlens/jlens       # the same, with Homebrew (macOS, Linux)
scoop bucket add jlens https://github.com/thkprado/scoop-jlens
scoop install jlens/jlens               # the same, with Scoop (Windows)
docker run --rm -i ghcr.io/thkprado/jlens --version   # the same, as a container
code --install-extension thkprado.jlens # the VS Code extension, with the same binary
cargo add jlens-core jlens-schema       # the Rust crates
npm install jlens-wasm                  # the browser ES module
pip install jlens                       # the Python package
Loading

jlens 0.16.0

jlens 0.16.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 15 Oct 18:47
v0.16.0
1f9686d

jlens closes its last speed gaps with jsonata-core, another JSONata engine in Rust: group-by is no longer quadratic in the size of a group, and paths of names and names mapped over many items take a fraction of the time they did. The CLI is still not on npm (see 0.14.0).

Changed

  • Group-by (items{type: ...}) joins the items of each group once, instead of copying the group at every item, which made it quadratic in the size of the group. The group-by case of the benchmark runs 64% faster, and a group of 200 items 8 times faster.
  • A path of names, such as customer.name or $i.code, looks its fields up directly while no array is met, instead of building a sequence at each step: the path case of the benchmark runs 67% faster (0.68 to 0.23 µs), sort + map 33%.
  • A grouping with a literal key, such as 'total' in items{type: {'total': ...}}, finds its group by the literal's text instead of building a string per item: group-by runs 12% faster.
  • A path step that reads an array passes it on without copying it, and the step after it maps over its items in place: $sum(items.quantity) runs 28% faster, $count(items) 29%.
  • The benchmark has two new cases: a name mapped over the items, and an object constructor.

Install

cargo install jlens-cli                 # the jlens command, including jlens lsp
brew install thkprado/jlens/jlens       # the same, with Homebrew (macOS, Linux)
scoop bucket add jlens https://github.com/thkprado/scoop-jlens
scoop install jlens/jlens               # the same, with Scoop (Windows)
docker run --rm -i ghcr.io/thkprado/jlens --version   # the same, as a container
code --install-extension thkprado.jlens # the VS Code extension, with the same binary
cargo add jlens-core jlens-schema       # the Rust crates
npm install jlens-wasm                  # the browser ES module
Loading

jlens 0.15.0

jlens 0.15.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 15 Oct 04:16
v0.15.0
c564855

jlens comes to VS Code: an extension on the Visual Studio Marketplace and Open VSX highlights JSONata and runs the language server from the bundled release binary. The jlens command itself is faster, with mimalloc as its allocator. The CLI is still not on npm (see 0.14.0).

Added

  • A VS Code extension, thkprado.jlens on the Visual Studio Marketplace and Open VSX (ADR 0013): highlighting of .jsonata files, and the diagnostics, completions and hovers of jlens lsp, which it starts from the release binary bundled in the package of each release platform, from jlens.server.path, or from the PATH. VS Code checks jlens.json against its JSON Schema as it is edited. Each release publishes the extension with the same version.
  • cargo xtask ab --b-allocator mimalloc measures the allocator alone, and --target builds for another local target, such as musl.

Changed

  • The jlens command uses mimalloc as its global allocator on every platform. Evaluations that allocate run 28 to 91% faster with the Linux release binaries, which are static musl binaries, and 14 to 56% faster on Windows. The library still leaves the choice of allocator to applications.

Install

cargo install jlens-cli                 # the jlens command, including jlens lsp
brew install thkprado/jlens/jlens       # the same, with Homebrew (macOS, Linux)
scoop bucket add jlens https://github.com/thkprado/scoop-jlens
scoop install jlens/jlens               # the same, with Scoop (Windows)
docker run --rm -i ghcr.io/thkprado/jlens --version   # the same, as a container
code --install-extension thkprado.jlens # the VS Code extension, with the same binary
cargo add jlens-core jlens-schema       # the Rust crates
npm install jlens-wasm                  # the browser ES module
Loading

jlens 0.14.0

jlens 0.14.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 13 Oct 23:12
v0.14.0
a954e77

The jlens command installs from the package managers people already use: Homebrew, Scoop and a container image on ghcr.io, with npm (npx jlens-cli) to follow. Each channel carries the attested release binaries, never rebuilt.

Added

  • The pages of the crates on crates.io and of jlens-wasm on npm link the book and the web editor; jlens-cli names the CLI guide as its documentation.
  • The jlens command on npm (ADR 0012): npx jlens-cli or npm install --save-dev jlens-cli runs the release binary of the platform, which npm installs from one of the packages jlens-cli-<os>-<cpu> (Linux x64 and arm64, macOS x64 and arm64, Windows x64), published with provenance and without install scripts. Not available yet: npm's spam detection rejected the new package jlens-cli-win32-x64, so jlens-cli 0.14.0 is not published; it will be once npm lifts the block.
  • The jlens command on Homebrew, for macOS and Linux: brew install thkprado/jlens/jlens installs the release binary from the tap thkprado/homebrew-jlens, which each published release updates.
  • The jlens command on Scoop, for Windows: after scoop bucket add jlens https://github.com/thkprado/scoop-jlens, scoop install jlens/jlens installs the release binary, and each published release updates the bucket.
  • A container image, ghcr.io/thkprado/jlens, for linux/amd64 and linux/arm64: the static release binary and nothing else, tagged with the version, the minor version and latest, and attested like the archives.

Install

cargo install jlens-cli                 # the jlens command, including jlens lsp
brew install thkprado/jlens/jlens       # the same, with Homebrew (macOS, Linux)
scoop bucket add jlens https://github.com/thkprado/scoop-jlens
scoop install jlens/jlens               # the same, with Scoop (Windows)
docker run --rm -i ghcr.io/thkprado/jlens --version   # the same, as a container
cargo add jlens-core jlens-schema       # the Rust crates
npm install jlens-wasm                  # the browser ES module
Loading

jlens 0.13.0

jlens 0.13.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 13 Oct 20:31
v0.13.0
04f3017

Host functions call the functions they receive (ADR 0011), in Rust and in WebAssembly: a helper written in the host language can apply a lambda, a built-in function or a partial application passed to it, within the evaluation's limits.

Added

  • The site links the repository: the book from its menu bar, the web editor from its header.
  • Host functions call the functions they receive (ADR 0011). In Rust, HostCall::function(i) gives a Callable for an argument that is a function (a lambda, a built-in function, a partial application, a chain or a host function); Callable::call applies it to JSON arguments, within the evaluation's depth, timeout, cancellation and size limits.
  • In WebAssembly, a JavaScript host function receives a JSONata function argument as a JavaScript function that calls it, with an arity property, as in jsonata-js (Callable in the TypeScript declarations). It is synchronous, and calling it after the host function has returned throws the new error U1005.
  • The web editor offers a host function, $try(step, fallback), which calls the function it receives and catches its error, and a Host functions example that uses it.

Changed

  • In WebAssembly, a function passed to a host function no longer arrives as undefined: a host function that tested for undefined sees a function instead.

Install

cargo install jlens-cli                 # the jlens command, including jlens lsp
cargo add jlens-core jlens-schema       # the Rust crates
npm install jlens-wasm                  # the browser ES module
Loading

jlens 0.12.0

jlens 0.12.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 13 Oct 17:54
v0.12.0
da4bc85

jlens is online: the book and the web editor are published at https://thkprado.github.io/jlens/, built from each release tag. The editor evaluates in a Web Worker and shares expressions by address, without a server.

Added

  • cargo xtask site builds the book and the web editor as one static site in target/site, with the editor in editor/; cargo xtask serve --site previews it. The web editor imports the package by its name, jlens-wasm, through its import map.
  • The book and the web editor are published at https://thkprado.github.io/jlens/ by the new Pages workflow, at each release tag. The crates and the npm package name the site as their homepage.
  • The web editor shares expressions: its Share button puts the expression, the input document and the bindings in the address (#s=, compressed), so that no server stores them, and opening that address restores them. Its header links the book.

Changed

  • The web editor evaluates in a Web Worker, so a slow expression no longer blocks typing; an evaluation still running when the expression changes is dropped.

Install

cargo install jlens-cli                 # the jlens command, including jlens lsp
cargo add jlens-core jlens-schema       # the Rust crates
npm install jlens-wasm                  # the browser ES module
Loading

jlens 0.11.1

jlens 0.11.1

Choose a tag to compare

View all tags
@github-actions github-actions released this 12 Oct 19:51
v0.11.1
0301282

A fix for dates formatted with a timezone that is not a number.

Fixed

  • $fromMillis and $now fail with the new code D3150 when the timezone does not start with a number ($fromMillis(0, undefined, "two")). They wrote a date with a garbled offset ("1970-01-01T00:00:00.0000N:aN"), where jsonata-js writes "NaN" in every component (see the compatibility page).

Install

cargo install jlens-cli                 # the jlens command, including jlens lsp
cargo add jlens-core jlens-schema       # the Rust crates
npm install jlens-wasm                  # the browser ES module
Loading

jlens 0.11.0

jlens 0.11.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 12 Oct 19:19
v0.11.0
c261591

Common expressions evaluate up to three times faster: a field of a single value is a lookup, comparisons with a string literal build no value, order-by evaluates each key once, and an evaluation sets up only what it uses. Against a Document converted once, the WebAssembly package is faster than jsonata-js on every case of the performance page.

Changed

  • The order-by operator (items^(price)) evaluates each item's keys once, on first use, instead of evaluating both items' keys at every comparison: the sort + map benchmark takes 842 µs instead of 1.90 ms. Results and errors are unchanged; a key with a side effect, such as one using $random, is drawn once per item (see the compatibility page).
  • A block ((...)) creates a frame for its variables only when its expressions may bind one: items.(unitPrice) takes 77 µs instead of 83 µs. NodeKind::Block has a new field, frame, set by post-processing.
  • A field name evaluated on a single value (price in items[price > 10]) is a lookup, without the sequences of a general path: the filter + aggregate benchmark takes 56 µs instead of 161 µs, the shipping rule 152 µs instead of 414 µs, group-by 575 µs instead of 842 µs.
  • = and != with a string literal (type = 'electronic') compare with the literal's text instead of building a string value each time: the filter + aggregate benchmark takes 44 µs instead of 56 µs, the shipping rule 127 µs instead of 150 µs.
  • The result is converted to JSON in one pass, counting its size against Options::max_result_size while converting it, instead of measuring it first: returning the whole 200-order input ($) takes 236 µs instead of 271 µs.
  • An evaluation sets up less before it starts: the system clock is read once, the first time $now, $millis, $random or $shuffle needs it, instead of twice at the start; the start time only with a timeout; and the input ($$) is kept in the root frame instead of being bound as a variable. Evaluating 1 takes 106 ns instead of 350 ns, the path benchmark 676 ns instead of 927 ns. Without Options::now, the timestamp is the clock when the evaluation first needs it rather than when it starts.

Install

cargo install jlens-cli                 # the jlens command, including jlens lsp
cargo add jlens-core jlens-schema       # the Rust crates
npm install jlens-wasm                  # the browser ES module
Loading