Repository navigation
v0.2.0
qj 0.2.0 is a rewrite of its core: qj now is jq 1.8.1, ported to Rust. jq's value and
number model, parser, compiler, bytecode VM, C builtins and builtin.jq run as jq's own
code does. qj's speed now comes from its input layer, which keeps jq's results exact:
- simdjson parsing;
- memory-mapped and streaming input;
- ordered parallel processing of NDJSON;
- evaluating simple filters directly on simdjson's tape.
Conformance
Conformance is checked byte for byte against the jq 1.8.1 binary: stdout, stderr and exit
code. The checks cover jq's own test suites, a corpus of about 39,000 cases (every builtin,
CLI option, input edge case and error path) and differential fuzzing, on macOS and Linux.
- Default mode matches jq on every case except qj's own
--help/--versiontext.
Where jq crashes or hangs, qj gives a sane answer instead. QJ_JQ_COMPAT=1means "be exactly jq 1.8.1, bugs included", and matches on every
case, with nothing normalized:- jq's name, help and version text;
- jq's crashes, with stack overflows at jq's own depth per OS, and its hangs;
- qj's extras turned off (globbing,
.gz/.zst,--threads).
- jq's own test suites pass 2,903/2,903, byte-exact.
What still differs, and why, is in docs/COMPATIBILITY.md.
Behaviour changes from 0.1.x
Output now matches jq exactly where 0.1.x differed:
- number formatting follows jq 1.8.1's decNumber literals, so
1.10stays1.10and
1e2prints as1E+2; - string escapes are jq's (
\u001b, not the input's\u001B); - error messages, exit codes, and the interleaving of stdout and stderr are jq's.
Other changes:
QJ_JQ_COMPAT=1changed meaning, as above. Arithmetic now always follows jq's
precision; it is no longer a separate switch.- The old evaluator and its NDJSON fast paths are gone, and so are
QJ_COREand
QJ_NO_FAST_PATH.--jsonland--debug-timingare still accepted and do nothing. - On Apple Silicon, qj uses every non-efficiency core, so Super and Performance cores on
M5 Pro/Max.
Performance
Measured against 0.1.4 on the same M5 Max:
| 0.1.4 | 0.2.0 | |
|---|---|---|
NDJSON 1.2 GB, -c . |
681 ms | 77 ms |
NDJSON, {type, repo: .repo.name, actor: .actor.login} |
147 ms | 57 ms |
| JSON 49 MB, pretty print | 192 ms | 48 ms |
JSON, group_by / sort_by / unique |
135–137 ms | 61–64 ms |
reduce building a 50,000-key object |
10.3 s | 10 ms |
| peak memory, 3.4 GB NDJSON | 3.5–3.8 GB | 100–160 MB |
Some cases are slower than in 0.1.4, though still well ahead of jq:
- single-threaded
select(...)over raw NDJSON, which 0.1.4 answered by scanning bytes
and copying lines through unchanged, so its output wasn't jq's; - tight loops on the VM, such as
limit/repeat.
See the README for comparisons with jq, jaq and
gojq.
Platforms
Prebuilt binaries for macOS (arm64, x86_64) and Linux (x86_64, aarch64), or
cargo install qj. Windows isn't supported.
Full changelog: v0.1.4...v0.2.0