Skip to content

Releases: adam2go/purejq

purejq 0.3.1

Choose a tag to compare

@adam2go adam2go released this 17 Jun 01:39

Robustness release — same jq semantics (777 conformance tests pass unchanged), no input can crash the parser.

An adversarial fuzz sweep (120k calls) found three inputs that escaped jq's error model; all now raise a clean JqParseError:

  • malformed \u escapes ("\uZZZZ") raised ValueError
  • Unicode-but-not-ASCII characters after . (e.g. ) raised AttributeError
  • deeply nested programs ([[[[…, {a:{a:…) raised RecursionError and could overflow the C stack under a raised recursion limit

The parser now caps structural nesting at a fixed depth that trips under CPython's default recursion limit on every platform, with a backstop catch for long flat pipe/operator chains.

purejq 0.3.0 — streaming + more CLI flags

Choose a tag to compare

@adam2go adam2go released this 16 Jun 04:59

Two feature areas this release, both verified byte-identical to jq 1.8.1.

Constant-memory streaming for huge files

New --stream flag parses input incrementally and emits jq's [path, leaf]
events, so you can process files larger than RAM:

purejq -n --stream 'fromstream(1|truncate_stream(inputs)) | select(.v > 0)' huge.json

On a 56 MB array of 2M records, reconstructing every element peaks at
22 MB RSS streaming vs 612 MB loading the array whole — 28x less memory,
identical output. Matches jq --stream event-for-event (scalars, nesting,
empty containers, top-level arrays, multi-value input); truncated input is
reported as a parse error.

More jq CLI flags

-R/--raw-input, -S/--sort-keys, -a/--ascii-output, --indent N,
--tab — all cross-checked against jq 1.8.1. Plus -V/--version and a
py.typed marker for type checkers.

Conformance unchanged at 751/781. New CLI test suite (24 cases).

purejq 0.2.1

Choose a tag to compare

@adam2go adam2go released this 15 Jun 14:08

Minor polish release.

  • purejq -V / --version, matching jq
  • README now leads with an animated demo (reproducible via demo/demo.tape)
  • Marked Alpha (was Pre-Alpha): 96.2% of jq's test suite, on PyPI, full CI matrix

No behavior changes to the engine; conformance unchanged at 751/781.

purejq 0.2.0 — static binding, fast paths, real benchmarks

Choose a tag to compare

@adam2go adam2go released this 10 Jun 12:46

A second performance round on top of 0.1.1, plus a benchmark suite that measures purejq against both the jq PyPI package (C bindings) and the jq binary.

Headline numbers

Embedded in Python (100k objects, in-process) — purejq is 6–40x faster than the jq C-binding package, which pays a full JSON serialization round-trip per call:

workload purejq jq PyPI (C bindings)
filter + count 56 ms 485 ms
group_by 114 ms 765 ms
transform + sort 141 ms 943 ms

End-to-end CLI (93 MB file, 1M objects) — purejq beats the C jq binary on parse-dominated runs:

workload purejq jq 1.8
single lookup 0.5 s 1.6 s
filter + count 1.1 s 2.0 s
group_by 2.3 s 4.0 s

PyPy: another 2–9x on top (map+aggregate: 2 ms).

Engine changes

  • Static call binding: names never redefined in a program are resolved to their prelude/Python builtin at compile time
  • Single-output fast paths: constants, variables, field chains and arithmetic over them compile to plain callables; binops, object literals, pipes, and builtin arguments (select/map/sort_by keys) skip generator machinery
  • reduce/foreach reuse one environment per fold

Conformance unchanged: 751/781 (96.2%) of jq's official test suite. Run the benchmarks yourself: python3 tools/bench.py 1000000.

purejq 0.1.1 — performance release

Choose a tag to compare

@adam2go adam2go released this 10 Jun 12:16

2–11x faster on common workloads, with conformance unchanged (751/781 of jq's official test suite).

Benchmarks (100k objects, CPython 3.13, M-series)

workload 0.1.0 0.1.1 speedup
field-access stream 19 ms 11 ms 1.7x
filter + count 347 ms 77 ms 4.5x
map + aggregate 48 ms 24 ms 2.0x
group_by 577 ms 121 ms 4.8x
transform + sort 1035 ms 220 ms 4.7x
nested iteration + unique 697 ms 64 ms 10.9x

What changed

  • Native C sorting when sort keys are uniformly strings or numbers — sort, sort_by, group_by, unique, unique_by no longer pay for Python-level comparisons in the typical case
  • Scalar fast path in the comparison core used by ==, <, sort, et al.
  • select, map, and not are now Python-native builtins instead of jq prelude definitions
  • Constant-key object construction ({id, name, x: ...}) skips the generator product machinery
  • .foo and .[] on the current input skip one generator layer

Reproduce with python3 tools/bench.py 100000 (compares against the system jq if installed).

purejq 0.1.0 — first public release

Choose a tag to compare

@adam2go adam2go released this 10 Jun 11:59

The first public release of purejq: a pure Python implementation of jq, in the spirit of gojq (Go) and jaq (Rust).

Highlights

  • No C extension, no binary — if Python runs, purejq runs: Pyodide/WASM, sandboxes, anywhere pip install is all you get
  • 96.2% of jq's official test suite (751/781 cases) — conformance is measured against jq's own test files, vendored in-repo; every known difference is documented
  • Full expression language: paths, all assignment operators, reduce/foreach, try/catch, label/break, ?// destructuring, string interpolation, all @formats, regex builtins, tostream/fromstream, date builtins, jq 1.8 additions
  • jq-compatible CLI (purejq) with -n -r -j -c -s -e -f --arg --argjson
  • Python API: compile once, run on many inputs; outputs are lazy iterators, infinite streams welcome
  • Arbitrary-precision integers (like gojq): big ints stay exact instead of rounding to doubles
  • Tested on CPython 3.9–3.14 and PyPy in CI; zero runtime dependencies (purejq[speed] optionally adds orjson for faster CLI input parsing)

Install

pip install purejq

Known limitations

  • jq's module system (import/include/modulemeta) is not implemented yet
  • Integer arithmetic is exact rather than double-rounded (deliberate)
  • A few error messages are worded differently

See expected_failures.txt for the complete, honest list.