Skip to content

v0.2.1

Choose a tag to compare

@github-actions github-actions released this 02 Oct 02:30
· 36 commits to main since this release

qj 0.2.1 makes 0.2.0 build everywhere 0.1.x did, and brings Windows back with a prebuilt
binary.

Platforms

0.2.0 is a port of jq's C code, which leans on the C library, and it only built on macOS
and glibc Linux. cargo install qj failed elsewhere (#3).
0.2.1 builds and runs on:

  • FreeBSD, NetBSD and Linux with musl (Alpine, static builds). CI builds qj on each and
    runs its test suite there. Before each release, jq_diff also runs on each against jq
    1.8.1 built there, from its release tarball. qj matches 39,230 of 39,282 cases on musl,
    39,205 of 39,215 on FreeBSD and 39,206 of 39,215 on NetBSD. What's left is:

    • qj's own help and version text (6 cases on each);
    • compat mode's stack-overflow depths, which were measured on macOS and glibc Linux
      only (2 or 3 cases on each);
    • on musl, the order of large outputs when stdout and stderr share a file (42 cases).
  • Windows, with a prebuilt qj-x86_64-pc-windows-msvc.zip. qj uses the same C
    runtime (UCRT) as jq's Windows release binary and does what that binary does:

    • text-mode I/O: \r\n line endings, Ctrl-Z ends input, -b for binary mode;
    • the C runtime's time functions, with jq's own strptime;
    • UTF-16 output to the console.

    CI compares qj with jq-windows-amd64.exe byte for byte on jq_diff's cases (the
    33,102 filter cases of jq's test suites and qj's corpus in every mode, and its
    command-line cases), plus cases built around what Windows does differently. All match
    except:

    • qj's own help text;
    • four lgamma results that differ in the last digit (jq.exe uses MinGW's lgamma);
    • one program nested deep enough to crash jq.exe's 2 MB stack, which qj answers.

    QJ_JQ_COMPAT=1 isn't supported on Windows. The known differences are listed in
    docs/LIMITATIONS.md.

Prebuilt binaries: macOS (arm64, x86_64), Linux (x86_64, aarch64) and Windows (x86_64), or
cargo install qj.

Fixes

  • Where musl's and the BSDs' C libraries differ from glibc's, qj now does what their
    jq does:

    • each C library's wording of a failed assert(), and the BSDs' flush of stdout when
      aborting;
    • the environment's locale on FreeBSD and NetBSD, so dates are localized as in jq;
    • musl's stdout, which writes its first line at once;
    • the C library's cbrt, fmax and fmin, where Rust's own copies had taken their
      place.
  • Math builtins on musl, FreeBSD, NetBSD and Windows. jq builds each optional libm
    function the platform's C library has, so jq on FreeBSD has j0, lgamma_r and the
    rest. qj only had them on glibc and macOS. Elsewhere they said "not found at build
    time", and jn/yn weren't builtins at all. Each platform now gets exactly what its
    C library has:

    • glibc and macOS: all of them;
    • musl: all but gamma;
    • FreeBSD and NetBSD: all but exp10;
    • Windows: the Bessel functions, as jq.exe has.
  • string * number used twice the result's memory. jq folds constants while it
    compiles, so "abc" * 333333333 took 2 GB in qj (1 GB in jq), even in a branch that
    never runs, and a repeat near jq's limit could run out of memory where jq didn't. It
    now peaks where jq does.

Testing

  • Every simdjson kernel the CPU runs is checked against the others on every test run, so
    CI's x86 runners cover the x86 kernels (icelake, haswell, westmere, fallback), not only
    the one the CPU picks.
  • New fuzz target for jq's front end (lexer, parser, compiler; it found the
    string * number bug), and the simdjson fuzz targets can now fuzz each kernel.

Full changelog: v0.2.0...v0.2.1