Skip to content

v0.5.0 — a generator a provider can point at their own API

Choose a tag to compare

@christosgkoros christosgkoros released this 13 Sep 11:29
· 15 commits to main since this release
28567bf

A tooling release. No change to the grammar or to the semantics of evaluation:
query-language-schema.json is byte-identical to 0.4.0 apart from its root description, and its
$id still names v0.4.0, because the $id version tracks the grammar and the grammar did not
move. Consumers pinning that $id have nothing to do.

What did move is the generator, which is now the thing an API provider uses when implementing
search: point it at the resource schema, state the slice of the language you can actually serve,
and get back a filter schema that permits exactly that slice plus a capability document that
describes it honestly. It also travels with the package for the first time, as a bin named
jql-generate, rather than being a file inside a repository nobody installs.

SPEC.md gains two clarifications in service of that, both about what an
implementation may claim rather than about what a filter means:
§2.1 says out loud that accepting part of a profile is permitted and
advertising it is not, and §2.2 documents the capability
document's top-level members. No filter valid under 0.4.0 becomes invalid.

Added

  • The generator selects capabilities, not just profiles
    (#11). Profiles are the unit
    a server advertises, but three shapes do not fit inside one: a backend with LIKE and no
    POSITION supports $like and not $contains; a key-value store cannot implement $exists at
    all; a provider compiling to a flat conjunctive index wants one AND level and no shorthand. Six
    new knobs, each available as a flag and as a JS API option — --operators, --drop-operators,
    --no-shorthand, --max-filter-depth, --limits, and --config to hold the combination. What
    is declined is absent from the generated schema, so a client learns it from validation rather
    than from an unsupported-operator at runtime. Defaults are unchanged: with none of them given
    the output is byte-identical to before.
  • --config <file>, and examples/pet.jql.config.json. The capability selection is a
    decision about the endpoint, not a shell invocation, so it goes in a JSON file checked in beside
    the resource schema and regenerated from. Its keys are the JS API's option names plus resource,
    out and capabilities; relative paths in it resolve against its own directory, an explicit
    flag beats it, and an unrecognised key is refused rather than ignored — a misspelled key is a
    capability that silently did not apply. npm run generate:example now runs through one.
  • --max-filter-depth <n> caps how deep $and/$or/$nor/$not may nest: 1 is a flat
    filter offering no logical operators at all, 2 permits one level of them. JSON Schema
    cannot count how deep an instance already is, so the filter is emitted as a chain of levels:
    level i offers the logical operators over level i+1 and the last level does not offer them
    at all. Every level shares the operand $defs, so the cost is n copies of a map of $refs.
    Field-level $not is bounded to a single application by the same flag — under Kleene logic
    ¬¬X ≡ X even for UNKNOWN, so a negated negation says nothing the plain constraint does not.
  • --limits <json|@file> puts the SPEC §7 numbers a provider actually enforces into the
    capability document. They were emitted unconditionally, so every document generated from the CLI
    claimed maxDepth: 10, maxClauses: 100, maxSetLength: 1000 whether or not that was true. Where
    --max-filter-depth is given and maxDepth is not, the enforced bound is published.
  • The generator is part of the package. tools/ was not in package.json files and there
    was no bin entry, so the tool the README points readers at could not travel with the package
    at all. It is now a bin named jql-generate, with json-query-language/generate exporting
    generateFilterSchema for programmatic use. This repository still publishes no artifacts
    (RELEASING.md), so the command is reachable from a clone or a git install and
    not from npmjs; what changed is that it is ready to be, and npm pack now contains it.
  • SPEC.md §2.2 documents the capability document's
    top-level members
    queryLanguage, profiles, fields, limits and filterSchema — in a
    table beside the existing per-field one. limits appeared in the example and in no table, and
    filterSchema, itemValues and nullable were emitted by the generator and described nowhere.
    No normative change to what the members mean.
  • SPEC.md §2.1 says what a partial profile may and may not do. The
    rule was already there — a profile other than core is implemented in full or not at all — but
    it read as a prohibition on the implementation rather than on the advertisement. An endpoint
    accepting part of a profile is not prohibited from existing; it states what it accepts per path
    and omits the incomplete profile from profiles.
  • examples/mcp-server/ — a runnable MCP server whose one tool, search_pets, takes a
    filter as its filter argument and nothing else. The tool's inputSchema is
    examples/pet.filter.json inlined verbatim; validation is ajv against that same file, and
    execution is the SQL compiler from experiments/filter-to-sql over an in-memory SQLite table,
    so the queries are real. node examples/mcp-server/demo.mjs drives it over stdio and prints a
    transcript: two filters that answer, one that shows the $unknownAs difference (4 matches
    against 7), and the three valid-but-wrong filters from README §Exposing search to an agent
    being rejected with a pointer at the clause. npm run example:mcp and
    npm run example:mcp:demo are the entry points.
  • The example is also the first place the $id-when-inlining hazard is written down: nested
    under properties.filter, a bundled schema's self-references resolve against its own $id, so
    removing the $id breaks it — ajv fails to compile it at all.

Changed

  • The capability document's profiles reports coverage rather than the request. It echoed
    whatever --profiles was given; it now lists only the profiles the final operator set covers in
    full, because SPEC.md §2.1 makes a partial profile one an
    implementation may not advertise. --drop-operators '$contains' therefore costs the strings
    claim, and the per-field operators lists carry what is on offer instead — with a warning on
    stderr naming the operator responsible. Declining a core operator drops core too, and warns
    that the result is not a conforming implementation. Nothing changes for a selection that is
    whole profiles, which is every invocation before this release.
  • Positioned as one JSON-Schema-described query language with two integration points, rather
    than as an agent interface. An earlier revision in this same unreleased window led with the MCP
    tool definition and moved §Exposing search to an agent ahead of the OpenAPI and generator
    sections; that ordering is reverted and the "search interface for agents" framing is gone from
    the README, the package.json description and the repository description. The agent use case
    keeps its section and its runnable server — it is one of the two things the schema is for, not
    the thing the document opens with.
  • The schema's root description likewise leads with the shared-grammar framing again, and
    mentions inlining as a tool's input schema second. Non-normative prose; no validator behaviour
    changes.
  • The error format is no longer mandated. SPEC.md §8 required
    RFC 9457 Problem Details with media type
    application/problem+json. It now requires only that a rejected filter be answered with
    400 Bad Request and that the response say which of the five conditions applies —
    malformed-query, unknown-field, unsupported-operator, invalid-operand,
    query-too-complex — because that is what a client branches on. The envelope is the API's own:
    an API with an established error format should express these conditions in it rather than carry
    a second format for one endpoint. RFC 9457 remains the RECOMMENDED default where there is none,
    and the type URIs, the pointer member and the recovery members (queryableFields,
    accepted) are unchanged as its encoding. This relaxes a requirement, so nothing that
    conformed before stops conforming.

Fixed

  • --max-depth accepted a value that was not a number. It was coerced with Number() and
    never checked, so --max-depth deep became NaN and silently stopped the walk at the first
    nested object. It and --max-filter-depth are both validated now.
  • An operator whose dependency was dropped is dropped with it. $flags carries
    dependentRequired: ["$regex"] out of the grammar, so --drop-operators '$regex' would have
    left $flags in properties with a rule naming a member additionalProperties: false forbids
    — present in the schema and impossible to use. The closure is read off
    $defs/ConstraintObject, so a dependency added later is handled by construction.
  • Generated filter schemas were not a narrowing (#8).
    tools/generate-filter-schema.mjs carried the published constraint object's
    dependentRequired rule but not its dependentSchemas one, so {"microchip": {"$unknownAs": false}} — a modifier with nothing to modify — passed a generated schema while
    query-language-schema.json rejected it. A server following the documented path (generated
    schema as the tool's inputSchema, published semantics behind it) then had to evaluate a filter
    with no predicate in it; the SQL compiler in experiments/filter-to-sql emitted
    coalesce((), FALSE) and the database answered with a syntax error. The generator now reads
    both dependency keywords off $defs/ConstraintObject instead of restating either, so a rule
    added there reaches generated schemas with the version that introduces it, and
    examples/pet.filter.json is regenerated: 12 of its 18 constraint objects gain the rule — the
    ones offering $unknownAs, which is every field that can be absent or null. The $comment
    justifying the rule is deliberately not copied along with it: a validator never reads it, and
    one copy per field is charged by the token to whoever inlines the schema in a tool definition.
    No change to the grammar — this is the generator agreeing with it.
  • The narrowing property is now tested as a property. tests/generator.test.mjs asserted it
    over a hand-written list of fifteen filters, which can only re-check the leaks someone already
    thought of — the keyword above was dropped for as long as the list existed. It now samples
    filters out of each generated schema's own vocabulary (tests/fuzz.mjs, seeded, deterministic)
    and asserts that every one the generated schema accepts is valid JQL, over three generated
    schemas; the run is checked for not being vacuous, in that it must accept a fraction of its
    samples and must reach every operator the schema offers. A second test pins what the generator
    does with each instance-constraining keyword of $defs/ConstraintObject, so adding one there
    fails the suite until it is handled.
  • experiments/filter-to-sql rejects a constraint object with no predicate in it rather than
    emitting an empty expression — malformed-query, at the pointer of the offending clause. Both
    schemas already reject these, so this only matters for a compiler reached another way, but the
    failure it replaces was a 500 from the database.
  • The prose in README §Errors, COMPARISON.md §4, the
    OpenAPI examples and experiments/filter-to-sql follows: they now describe Problem Details as
    the recommended shape rather than the required one, and name the failing condition where they
    previously said "problem". The examples still model RFC 9457, since it is still the default a
    greenfield API should pick.