Skip to content

schema-calc v0.5.0

Choose a tag to compare

@fMindstormer fMindstormer released this 10 Nov 23:57
· 5 commits to main since this release

A product by a listed rate. A field whose values a contract lists can now multiply another field, abs() drops a sign, and the lines of a list can leave fields out and join their own relations with and and or. With these, BR-CO-17 of EN 16931, each VAT category's tax as its taxable amount times its rate, compiles into plain JSON Schema beside the seven document-total rules of 0.4.0, and the schemas give the official Schematron's verdict on every one of 804 test invoices, rule by rule. Try it in any browser; nothing is sent to a server.

What's new

  • Products by a field with listed values. "values": {"rate": ["0", "7", "19"]} lists the values a field may take, on the record or in a list's declaration for its items' fields. The field then accepts only those, each in its one valid spelling, and may multiply another field: 100 * tax <= rate * base. The compiler builds the relation once for each value, with the value as a coefficient, and an anyOf chooses among them, so the schema is about the sum of their sizes: 68 kB for that relation at four integer digits. One such field per relation, multiplying a field rather than a sum, with up to 32 values. Products of two free fields remain out of reach.
  • abs(x), an amount without its sign, for one field or one line's field. It leaves out the sign tests that a signed amount makes at every digit, so it costs less than x: abs(a) <= b is 17 kB, a <= b 30 kB.
  • Coefficients ending in zeros cost nothing. A coefficient c × 10ᵏ now reads its field k columns later with coefficient c. total = k * price + shipping compiles to 18, 16 and 14 kB with k = 10, 100 and 1,000, against 80, 401 and 751 kB in 0.4.0.
  • Optional fields on a line. "lines": {"maxItems": 10, "fields": ["net", "tax", "rate"], "optional": ["rate"]} lets each line leave rate out; in arithmetic an absent field counts as zero, as on the record.
  • and, or and has on each line. A constraint on each line can join relations and presence tests on that line: has lines.rate and lines.tax <= lines.net or no lines.rate and lines.tax = 0.

EN 16931: BR-CO-17

The EN 16931 page now compiles BR-CO-10 to BR-CO-17. In both Schematrons, BR-CO-17 leaves a tolerance of one currency unit, not one cent: the tax amount, without its sign, must come within a unit of the taxable amount, without its sign, times the rate, rounded to the cent, strictly in UBL and inclusively in CII. Rounding moves the bounds by only half a cent, so the rule is linear for each rate: −99.5 ≤ rate × |taxable| − 100 × |tax| < 99.5 in UBL, and the same with 100.5 in CII. A category without a rate, or at zero, must have a tax that rounds to zero. The contracts list the ten rates of the official examples, 0, 6, 10, 12, 15, 19, 21, 22, 25 and 27; a category at any other rate fails the schema, and a contract for real invoices lists the rates its sellers charge.

On the 34 official examples and 770 copies with an amount changed or an optional element removed, all 804 verdicts agree, rule by rule, in Ajv and in Python jsonschema with its default options. Five new changes to each invoice with a VAT category test BR-CO-17 alone: the first category's tax moved by ±1.00 or ±0.50, with the totals moved with it, and its rate removed. A whole unit is where the syntaxes part: UBL rejects all 38 such copies, CII 5 of 26. Each schema is 10.5 MB, most of it still for adding up 20 signed lines; BR-CO-17 alone takes 874 kB, about 95 kB per nonzero rate. Ajv prepares one in under 20 seconds and checks an invoice in under a millisecond; Python jsonschema takes about a second per invoice.

Check it without Schema Calc

Python 3 and jsonschema only, in an empty directory:

python3 -m pip install jsonschema==4.25.1
base=https://github.com/fMindstormer/schema-calc/releases/download/v0.5.0
for f in validate.py invoice.schema.json invoice.valid.json invoice.invalid.json; do curl -fLO "$base/$f"; done
python3 validate.py invoice.schema.json invoice.valid.json invoice.invalid.json
ACCEPT  invoice.valid.json
REJECT  invoice.invalid.json

The line-items schema and records work the same way with lines.schema.json, lines.valid.json and lines.invalid.json.

Validator compatibility

The flat test corpus grew from 43,319 to 45,451 records across 92 schemas, adding abs(), coefficients ending in zeros, optional fields of a line with and, or and has on each line, and products by fields with listed values, including fractional and negative values, against an independent oracle. All of them pass Ajv and Python jsonschema, and twelve validators in five languages ran them (table). None gave a wrong answer. Nine load every schema. @cfworker/json-schema needs --defs, as before: it refuses the 80 flat schemas that nest one embedded resource in another. @exodus/schemasafe times out loading the 4 largest, from 364 to 410 kB, and with --defs now loads every schema; the coefficient schemas that timed out in 0.4.0 are now 31 and 37 kB. jsonschema-rs runs out of memory on 53 (51 with --defs), including all five shipped ones.

For library users

  • Compiled output changed in two places, with the same verdicts. Presence tests now state type: "object", which Ajv's strict mode asks for: has paid is {type: "object", required: ["paid"], properties: {paid: true}}. And a coefficient ending in zeros compiles to the smaller schema above. The shipped example schemas are byte-for-byte the same as in 0.4.0.
  • A relation with a product comes from parseConstraint with by, the field with listed values, and products, the terms it multiplies; instantiate(relation, units) returns the plain relation for one value, given in the smallest unit (1900 for 19.00 at scale two). Such a relation counts in units of 10^(-2 × scale), so its other coefficients and its constant are scaled by 10^scale.
  • A term may carry a third element, true, when it is read with abs(): ["a", 1, true].
  • normalizeSpec returns values when a field lists them, and a list's declaration returns its own optional and values. formatAmount(units, scale) writes an amount in its one valid spelling.
  • The tally encoding refuses listed values and abs(), with an error that names the flat encoding.

Files

invoice.schema.json is the compiled invoice contract, unchanged from 0.4.0; invoice.valid.json and invoice.invalid.json are the correct record and the one a cent off. validate.py is the standalone validator: it needs only jsonschema and never imports the compiler. schema-calc-0.5.0.zip is the source, with a MANIFEST.sha256 inside; schema-calc-0.5.0.zip.sha256 is its checksum and SHA256SUMS covers every file.

Scope

Sums and differences of decimal strings, signed if declared, their whole-number multiples, abs(), products by a field with listed values, and sums over lists, related by =, <=, >=, < or > and joined by and and or with presence tests, on the record or on each line: up to 16 fields, 4 lists of 20 items, 8 constraints, 12 digits and 32 listed values per field. No other products of fields, division, !=, parentheses, not, indexing into a list or other nested objects. The tally encoding takes no lists, signed amounts, optional fields, listed values, abs() or joined relations. A research prototype; it is not published to npm.