Skip to content

v0.7.1

Choose a tag to compare

@markhammond markhammond released this 11 Nov 15:51
· 6 commits to main since this release
8f153f4

ChalkQL v0.7.1

A security release. Upgrade if you use entitlements.

  • An entitled table is now a barrier. A condition a statement puts round an entitled table is evaluated only on the rows the policy admits. Before, it could be evaluated on rows the policy was about to exclude, so an error could quote such a row's value, and whether a statement failed could reveal whether such a row existed.
  • A CASE branch runs only for the rows that take it. A guard such as CASE WHEN x <> 0 THEN y / x END now guards.
  • Engines can share a planner client. Disposing one engine no longer breaks another on the same client.
  • The packages carry their licence. Each now includes the Elastic License 2.0 text; earlier packages named an empty licence file.

There is no change to the API, the protocol or the IR.

Targets .NET 10. A JDK 21 or newer is needed only to run the planner locally. Assemblies and namespaces retain the Chalk.* names.

NuGet packages:

Package Licence
ChalkQL NuGet Elastic License 2.0 engine, planner client, catalogue, execution, entitlements and standing views
ChalkQL.Sources NuGet Elastic License 2.0 POCO, Akade.IndexedSet, ADO.NET, DuckDB and source conformance implementations

Security: an entitled table is a barrier

Some hosts put a condition round an entitled statement, as an alert rule does:

SELECT * FROM (<entitled statement>) AS t WHERE <condition>

The planner could move that condition, or one in the statement's own WHERE, into the entitled table's row filter. It was then evaluated together with the policy's row predicate, over every row the table held. The rows returned and the masks applied were always right, but two things leaked:

  • A value in an error. CAST(Callsign AS DOUBLE) > 1 failed quoting the first row the table held, for a principal not shown that row.
  • Whether a row exists. CAST(CASE WHEN Callsign = 'BANYAN 41' THEN 'x' ELSE '0' END AS INTEGER) = 0 failed exactly when a row with that callsign existed, whether or not the principal could see it. A division by such a CASE did the same.

Now each entitled table is fenced, as a PostgreSQL security_barrier view is. A condition crosses into it only when it cannot raise an error or fail depending on the row:

  • Cross the fence: comparisons; AND, OR and NOT; the IS tests; IN lists; LIKE; lossless casts; context lists; COALESCE, CASE, UPPER, LOWER and TRIM over those; and anything that reads no column, such as @now - INTERVAL '10' MINUTE.
  • Stay above it: arithmetic, a cast that can fail, a division and other functions. These run here, over the rows the policy admitted.

So an error can quote only a row the principal is shown, and whether a statement fails says nothing about a row they are not.

What still pushes down to the source and its indexes, together with the table's row predicate:

  • conditions that cannot raise;
  • joins on such conditions;
  • aggregates;
  • an ORDER BY on columns;
  • bounds.

What no longer reaches the source for an entitled table: a condition that can raise, or a sort on such an expression. These now run locally over the admitted rows. For a large table in a database, check the cost of statements that filter by such an expression.

A CASE branch runs only for the rows that take it

The vectorised executor evaluated every branch of a CASE over every row and chose afterwards. So a branch could raise for a row its condition did not choose:

  • CASE WHEN x <> 0 THEN y / x END divided by zero;
  • a cast guarded by a condition failed on the very rows it guarded against.

Each branch is now evaluated only for its own rows, as SQL means and as the reference executor already did. A branch still shares work with the rest of the statement: in CASE WHEN UPPER(name) = 'X' THEN UPPER(name) END, UPPER(name) is computed once. A CASE that returns a composite value is unchanged.

Engines can share a planner client

Disposing an engine disposed the planner client it was given, whatever else used it. A second engine on the same client then failed at its next refresh that replaced a table, and no engine could be made on the client. That bites a host that builds a new engine for a changed rule set and disposes the old one while both exist.

  • The last engine disposes it: the client is now disposed with the last engine that uses it. An engine with a client of its own disposes it, as before. Disposing one of several engines leaves the others planning.
  • A retired client is refused: an engine made on a client already disposed that way is refused when it is made, naming the client.

Packages

  • The licence is included: both packages now carry the repository's licence text, the Elastic License 2.0. Every package since 0.5.0 named a licence file that was empty.
  • An empty licence cannot ship: packing now refuses a missing or empty licence.

Changed

  • The barrier: a condition that can raise, or a sort on such an expression, is evaluated above an entitled table's barrier, not pushed to its source or index.
  • CASE: a branch is evaluated only for the rows that take it.
  • Planner client: it is disposed with the last engine that uses it.

Compatibility

  • No change to the API surface, the protocol or the IR. A 0.7.0 planner serves a 0.7.1 client, and a 0.7.1 planner serves a 0.7.0 client.
  • The barrier is the planner's: a 0.7.1 client planning against a 0.7.0 planner is not protected. The planner ships inside the ChalkQL package, so upgrading the package upgrades both.
  • Planner clients: a host that disposed an engine and then made another on the same client now gets an error when the second engine is made, rather than later.
  • Statements whose result changes: a statement that relied on a CASE branch raising for rows it did not take, or on an error naming a row the principal was not shown, now behaves as SQL defines.

Full Changelog: v0.7.0...v0.7.1

Licence and attribution

ChalkQL is source available under the Elastic License 2.0 (ELv2).

Apache Calcite and its JVM dependencies are bundled into the planner artefact distributed with ChalkQL. Their licences and attribution are recorded in THIRD-PARTY-NOTICES.txt.