Skip to content

Server v3.0.0

Choose a tag to compare

@johngrimes johngrimes released this 02 Sep 01:32
· 13 commits to main since this release
server-v3.0.0
3dccd04

Replaces the ViewDefinition and SQL query operations with the two operations defined by the SQL on FHIR specification, $sql-run and $sql-export, adds a server-backed $jobs list, and hardens token validation, schema evolution and warehouse I/O.

Breaking changes

This is a major version release. Operators upgrading from server v2.0.x should review the following before deploying:

  • $viewdefinition-run, $viewdefinition-export and $sqlquery-run have been removed. They are replaced by $sql-run (synchronous) and $sql-export (asynchronous), which follow the SQL on FHIR specification and accept both SQL and ViewDefinition subjects. Conformance now advertises the specification canonicals. The configuration flags pathling.operations.viewDefinitionRunEnabled, pathling.operations.viewDefinitionInstanceRunEnabled and pathling.operations.viewDefinitionExportEnabled are replaced by pathling.operations.sqlRunEnabled and pathling.operations.sqlExportEnabled, and the authorities pathling:view-run and pathling:view-export by pathling:sql-run and pathling:sql-export.
  • The read interaction has its own operation authority. Reading a resource by id now requires the pathling:read-resource operation authority in addition to read access for the type being read. A token holding only pathling:read no longer authorises the read interaction.
  • Resolving a stored ViewDefinition or SQL query requires read access to that resource type. With authorisation enabled, reading a ViewDefinition from storage requires read access to ViewDefinition, and reading a SQLView Library requires read access to Library. This is enforced at every storage-read seam, layered on top of the existing per-projected-resource checks. A resource supplied inline in the request body is exempt.
  • Symmetric token signing algorithms are rejected at startup. Tokens are verified against a public JWKS, so only asymmetric algorithm names are permitted in the new pathling.auth.tokenSigningAlgorithms setting. Any other value fails validation at startup rather than at request time.
  • The SQL query row cap and wall-clock timeout have been removed. pathling.sqlQuery.maxRows and pathling.sqlQuery.timeoutSeconds no longer exist (#2681). Queries are no longer bounded by the server; bound them at the request or infrastructure level where that is required.

New features

$sql-run and $sql-export

Two operations replace the four they supersede. $sql-run executes a query synchronously and returns the result; $sql-export runs the same work asynchronously against the bulk-data async pattern and writes the result to files. Both accept either inline SQL or a ViewDefinition as the subject, in line with the SQL on FHIR specification. The output formats depend on the kind of subject: NDJSON, CSV and JSON are available for both, and Parquet and FHIR for SQL subjects. The set of supported export formats is now declared in the CapabilityStatement.

Read the docs: SQL run, SQL export

SQLView composition

A SQL query can now reference a stored SQLView Library by canonical URL, and those views can themselves reference further views. The server resolves the dependency graph, detects cycles and diamonds, and enforces a configurable depth limit through pathling.sqlQuery.maxDependencyDepth. A SQLView can also be run or exported directly as the subject of an operation.

$jobs

A system-level operation that lists asynchronous jobs, with derived status and a registry snapshot, gated by the new pathling:jobs authority. Job ownership is enforced on the delete path, so a job can only be cancelled by the principal that created it. The admin UI has a corresponding jobs page with cancellation.

Read the docs: Jobs

Schema drift handling

pathling.storage.schemaAutoMerge now migrates every drifted table at server startup, not only on update, and refreshes the in-memory dataset so subsequent reads observe the new schema without a restart. A write into a table that is wider than the incoming data succeeds, and where a stored table and the running encoders cannot be reconciled at all the request fails with an error naming the resource type, the direction of the difference, the field paths and the remedy.

Token signing algorithm control

pathling.auth.tokenSigningAlgorithms pins the accepted JWS algorithms. When left empty, the accepted algorithms are derived from the keys published in the issuer's JWKS at verification time, so key rotation at the identity provider takes effect without a restart (#2708).

Other

  • SQL queries accept window functions, named windows and frame boundaries (#2649, #2650), and DESCRIBE of a registered view for schema introspection (#2651).
  • The spec-recommended json format is accepted on the SQL operations (#2686).
  • VOID columns, which have no Parquet representation, are rejected up front on both the synchronous and asynchronous Parquet paths rather than failing mid-write.
  • spark-hadoop-cloud is bundled, enabling the S3A magic committer for S3 warehouses.
  • The Helm chart accepts additional trusted CA certificates, and grants the driver the RBAC permissions needed to manage executor scratch PVCs.
  • Ping and pull download concurrency and socket timeout are configurable through pathling.import.pnp.maxConcurrentDownloads and pathling.import.pnp.downloadSocketTimeout.
  • The cache backend first-byte timeout is configurable, and the cache no longer truncates or stalls large bulk-data downloads.

Bug fixes

  • Unauthenticated requests returned an opaque 500 instead of a 401 challenge (#2667).
  • Asynchronous exports failed with Wrong FS on warehouses that are not on the default filesystem (#2666).
  • Narrowing encoding.openTypes against an existing warehouse made every write for that resource type fail with an opaque 500 (#2697).
  • Deleting a running job raced the Spark work it was cancelling (#2687).
  • Spark analysis errors on $sql-run and $sql-export were reported as an opaque 500 rather than a 422 naming the subject (#2723).
  • Column names matching a relatedArtifact.label were rewritten into the generated view identifier (#2730).
  • Columns backed by java.time values (TIMESTAMP_NTZ, day-time and year-month intervals) failed with a 500 when serialised to ndjson or json (#2731).
  • The admin UI stayed on a stale cached bundle after the server was upgraded (#2677).
  • The session expiry dialog's login button did nothing, and later expiries were silent (#2676).
  • The jobs page showed a spurious session expiry dialog when logged out (#2678).
  • The SQL on FHIR view card showed the error prefix twice (#2688).
  • Job cards reported the same failure both persistently and as a notification (#2689).

Dependencies

  • Pathling core libraries 9.7.1 to 9.9.0
  • Spring Boot 3.5.14 to 3.5.16
  • Jackson 2.18.6 to 2.22.1
  • Netty 4.1.133.Final to 4.1.137.Final
  • fhir-bulk-java 1.0.4 to 1.1.0
  • react-router 7.10.1 to 7.18.0 in the admin UI, addressing XSS and DoS advisories

Full Changelog: server-v2.0.1...server-v3.0.0