Server v3.1.0

Adds terminology and external table dependencies to the SQL on FHIR operations, so that a query can join to value sets, concept maps and operator-configured tables, and updates the bundled core libraries to 10.0.0.

New features

Value set and concept map dependencies

A SQLQuery or SQLView can now declare a ValueSet or ConceptMap as a relatedArtifact dependency, in the same way that it declares a view. The server exposes the content under the dependency's label as a relation that the SQL joins to, following the SQL on FHIR Terminology in SQL page:

• A value set becomes a relation with the columns system, version, code, display and inactive.
• A concept map becomes a relation with the columns source_system, source_version, source_code, source_display, target_system, target_version, target_code, target_display and relationship.

Content is taken from a ValueSet or ConceptMap supplied in the context parameter, or failing that from the configured terminology server or local terminology store. Each dependency is resolved once per job, before any SQL runs. The size of each relation is capped by pathling.sqlQuery.valueSetMaxMembers and pathling.sqlQuery.conceptMapMaxMappings (both default 100,000). Reading a value set or concept map requires only the authority of the operation. The admin UI's "Provide SQL" tab can now bind a label to a value set or concept map, and a query with only terminology dependencies can be run.

Read the docs: SQL on FHIR, Terminology

External tables

Operators can configure external tables through pathling.sqlQuery.externalTables, giving each a canonical URL, a path and a format (Delta by default). A SQLQuery or SQLView can declare one as an ordinary relatedArtifact dependency and read it by its label in both $sql-run and $sql-export (#2750). Functions that return the storage path of a scanned file, such as input_file_name(), are now rejected so that the configured path is not disclosed.

Other

• The SQL queries and SQL views in the admin UI's query picker are ordered alphabetically by title (#2847).
• Views without a canonical URL are no longer listed in the source picker of the admin UI's SQL query form, since they cannot be referenced.
• The output format, row limit and CSV header controls have been removed from the admin UI's SQL query form, as the result card only shows the first 10 rows as a table.

Bug fixes

• Temporary views were namespaced with a caller-supplied request id, which the SQL validator then trusted as a request-scoped view (#2770). Views are now namespaced with a server-generated random id. Contributed by @developer-rpai in #2774.
• $sql-export matched a kick-off to an existing job without considering the inlined context, returning stale results (#2757).
• A CSV $sql-run that failed during evaluation returned a 200 with only the header row (#2742).
• $sql-run wrote DATE and TIMESTAMP values in the json and ndjson formats as locale strings rather than ISO 8601 (#2828).
• An unresolved SQL on FHIR dependency returned a 404 that did not name the parameter at fault (#2775).
• A subquery over a nested SQLView's own dependency was rejected as an unauthorised data source (#2759).
• The admin UI reported a 404 carrying an OperationOutcome only as "Resource not found", hiding the diagnostics, such as which query dependency could not be resolved.
• Writing Parquet or NDJSON to a directory that already held files discarded the write. Bulk export files are now named <type>.<partId>-<writeId>.<ext> (#2823).

Dependencies

• Pathling core libraries 9.9.0 to 10.0.0. This includes breaking changes to memberOf and getReferenceKey(), and a new local terminology store format that is migrated in place on first use; see the 10.0.0 release notes.
• Tomcat 10.1.55 to 10.1.60
• Jackson 2.22.1 to 2.22.3
• org.lz4:lz4-java 1.8.0 replaced with the at.yawk.lz4 fork 1.11.4

Full Changelog: server-v3.0.0...server-v3.1.0