Skip to content

v5.1.0

Choose a tag to compare

View all tags
@JoshuaSBrown JoshuaSBrown released this 01 Apr 02:31
· 2 commits to master since this release
v5.1.0
d7cd612
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Summary

This release introduces multi-engine schema support (JSON Schema + LinkML), a schema factory abstraction for the core server, full schema CLI support in the Python client, and correlation ID propagation through database maintenance paths. Infrastructure improvements include core unit tests in CI, consolidated Foxx test database configuration, and removal of dead code.

Images

docker pull savannah.ornl.gov/datafed/core:v5.1.0
docker pull savannah.ornl.gov/datafed/web:v5.1.0
docker pull savannah.ornl.gov/datafed/foxx:v5.1.0
docker pull savannah.ornl.gov/datafed/gcs:v5.1.0
docker pull savannah.ornl.gov/datafed/repo:v5.1.0

MINOR - Features & Enhancements

New functionality and improvements that maintain backward compatibility

[DAPS-1855] - feature: core - Add SchemaServiceFactory to decouple schema validation and storage logic (#1891)

  • Introduces ISchemaValidator, ISchemaStorage, and SchemaServiceFactory abstractions
  • SchemaHandler extracted from ClientWorker into dedicated handler class
  • Supports pluggable schema engines: json-schema (ArangoDB-backed) and linkml (external SchemaDBApi)

[DAPS-1912] - refactor: core - Register LinkML storage engine with schema handler (#1912)

[DAPS-1902] - feature: core, common - Get schema API client into compliance with API spec file (#1903)

  • SchemaAPIClient updated to match OpenAPI spec for external schema service communication

[DAPS-1893] - feature: common - Proto3 add fields for schema type, format, and metadata format (#1894)

  • SchemaCreateRequest gains type (field 6) and format (field 7)
  • SchemaData gains type (field 13) and format (field 14)
  • MetadataValidateRequest gains format (field 3)
  • Additive proto3 changes; existing clients remain wire-compatible

[DAPS-1896] - feature: foxx - Schema format and type added to schema create API (#1897)

  • Schema create and revise endpoints accept type and format fields
  • Schema ID responses now include version suffix (e.g. "my_schema:0" instead of "my_schema")

[DAPS-1830] - refactor: core, foxx - Refactored JSON schema integration (#1892, #1899, #1898)

  • Foxx routers use dynamic db_config.js for graph/database name instead of hardcoded "sdmsg"
  • New db_env.js provides shared environment configuration for arangosh scripts
  • JsonSchemaValidator and ExternalSchemaValidator implement ISchemaValidator interface
  • ArangoSchemaStorage and ExternalSchemaStorage implement ISchemaStorage interface

[DAPS-1663] - feature: core - LogContext and correlation ID propagated to dbGetRaw, dbMaintenance, metricThread, and TaskWorker (#1885)

  • IAuthenticationManager interface methods now accept LogContext parameter
  • DatabaseAPI::dbGetRaw forwards x-correlation-id header to Foxx via curl
  • purgeTransferRecords and uidByPubKey gain LogContext propagation

[DAPS-1857] - feature: python_client - Schema CLI commands and CommandLib API support (#1859, #1895)

  • New schema command group: view, create, revise, update, search, delete, export, list
  • CommandLib.API gains corresponding methods: schemaCreate, schemaRevise, schemaUpdate, schemaView, schemaSearch, schemaDelete, schemaExport, schemaList
  • Supports --type (json-schema | linkml) and --format (json | yaml) options
  • Handles protobuf def reserved keyword via setattr/getattr workaround

[DAPS-1919] - feature: python, web - Support LinkML schema type (#1921)

  • Web UI schema dialogs support schema type selection and format-aware editing
  • Python client passes schema type through create/revise workflows

[DAPS-1906-1] - feature: ci - Add core unit testing to CI pipeline (#1907)

  • New run-core-unit-job stage runs CTest from intermediate build image
  • Filters out integration and fixture tests via -LE "integration|fixture"

PATCH - Bug Fixes, Technical Debt & Improvements

Bug fixes, refactoring, documentation, and other improvements

[DAPS-1914] - fix: web - Bug in schema ID name/version mismatch (#1915)

  • Schema picker now strips version suffix before re-appending, preventing double-suffixed IDs

[DAPS-1890] - fix: test - Mock core server build (#1890)

  • Mock AuthMap, AuthenticationManager, ClientWorker, and Condition updated for LogContext interface changes
  • Mock PublicKeyTypes.hpp aligned with core server types

[DAPS-1913] - refactor: foxx - CI test scripts consolidate database name and default to separate database for tests (#1917)

  • Test scripts read DATAFED_DATABASE_NAME from environment, defaulting to sdms_test
  • New DATAFED_ALLOW_TESTING_PROD_DATABASE guard prevents accidental production database use in tests
  • ALLOW_PRODUCTION_DB environment variable required to run against production database name

[DAPS-1916] - tests: core - Add integration tests for schema client handler to CMake (#1918)

  • Integration test suites for SchemaHandlerJson, SchemaHandlerLinkML, SchemaAPIClient, and SchemaServiceFactory
  • CTest fixtures manage Docker containers (ArangoDB, Prism mock) for integration tests

[DAPS-1887] - refactor: repo - Remove dead FUSE filesystem code (#1888)

  • Deleted repository/filesys/CMakeLists.txt and repository/filesys/fusemain.cpp

[DAPS-1910] - upgrade: web - Playwright 1.51.1 version (#1911)

[DAPS-1830] - refactor: common - TraceException::getErrorCode() marked const

Migration Notes

Foxx schema ID format change: Schema API responses now return composite IDs with version suffix (e.g. "my_schema:0" instead of "my_schema"). All internal consumers (core server, web UI, Python client) are updated in this release. External tools parsing Foxx schema responses directly will need to account for the new format.

Test database safety: Foxx test infrastructure now defaults to sdms_test database. CI pipelines targeting the production sdms database must set ALLOW_PRODUCTION_DB=true and DATAFED_DATABASE_NAME=sdms.

Contributors

Full Changelog: v5.0.0...v5.1.0

Assets 2
Loading