2022.2.1

New features in this release mostly concern themselves with metrics, tracking and integrations:

• Matomo tracker or tag manager are now supported via MATOMO_URL, and MATOMO_CONTAINER and/or MATOMO_SITE_ID environment variables. (Needs more integration testing.)
• Metrics for bibliographic data accesses via GUI and API are now exported in Prometheus format under /metrics path.
• The service now allows site visitors to log in via Datatracker, showing the requisite widget if DATATRACKER_CLIENT_ID and DATATRACKER_CLIENT_SECRET environment variables are provided at deploy time. (#69)
  • OAuth2 flow is implemented, but needs further integration testing and clarification of the user info endpoint schema.
  • Callback URL is expected to be https://<hostname>/datatracker-auth/callback/
• Multiple additional API secrets can now be provisioned for management API and management GUI access via EXTRA_API_SECRETS environment variable. (#68)

Added containers

There are new services in the bundled Compose file configuration.

• A container that exports Celery-level metrics in Prometheus format by monitoring Redis backend.
• A pre-configured Prometheus instance monitoring three targets (web server, and two targets for async task runner).
• A Grafana instance provisioned with Prometheus as data source and a couple of dashboards for monitoring API and GUI bibliographic data access metrics.

Notable regressions

• We now explicitly run only one worker per container, both in case of Hypercorn serving web GUI and in case of Celery running async task worker. CPU_COUNT variable is no longer supported.

  If needed, web and celery containers themselves can be scaled horizontally. In such custom configuration it’s important to also ensure Prometheus is set up to discover scaled instances in order to import and aggregate bibliographic data access statistics from all of them. Bundled Compose configuration does not support that scenario at this time.

• Links to get bibliographic item data in Relaton JSON or XML are not shown to site visitors for the time being, as they were simply linking to API endpoints and API now requires header-based authentication—so those links wouldn’t work anymore.

  They can be reenabled in future if necessary, using a different mechanism that e.g. relies on Datatracker OIDC authentication.

Other changes

Various smaller fixes and improvements include:

• List of indexable datasets in OpenAPI spec for management endpoints is now generated dynamically and no longer erroneously includes doi
• Search query formats, changed some time back, are now documented in OpenAPI spec
• Search widget on landing page now features help explaining different query methods
• json_struct method has been added to web GUI search widget
• Version displayed in OpenAPI specs and built documentation now reflects current codebase version (such as 2022.2.1 for this release)
• and more.

2022.1.29_2

This release affects OpenAPI specifications:

• OpenAPI spec for xml2rfc-style paths was brought up to date. It no longer lists all the directories, but bibxml to bibxml9 are covered.
• Bibliographic item schema definitions in OpenAPI specification are now automatically generated from Pydantic model definitions (they can be explored in application/json response schema under retrieval endpoint).
• OpenAPI specs no longer refer to bibliographic items as “citations”.

2022.1.29_1

This release makes Datatracker’s bibxml API key required and validated on each request to bibliographic item data search and retrieval endpoints (#69), and lays foundation for further Datatracker integration (such as retrieving latest Internet Draft revisions).

It also removes accidental trailing dots or .xml extension when using web GUI’s simple search widget as a minor convenience measure.