Skip to content

v0.4.0

Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 01 Nov 18:58
· 16 commits to dev since this release
Immutable release. Only release title and notes can be modified.
v0.4.0
This tag was signed with the committer’s verified signature.
uniqueg Alex Kanitz
SSH Key Fingerprint: 9NjZS21iWrTwyzKfdaTSze0mbHhsu+9cNOFOXtblYdk
Verified
Learn about vigilant mode.
ed1c90c
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

proTES now processes tasks with FOCA's plugin engine instead of its own
middlewares, on FOCA 0.19: plugins check or change tasks, choose the TES
instances and their order, and record the provenance of each task, which
users read through the API. The image ships the routing plugins of
foca-plugins (tes.routing.distance and generic.routing.random), so that,
as in 0.3.0, tasks go to the TES instance closest to their input data, with
random order as the fallback. The release is built from locked, hash-checked
dependencies, and removes forwarded_to from task logs. Several changes are
breaking: read the upgrade
steps
first. See the documentation for
details.

Breaking changes

  • Plugins replace the middlewares: new tasks pass through a pipeline of FOCA
    plugins (plugins.pipeline), and custom.middlewares is refused at
    startup. The middleware management API and its collection are removed, and
    custom middlewares must be ported to plugins. Tasks that no TES instance
    accepts fail with 503 (was 500), requests that already passed proTES
    with 508 Loop Detected (if server.public_url or plugins.gateway_id
    is set), and tasks with tags under gateway. other than
    those of the clients' choices with 400 (see
    Plugins)
    (#333, #346; #315, #320; #316, #317)
  • proTES runs on FOCA 0.19 (was 0.17): custom configs must apply the config
    changes of FOCA 0.18 and 0.19, and the image and the Helm chart of this
    release must be deployed together (see FOCA 0.18 and
    0.19    )
    (#331, #338; #359, #360)
  • TES instances are listed in custom.tes.backends (in the Helm chart,
    tes.backends) instead of custom.tes.service_list and
    custom.tes.discovery (see TES instance
    list    )
    (#332, #340)
  • API: logs[].metadata.forwarded_to is removed without a deprecation
    period, also from tasks stored by earlier versions; clients read the TES
    instance and the remote identifier from the gateway tags gateway.backend
    and gateway.backend_task_id. custom.storeLogs is refused at startup
    (see Forwards in task
    logs    )
    (#324, #374)
  • Access control: custom lists of security.access_control.policies must
    add the rules users-list-plugins, users-get-plugins,
    owners-get-provenance and admins-get-provenance, or users get 403
    (#322, #352; #218, #357)
  • Database: proTES uses the new collections plugin_builds, pipelines,
    failed_runs and plugin_store, and creates indexes on startup: its
    database user needs the right createIndex (see Task
    provenance    )
    (#334, #349)

Features

  • The image ships the routing plugins tes.routing.distance and
    generic.routing.random of foca-plugins, pinned and hash-checked in
    uv.lock, and the default pipeline is distance routing with random order
    as the fallback again, as in 0.3.0. With the default settings, the hosts of
    inputs are looked up at DB-IP and Nominatim; set provider: offline and a
    GeoIP database, or FOCA's foca.routing.ordered, to avoid third parties
    (see
    Routing)
    (#379, #381)
  • Task provenance: proTES stores the provenance of each task's plugin
    pipeline run and serves it with GET /tasks/{id}/provenance to the task's
    owner and to admins; new tasks have FOCA's gateway tags (gateway.*)
    that summarize it (#334, #349; #322, #352)
  • Clients may choose the plugins and the TES instances of a task with the
    tags gateway.pipeline (if the admin enables it) and gateway.backends,
    and list what they may choose with GET /plugins; each choice applies to
    its task only (#218, #357)
  • Helm chart: values for the plugins (plugins) and for the settings of the
    identity provider's issuer (#335, #348; #337, #341)

Security

  • Requests to TES instances discovered in service registries go through
    FOCA's guarded HTTP client, which checks the address of each connection,
    so that a host whose addresses change after discovery can't lead requests
    with proTES's token to private addresses (#339, #347)
  • .env files, PEM, key and PKCS#12 files are excluded from the Docker
    build context everywhere, also below pro_tes/ (#361, #367)
  • The dev-image push job, which holds the Docker Hub credentials, now derives the version with build requirements installed from uv.lock with hash checks, exported by a digest-pinned uv image (#376, #378)

Fixes

  • The forwarding of a task, with its state request, takes at most 55 seconds
    with the default settings (plugins.defaults.forward_timeout: 20, was
    FOCA's 60), within the 60 seconds of common proxies' read timeouts; other
    work, e.g., registry reads, token requests and database writes, comes on
    top. Tokens of service registries are no longer logged (#364, #369)
  • Middlewares kept the order of the TES instances, and the changes of failed
    middlewares were undone, before the plugins replaced the middlewares
    (#314, #319; #313, #318)

Documentation

  • The upgrade path from 0.3.0 (#362, #363, #368)
  • A recipe for images derived from the image of proTES with other plugins,
    pinned by hash, tested in CI with a test plugin (#323, #355)
  • The org-wide community files replace those of the repository, including
    the contributing guide (#325, #326; #370, #372)
  • The README logo, the Zenodo DOI and the docs badge (#350, #351; #307,
    #309, #311)

Maintenance

  • CI's lint, test, chart and docs jobs install the hash-checked versions from a new uv.lock (uv sync --locked), and Dependabot updates pyproject.toml and the lock together (#375, #377)
  • CI passes in forks, uses the org's shared actions for the CI gate and for
    scanning images, checks pull request titles under the gate, and checks the
    chart before signing anything; workflow and job names are aligned and
    prepared for the repository name protes (#327, #328; #330, #342; #329,
    #343; #371, #373; #353, #354; #310, #312; #356, #358)
  • Sourcery skips rule-test fixtures (#306)
  • Dependabot updates of the pre-commit hooks and of the GitHub Actions (#344,
    #345)

Commits

Full Changelog: v0.3.0...v0.4.0

Contributors

uniqueg and dependabot
Assets 5
Loading