Repository navigation
TESK API 2.0.0 is a major release because of the breaking changes listed below, which deployments and clients of 1.2.0 must check. It brings TES 1.1 to TESK: its task fields, including ignore_error, its list filters and task states, and output file logs. It also brings optional access for tokens of allow-listed service clients (e.g., a TES gateway that calls TESK with its client-credentials token), opt-in CORS for browser clients of configured origins, and experimental support for private executor images, and it runs on Java 21 and Spring Boot 4 instead of the end-of-life Spring Boot 2.7, serves its API description as OpenAPI 3.1 at /v3/api-docs, and validates tokens with the resource server of Spring Security instead of the end-of-life Spring Security OAuth. The new options are off by default, but deployments may have to check their JVM options and the order in which they upgrade TESK API and the taskmaster, and any properties of Spring Boot itself that they set, and clients may notice changed answers, e.g., CANCELING before CANCELED, 400 Bad Request for wildcard outputs without path_prefix, pages of at most 256 tasks from GET /tasks without page_size, 401 Unauthorized for tokens sent in the URL, and the moved API description: read the upgrade notes before you upgrade. ignore_error and output file logs need a taskmaster and filer of tesk-core 0.12.0 or later, deployed after this release, not before it; uploading the matches of wildcard outputs needs a taskmaster whose filer supports path_prefix (tesk-core#134).
Breaking changes
- The image runs TESK API on Java 21 instead of Java 8; JVM options in
JAVA_OPTSthat Java 21 removed stop the JVM at startup (#162, #161) - A single task's executors are selected by the labels
taskmaster-nameandjob-type=executor, so that TESK API works with taskmasters that also label the filers' jobs (tesk-core 0.12.0 and later); upgrade TESK API first and such a taskmaster after the rollout, or both at once with theRecreatestrategy (#182, #183) - Access tokens are read only from the
Authorization: Bearerheader; anaccess_tokenparameter in the URL or a form is ignored (#165, #175) - Outputs with wildcards but no
path_prefixare refused with400 Bad Request, unless they go tofile:URLs or URLs without a scheme (taken asfile:), as TES 1.1 requires (#148, #152) GET /taskswithoutpage_sizereturns pages of at most 256 tasks, the default of TES, instead of all tasks at once (#137, #158)- Tasks report
CANCELINGwhile the pods of a cancelled task still run, andPREEMPTEDwhen Kubernetes disrupted all failed executors (#144, #151) - TESK API runs on Spring Boot 4.1 (Spring Framework 7, Spring Security 7, Jackson 3) instead of 2.7; properties of Spring Boot itself that Spring Boot 3 or 4 renamed or removed are ignored, log timestamps are in ISO 8601 format, and the messages of
400 Bad Requestfor unreadable JSON may be worded differently (#169, #185) - The API description is OpenAPI 3.1 at
/v3/api-docs(springdoc-openapi instead of the unmaintained Springfox) instead of Swagger 2.0 at/v2/api-docs, which serves the same OpenAPI 3.1 description until the next minor or major release removes it (#188); the Swagger UI reads its configuration from/v3/api-docs/swagger-configinstead of/swagger-resources(#166, #185)
Features
- Optional access for tokens of allow-listed service clients (OAuth 2.0 client credentials grant), checked locally against the issuer's keys, as principals of their own that create, list, get and cancel their own tasks; settings
TESK_API_AUTHORISATION_SERVICE_CLIENTS_*(#153, #154) - The task fields of TES 1.1 are accepted:
outputs[].path_prefixis passed to the taskmaster,inputs[].streamableis ignored, andbackend_parameters_strict: trueis refused; service info reports TES1.1.0(#133) executors[].ignore_error: truelets a task go on after the executor fails, with a taskmaster that supports it (tesk-core 0.12.0 and later) (#159, #174)logs[0].outputslists the files that the outputs filer uploaded, with a taskmaster and filer that record them (tesk-core 0.12.0 and later) (#160, #176)GET /tasksfilters bystate,tag_keyandtag_value(#143, #149)backend_parametersare no longer stored or returned; ignored keys are logged and listed insystem_logs, and service info liststesResources_backend_parameters([]) (#145, #147)- Experimental: private executor images, with registry credentials in the executor fields
docker_registry,docker_loginanddocker_pass, passed to Kubernetes as image-pull Secrets owned by the taskmaster's job;docker_loginanddocker_passgo only into the Secret and are never returned, whiledocker_registryis stored and returned (#55, #141) - Optional CORS for browser clients of the origins listed in
TESK_API_CORS_ALLOWED_ORIGINS(exact origins, no wildcards; none by default): their preflight requests are answered without a token, also with theauthprofile, while other requests need a token as before (#187, #195)
Security
- Tokens are validated with the resource server of Spring Security instead of the end-of-life Spring Security OAuth; the
401for an invalid token no longer repeats the token, malformed bearer headers are refused without a call to the user info endpoint, and calls to that endpoint time out (#165, #175) - SnakeYAML 2 instead of 1.33 (CVE-2022-1471), with version 28 of the Kubernetes Java client (#164, #178)
- Spring Boot 4.1 brings supported versions of Spring, Tomcat and their dependencies with fixes for the vulnerabilities that the image scan had accepted; the list of accepted vulnerabilities is empty (#169, #170, #185)
Fixes
- Inputs and outputs with
"type": nullare accepted asFILE, as whentypeis left out, as TES 1.1 allows (#146, #150) - Inputs are validated without a JavaScript engine, which Java 21 lacks; the message for an input without a source now names the check (
inputs[0].urlOrContentGiven) (#162, #161) - Run outside a cluster with a token in its kubeconfig file, TESK API can cancel tasks: patches keep the token's
Bearerprefix (#164, #178) - A user info answer without
subgives401instead of500(#165, #175) - A groups claim of
nullgives no group authorities instead of one namednull, and a task that can't be serialized fails its creation with500instead of creating a job that can't run (#179, #184) - Behind a reverse proxy on an internal address, TESK API takes its public URL (e.g. the server URL of the API description, which the Swagger UI uses for "Try it out") from the
X-Forwarded-*headers also outside Kubernetes:server.forward-headers-strategy=nativereplacesserver.use-forward-headers, which Spring Boot no longer reads (#192, #193) GET /tasksreads only the filers' jobs of the page's tasks, by the labeltaskmaster-nameof tesk-core 0.12.0 and later, instead of all filers' jobs in the namespace, which output file logs can make large; while a task of the page may have filers but has no labelled filer job (e.g. one of an older taskmaster), it also reads the unlabelled filers' jobs of the namespace, so that such tasks are still matched (#177, #196)
Documentation
Maintenance
- Version 28 of the Kubernetes Java client instead of 11 (#164, #178)
- The version overrides of the Spring Boot 2.7 era are gone, and Dependabot updates Spring Security, WireMock (now 3) and OpenAPI Generator (now 7, generating code for Spring Boot 3 and later) again (#168, #170, #185)
- Tests cover all lines and branches, enforced by JaCoCo and Codecov (#179, #184)
- CI runs of pull requests are cancelled only by newer commits (#142)
- Dependabot labels its pull requests with
type: dependenciesonly (#180, #181) - The tests run on JUnit 5 (Jupiter) instead of JUnit 4;
CONFIGMAP,CONFIGMAPNAMESPACEandCONFIGMAPKEYare read through Spring's environment instead of directly from the process's environment variables (#167, #194) - Updates of the docs dependencies (mkdocs-material 9.8.1, pymdown-extensions 13.1) and of the GitHub Actions (docker/login-action 4.8.0, shared actions of elixir-cloud-aai/.github 1.1.0) (#171, #173)
- Release preparation (#156, #155, #190, #191, #197, #198)
Commits
- feat: support private executor images by @pavelnikonorov in #141
- feat: accept the task fields of TES 1.1 by @anuragxxd in #133
- ci: cancel PR runs only for new commits by @uniqueg in #142
- feat: drop unsupported backend_parameters by @uniqueg in #147
- feat: filter tasks by state and tags by @uniqueg in #149
- fix: accept a null type of inputs and outputs by @uniqueg in #150
- feat: report the CANCELING and PREEMPTED states by @uniqueg in #151
- feat: refuse wildcards without path_prefix by @uniqueg in #152
- feat(auth): accept tokens of allow-listed clients by @uniqueg in #154
- feat: default page_size to 256 as TES specifies by @uniqueg in #158
- build: move to Java 21 by @uniqueg in #161
- feat: support executors[].ignore_error by @uniqueg in #174
- feat: report output file logs by @uniqueg in #176
- refactor: use Spring Security's resource server by @uniqueg in #175
- build: update the Kubernetes client by @uniqueg in #178
- ci: let Dependabot label all its pull requests by @uniqueg in #181
- fix: select only executor jobs for a single task by @uniqueg in #183
- test: reach 100% test coverage by @uniqueg in #184
- build: prepare the 2.0.0 release by @uniqueg in #155
- build: move to Spring Boot 4 by @uniqueg in #185
- build: update the 2.0.0 release prep by @uniqueg in #191
- build(deps): bump the docs-dependencies group across 1 directory with 2 updates by @dependabot[bot] in #171
- build(deps): bump the github-actions group across 1 directory with 4 updates by @dependabot[bot] in #173
- fix: honour forwarded headers outside Kubernetes by @uniqueg in #193
- test: move the tests to JUnit 5 by @uniqueg in #194
- feat: allow CORS for configured origins by @uniqueg in #195
- perf: list only the page's filer jobs by @uniqueg in #196
- build: update the 2.0.0 release prep again by @uniqueg in #198
New Contributors
- @pavelnikonorov made their first contribution in #141
- @anuragxxd made their first contribution in #133
Full Changelog: v1.2.0...v2.0.0