Skip to content

tw runs dump fails on nextflow.log step: SDK sends Accept: application/json against a text/plain endpoint #621

@robsyme

Description

@robsyme

Summary

tw runs dump consistently fails on its final step (downloading nf-{workflowId}.log) when run against current Seqera Platform (api ≥ 1.150.0). The call goes out with Accept: application/json against an endpoint that produces only text/plain, so Micronaut rejects it with a NotAcceptableException and the CLI surfaces it as:

ERROR: Unexpected error while processing request - Error ID: <id>

The failure is deterministic, not transient — every retry produces a fresh Error ID for the same reason.

Reproduction

Tested with the official v0.28.0 release (tw v0.28.0 (build 7fb3fef)) on macOS arm64 against api.cloud.seqera.io (api 1.150.0):

$ tw runs dump -i=4cNOpOa11ljJDB -o=archive.tar.gz -w=162690105070735
- Tower info
- Workflow general information
- Workflow metadata
- Workflow load data
- Workflow launch
- Workflow metrics
- Task details
- Workflow nextflow.log

 ERROR: Unexpected error while processing request - Error ID: 2H5f26TSIej0jXhJoqdFlM

Wire trace (tw --verbose ...):

10 > GET https://api.cloud.seqera.io/workflow/4cNOpOa11ljJDB/download?fileName=nf-4cNOpOa11ljJDB.log&workspaceId=...
10 > Accept: application/json
10 > User-Agent: tw/0.28.0 (osx-arm64)

10 < 400
10 < Content-Type: application/json
{"message":"Unexpected error while processing request - Error ID: 2H5f26TSIej0jXhJoqdFlM"}

Server log for that Error ID confirms the cause:

io.micronaut.http.server.exceptions.NotAcceptableException:
  Specified Accept Types [application/json] not supported. Supported types: [text/plain]
    at io.micronaut.http.server.RequestLifecycle.onRouteMiss(RequestLifecycle.java:520)

Root cause

The backend route GET /workflow/{workflowId}/download (operationId DownloadWorkflowLog) in WorkflowExController.groovy declares @Produces(MediaType.TEXT_PLAIN) after seqeralabs/platform#10845 (merged 2026-04-22). The same is true of GET /workflow/{workflowId}/download/{taskId} (DownloadWorkflowTaskLog).

The tower-java-sdk client used by tw CLI sends Accept: application/json for these calls regardless of SDK version. Verified with SDK 1.114.0 (tw v0.25.0), 1.133.0 (tw v0.26.0/v0.27.0), and 1.145.0 (tw v0.28.0) — all three send application/json.

The spec itself (seqera-api.yml in tower-sdk-gencode) is correct: response 200 declares text/plain and response 400 declares application/json. The bug is in the OpenAPI generator's stock jersey2 template (org.openapi.generator 7.17.0): localVarAccepts is built from every response's content type across all status codes, and ApiClient.selectHeaderAccept prefers JSON when present:

for (String accept : accepts) {
    if (isJsonMime(accept)) return accept;
}

So the application/json from the 400 error response leaks into the success-path Accept header. Any File-returning endpoint with a JSON error schema in the spec is affected the same way.

Affected code

src/main/java/io/seqera/tower/cli/commands/runs/DumpCmd.java:285:

File nextflowLog = workflowsApi().downloadWorkflowLog(
    workflow.getId(),
    String.format("nf-%s.log", workflow.getId()),
    workspaceId
);

addTaskLog at the same file calls workflowsApi().downloadWorkflowTaskLog(...), which has the same Accept-header bug. The 400 is currently masked by the 404/400 fallback in addTaskLog, but the call-site fix should still send the correct Accept on those calls so a real 400 isn't silently swallowed in future.

Fix

The endpoint serves a .log file — the body is genuinely text/plain, so the correct Accept header is text/plain (or */*). Two paths:

  1. Targeted call-site fix (this repo): override the Accept header for these two calls without waiting for an SDK regeneration. Either invoke the lower-level ApiClient directly with explicit localVarAccept = "text/plain", or wrap the call in a small interceptor that rewrites the Accept header for /workflow/{id}/download and /workflow/{id}/download/{taskId}. Single-PR, single-release fix.
  2. SDK fix (tower-sdk-gencode), one of:
    • Spec edit: drop content: { application/json: ... } from the 4xx responses on DownloadWorkflowLog and DownloadWorkflowTaskLog so localVarAccepts collapses to ["text/plain"]. Targeted, low-risk; doesn't fix the generator-wide problem.
    • Custom mustache template override: patch the jersey2 api.mustache so localVarAccepts is built from 2xx responses only. Generator-wide fix; introduces a template fork to maintain.

A platform-side mitigation (widening @Produces to also accept application/json) is being considered as a fast path to unblock the field, but the canonical fix lives here in the CLI.

Customer impact

tw runs dump is currently broken for everyone on every released tw version against current Platform. Surfaced to support as FD-7502 (Atlas Data Storage); likely to surface in more tickets as customers exercise the dump flow.

Workarounds (until fixed)

  • The first seven steps of the dump still complete; the partial archive on disk has everything except nf-*.log.
  • nf-{workflowId}.log is uploaded to the workflow's S3/GCS work directory by Nextflow, so customers can pull it directly from the work directory rather than via the dump endpoint.

Related

  • seqeralabs/platform#10845 — the change that exposed the latent client bug by tightening the route's @Produces.
  • FD-7502 — original support ticket.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions