Stop severing skill artifact pulls at fixed timeouts - #6224
Merged
Conversation
Installing a skill whose OCI artifact takes a while to pull failed twice over, and neither failure named the real cause. The client gave up after 30s and reported the server as unreachable, when the server was healthy and mid-pull. Abandoning the request also cancels its context server-side, so no work survived for a retry to reuse. Past that, the skills router sat among the standard routers and inherited their flat 60s cap. The workload router is already exempt from it, on the grounds that image pulls take minutes; skills pull the same way. - classify timeouts as ErrRequestTimeout, separate from unreachability, and leave caller cancellation as neither - raise the client default and allow TOOLHIVE_API_TIMEOUT to override it - give the skills router per-route timeouts, long ones on the routes that move artifacts Closes #6212
samuv
requested review from
ChrisJBurns,
JAORMX,
amirejaz,
aponcedeleonch,
jhrozek,
rdimitrov and
reyortiz3
as code owners
August 6, 2026 08:02
JAORMX
previously approved these changes
Aug 6, 2026
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #6224 +/- ##
==========================================
- Coverage 72.48% 72.47% -0.01%
==========================================
Files 739 739
Lines 76728 76751 +23
==========================================
+ Hits 55613 55622 +9
- Misses 17151 17163 +12
- Partials 3964 3966 +2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
rdimitrov
approved these changes
Aug 6, 2026
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Installing a skill whose OCI artifact takes a while to pull fails, and the error blames the wrong thing:
The server is running. It was pulling the artifact. Investigating turned up two independent ceilings, and fixing only the client one is not enough — I verified that by fixing the client first and watching the same install fail again, 30s later:
1. The client gave up at 30s and reported unreachability.
WithTimeoutexisted but was not reachable from the CLI, so there was no way out. Worse, abandoning the request cancels its context server-side, so the server logscontext canceledand discards the work — retrying restarts from nothing rather than resuming.2. The skills router inherited a flat 60s cap. It sat in
standardRoutersinsetupDefaultRoutes, all of which getmiddleware.Timeout(60s). The workload router is already mounted outside that set, with the comment "image pulls can take minutes". Skills pull the same artifacts the same way; they were simply never given the same treatment.Changes:
ErrRequestTimeout, distinct fromErrServerUnreachable, so a healthy-but-slow server is not reported as absent. Caller cancellation (Ctrl-C) is classified as neither and surfaces ascontext.Canceled.TOOLHIVE_API_TIMEOUT(a Go duration, e.g.45s,30m) as an override. An explicitWithTimeoutstill outranks the env var; an unparseable or non-positive value is ignored with a warning rather than disabling the timeout.SkillsRoutergains per-route timeouts mirroringWorkloadRouter— long oninstall,sync,upgrade,build,push; short on the read routes — and is mounted outsidestandardRouters.Closes #6212
Type of change
Test plan
task test)task lint-fix)New tests:
TestTimeoutFromEnv(table: unset / durations / whitespace / bare number / garbage / zero / negative),TestNewDefaultClientTimeoutPrecedence(default → env → explicit option, plus env beating the client discovery installs), and three classification tests using real sockets — a slow server yieldsErrRequestTimeoutand notErrServerUnreachable, a closed port yields the reverse, and a cancelled caller context yields neither.Manually, a cold pull of
security-reviewagainst a localthv serve:Internal Server Errorandcontext deadline exceededserver-sidecontext canceled/context deadline exceededin the server logDoes this introduce a user-facing change?
Yes. Skill operations that pull OCI artifacts no longer fail on slow or large pulls. Timeout failures now say the request timed out instead of claiming the server is unreachable, and
TOOLHIVE_API_TIMEOUToverrides the client limit.Special notes for reviewers
Two things worth a deliberate look:
Scope. The issue is written about the client timeout, and this PR also changes server-side routing. I would normally split that, but the server cap makes the client fix useless on its own — the user-visible bug is not fixed by either half alone.
The default. The client default is now generous, on the reasoning that this timeout is a backstop against a wedged local server rather than a budget for the operation: the transfer size is unbounded, and cutting it short destroys the work instead of deferring it. If you would rather keep a short default and have callers opt in per-operation, say so — it is a one-line change plus threading an option through the pull-backed commands.
PluginsRouteris instandardRoutersandpluginsvcpulls artifacts too, so it likely has the same 60s ceiling. Left alone here to keep this PR to the reported bug; happy to file it.Generated with Claude Code