fix: cap HTTP-loaded script body size to prevent unbounded read DoS#107
Merged
Conversation
The HTTP loader returned resp.Body straight to compilers that io.ReadAll without limit, so a misbehaving server could exhaust process memory on any caller that loads scripts from arbitrary URLs. Changes: - Add HTTPOptions.MaxBodySize int64 with sentinel encoding: 0 falls back to DefaultMaxBodySize (10 MiB), negative disables the cap entirely. DefaultHTTPOptions() sets MaxBodySize to the default so existing callers (only inference.go in production) inherit the protection. - Add WithMaxBodySize(int64) option helper, mirroring the existing WithBasicAuth / WithTimeout shape. - Add ErrScriptTooLarge sentinel. - In GetReaderWithContext, switch from streaming resp.Body to an eager read of LimitReader(resp.Body, limit+1); the +1 trick distinguishes exactly-at-limit (pass) from over-limit (fail). On overflow, return ErrScriptTooLarge wrapping the URL and limit. - Returns io.NopCloser(bytes.NewReader(buf)) so downstream Close() is a no-op now that the loader owns the response body. Eager buffering matches what every existing compiler does (io.ReadAll under the hood), so no caller is affected. The Loader interface doesn't promise streaming. Closes #98
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
errcheck (check-blank: true) flagged the _, _ = w.Write(body) form. Use an if-err pattern with t.Logf so the broken-pipe noise surfaces in the test log without failing it.
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds a default maximum response size for HTTP-loaded scripts to prevent unbounded reads that could lead to memory exhaustion when loading from arbitrary URLs.
Changes:
- Introduces
HTTPOptions.MaxBodySize(default 10 MiB) plusWithMaxBodySizehelper to configure/disable the cap. - Updates
FromHTTP.GetReaderWithContextto eagerly read with a size limit and returnErrScriptTooLargeon overflow. - Adds
ErrScriptTooLargesentinel and new test coverage for max-body-size behavior.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| platform/script/loader/fromHTTP.go | Adds default size cap/config and enforces it during HTTP body reads. |
| platform/script/loader/errors.go | Adds a new sentinel error for oversized scripts. |
| platform/script/loader/fromHTTP_test.go | Adds tests validating max-body-size behavior (default, override, disable). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- fromHTTP.go: limit+1 wraps when MaxBodySize == math.MaxInt64, which made io.LimitReader return EOF immediately and broke the cap. Compute readLimit separately and only add 1 when limit < math.MaxInt64; the overflow branch is unreachable for MaxInt64 (no realistic body reaches it), which is fine. - fromHTTP_test.go: serveBody now takes the subtest's t so logs associate to the right test, and streams via io.CopyN over a small repeatingByte reader so multi-MiB cases don't allocate a full buffer up front.
Sonar (rule go:S3776) flagged GetReaderWithContext at cognitive complexity 17/15. Hoist the limit-resolve + LimitReader + overflow-check block into a new (l *FromHTTP).cappedBody(resp) helper. The HTTP wiring (request build, auth, headers, status check) stays in the main method; the size-cap policy lives next to its own godoc.
|
robbyt
added a commit
that referenced
this pull request
May 10, 2026
* docs: add CHANGELOG.md (Keep a Changelog 1.1.0) Closes #102 Repo had no CHANGELOG.md; users had to read GitHub release notes to find what changed between versions. With v1 ahead and a queue of breaking changes (#86, #87, #88, #89, #90, #91, #104), having a CHANGELOG before landing those gives downstream code reviewers and IDEs visibility into what's coming. Format follows Keep a Changelog 1.1.0 with the standard six headings (Added / Changed / Deprecated / Removed / Fixed / Security). [Unreleased] section captures the run of merged-but-not-released PRs since v0.7.0: - #103 polyscript.New[E] generic constructor (deprecates 12 FromXxx) - #105 slog.Handler optional in engine subpackages - #106 RequestToMap mutation fix - #107 HTTP loader MaxBodySize cap - #110 unify nil-handler on slog.Default(); drop stdout fallbacks - #113 WithLogHandler(nil)/WithLogger(nil) → no-op - #114 drop redundant nil-guards; tighten WithLogHandler doc - #115 docs/LOGGING.md - #117 fix bare top-level json.Number leak - #118 WithGlobals additive; drop dead URL check - #119 extism Eval test coverage Backfilled the three releases the issue called out (v0.5.0, v0.6.0, v0.7.0) from existing GitHub release notes, sorted into Keep a Changelog categories. Earlier releases (v0.0.x through v0.4.0) intentionally not backfilled per the issue scope; can be added in a follow-up if desired. Out of scope: the optional CI gate that fails PRs not touching CHANGELOG.md. Adds noise to small bug PRs and wants opt-out-label infrastructure to support it; better as a separate issue. * docs(CHANGELOG): clarify deprecated FromXxx still take positional handler Copilot review noted that "no constructor demands a handler" is misleading because the deprecated FromXxx constructors still take a positional `logHandler slog.Handler` argument (even though nil is accepted). Reword to distinguish the new generic constructor (no handler arg) from the deprecated ones (still take it, but nil is OK). --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The HTTP loader returned
resp.Bodystraight to compilers thatio.ReadAllwithout limit, so a misbehaving server could exhaust process memory on any caller that loads scripts from arbitrary URLs.HTTPOptions.MaxBodySize int64with sentinel encoding:0falls back toDefaultMaxBodySize(10 MiB), negative disables the cap.DefaultHTTPOptions()populates the default so existing callers (onlyinference.goin production) inherit protection without code changes.WithMaxBodySize(int64)option helper mirroringWithBasicAuth/WithTimeout.ErrScriptTooLargesentinel.GetReaderWithContextswitches from streamingresp.Bodyto an eager read ofio.LimitReader(resp.Body, limit+1); the+1trick distinguishes exactly-at-limit (pass) from over-limit (fail). On overflow, returnsErrScriptTooLargewrapping the URL and limit.io.NopCloser(bytes.NewReader(buf))so downstreamClose()is a no-op now that the loader owns the response body.Eager buffering matches what every compiler already does (
io.ReadAllunder the hood) and theLoaderinterface doesn't promise streaming, so no caller is affected.Test plan
TestFromHTTP_MaxBodySizecovers six cases: under limit, exactly at limit, over limit (assertserrors.Is(err, ErrScriptTooLarge)),WithMaxBodySize(-1)disables the cap, zero-value field falls back to default, default options enforce the 10 MiB cap.go test -race -count=1 ./...— greengo vet ./...— cleanCloses #98
https://claude.ai/code/session_01C61VEAmjxSnX5Xhbab8NvL
Generated by Claude Code