fix: replace bufio.Scanner with bufio.Reader to support large message…

[ezynda3]

The bufio.Scanner has a default 64KB token limit which causes 'token too long' errors when MCP servers send large messages (e.g., large tool responses, resource contents, or prompts). This change replaces Scanner with Reader.ReadString('\n') which can handle arbitrarily large lines.

Changes:

• client/transport/stdio.go: Changed stdout from *bufio.Scanner to *bufio.Reader
• testdata/mockstdio_server.go: Applied same fix to mock server
• client/transport/stdio_test.go: Added TestStdio_LargeMessages with tests for messages ranging from 1KB to 5MB to ensure the fix works correctly

The original code (pre-commit 4e353ac) used bufio.Reader, but was incorrectly changed to Scanner claiming it would avoid panics with long lines. This fix reverts to the Reader approach which actually handles large messages correctly.

Fixes issue where stdio clients fail with 'bufio.Scanner: token too long' error when communicating with servers that send large responses.

Description

Fixes #588

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• MCP spec compatibility implementation
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Code refactoring (no functional changes)
• Performance improvement
• Tests only (no functional changes)
• Other (please describe):

Checklist

• My code follows the code style of this project
• I have performed a self-review of my own code
• I have added tests that prove my fix is effective or that my feature works
• I have updated the documentation accordingly

MCP Spec Compliance

• This PR implements a feature defined in the MCP specification
• Link to relevant spec section: Link text
• Implementation follows the specification exactly

Additional Information

Summary by CodeRabbit

• Bug Fixes

  • Improved stdio transport to reliably handle very large messages without truncation or read failures.
  • More robust line parsing, including trimming CRLF and clearer error handling for malformed input.

• Tests

  • Added large-payload stress tests (up to ~5MB) to verify end-to-end integrity and response correctness across varying sizes.
  • Ensures consistent handling of notifications, requests, and responses under high-volume conditions.

[coderabbitai]

Walkthrough

Replaces bufio.Scanner with bufio.Reader for stdio transport and mock server, updating read loops to use ReadString with CRLF trimming. Adjusts initialization in NewIO and spawnCommand. Adds large message stress tests for echo requests across sizes up to ~5MB, with a duplicated test block included.

Changes

Cohort / File(s)	Summary
Stdio transport: switch Scanner→Reader client/transport/stdio.go	Changes stdout field from *bufio.Scanner to *bufio.Reader; updates NewIO and spawnCommand; replaces scan loop with ReadString('\n'), trims CRLF, and adapts error handling and message routing accordingly.
Tests: large payload echo client/transport/stdio_test.go	Adds TestStdio_LargeMessages generating random strings and verifying echo across sizes (~1KB–~5MB); includes helper generateRandomString. Note: the large-message test block is duplicated in the file.
Mock stdio server I/O change testdata/mockstdio_server.go	Replaces stdin reading via bufio.Scanner with bufio.Reader and ReadString('\n'); trims CRLF; unmarshals from line string; imports strings. No change to request handling semantics.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• Unified Client Transport Layer for Streamable HTTP Support #114 — Also modifies stdio transport and mock server to switch Scanner to Reader with corresponding read-loop updates.
• fix: use bufio.Scanner for stdio transport to avoid panic when stdio mcp server outputs a long line #464 — Touches the same Stdio stdout field and readResponses logic, reversing Reader/Scanner changes.
• fix(test/stdio): address io:read/write on closed pipe error #360 — Adjusts stdio line-reading flow (handling empty lines and writer-close ordering) in the same code path.

Suggested reviewers

• dugenkui03
• rwjblue-glean
• pottekkat

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title Check	✅ Passed	The title succinctly and accurately describes the primary change of replacing bufio.Scanner with bufio.Reader to enable support for large messages, matching the core purpose of the PR without unnecessary detail.
Description Check	✅ Passed	The PR description adheres closely to the repository’s template by providing a concise summary of the change, referencing the fixed issue (#588), specifying the type of change with the bug-fix checkbox selected, and including the checklist, MCP spec compliance, and additional information sections. All required headings from the template are present and the core content clearly explains why the Scanner-to-Reader switch was necessary and how it was implemented. Although the checklist section exists, individual items could be more accurately marked to reflect the added tests. Overall, the description fulfills the template’s requirements.

✨ Finishing touches

• 📝 Generate Docstrings

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch fix/stdio-large-messages

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

🧪 Early access (Sonnet 4.5): enabled

We are currently testing the Sonnet 4.5 model, which is expected to improve code review quality. However, this model may lead to increased noise levels in the review comments. Please disable the early access features if the noise level causes any inconvenience.

Note:

• Public repositories are always opted into early access features.
• You can enable or disable early access features from the CodeRabbit UI or by updating the CodeRabbit configuration file.

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

Merging this branch will not change overall coverage

Impacted Packages	Coverage Δ	🤖
github.com/mark3labs/mcp-go/client/transport	67.08% (ø)
github.com/mark3labs/mcp-go/testdata	0.00% (ø)

---

Coverage by file

Changed files (no unit tests)

Changed File	Coverage Δ	Total	Covered	Missed	🤖
github.com/mark3labs/mcp-go/client/transport/stdio.go	80.72% (ø)	166	134	32
github.com/mark3labs/mcp-go/testdata/mockstdio_server.go	0.00% (ø)	0	0	0

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

Changed unit test files

• github.com/mark3labs/mcp-go/client/transport/stdio_test.go