Skip to content

feat: optional compression for RPC responses#6951

Merged
LesnyRumcajs merged 1 commit intomainfrom
optional-compression-rpc
Apr 21, 2026
Merged

feat: optional compression for RPC responses#6951
LesnyRumcajs merged 1 commit intomainfrom
optional-compression-rpc

Conversation

@LesnyRumcajs
Copy link
Copy Markdown
Member

@LesnyRumcajs LesnyRumcajs commented Apr 21, 2026

Summary of changes

Made compression optional and with a configurable threshold. For eth_chainId this yields:

Lotus: average 0.0435 ms
Forest main: average 0.2039 ms
Forest this PR: average 0.0268 ms

That's over 7x improvement for, AFAIK, most widely used RPC method.

This will improve perf of most other methods as well, but I don't have numbers for those.

Changes introduced in this pull request:

  • Set the HTTP response compression threshold for the RPC server to 1 KiB. Configurable via FOREST_RPC_COMPRESS_MIN_BODY_SIZE; set a negative value to disable compression entirely. Small JSON-RPC responses such as eth_chainId are no longer gzip-encoded, yielding a large throughput and latency improvement on high-QPS workloads.

Reference issue to close (if applicable)

Closes

Other information and links

Lotus

--- lotus (http://127.0.0.1:1234/rpc/v1) ---
warmup (200 requests)...
Summary:
  Success rate: 100.00%
  Total:        4472.2850 ms
  Slowest:      9.6577 ms
  Fastest:      0.0199 ms
  Average:      0.0435 ms
  Requests/sec: 223599.3438

  Total data:   41.96 MiB
  Size/request: 44 B
  Size/sec:     9.38 MiB

Response time histogram:
  0.020 ms [1]      |
  0.984 ms [999908] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  1.947 ms [77]     |
  2.911 ms [6]      |
  3.875 ms [2]      |
  4.839 ms [1]      |
  5.803 ms [0]      |
  6.766 ms [0]      |
  7.730 ms [1]      |
  8.694 ms [0]      |
  9.658 ms [4]      |

Response time distribution:
  10.00% in 0.0303 ms
  25.00% in 0.0340 ms
  50.00% in 0.0401 ms
  75.00% in 0.0485 ms
  90.00% in 0.0583 ms
  95.00% in 0.0663 ms
  99.00% in 0.0946 ms
  99.90% in 0.2306 ms
  99.99% in 0.8939 ms

Forest (this PR)

--- forest (http://127.0.0.1:2345/rpc/v1) ---
warmup (200 requests)...
Summary:
  Success rate: 100.00%
  Total:        2762.6859 ms
  Slowest:      2.9948 ms
  Fastest:      0.0132 ms
  Average:      0.0268 ms
  Requests/sec: 361966.5921

  Total data:   41.01 MiB
  Size/request: 43 B
  Size/sec:     14.84 MiB

Response time histogram:
  0.013 ms [1]      |
  0.311 ms [999825] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.610 ms [85]     |
  0.908 ms [51]     |
  1.206 ms [21]     |
  1.504 ms [2]      |
  1.802 ms [5]      |
  2.100 ms [3]      |
  2.398 ms [1]      |
  2.697 ms [4]      |
  2.995 ms [2]      |

Response time distribution:
  10.00% in 0.0191 ms
  25.00% in 0.0208 ms
  50.00% in 0.0235 ms
  75.00% in 0.0279 ms
  90.00% in 0.0346 ms
  95.00% in 0.0436 ms
  99.00% in 0.0916 ms
  99.90% in 0.1392 ms
  99.99% in 0.5124 ms

Forest (main)

--- forest (http://127.0.0.1:2345/rpc/v1) ---
warmup (200 requests)...
Summary:
  Success rate: 100.00%
  Total:        20494.5015 ms
  Slowest:      5.6443 ms
  Fastest:      0.0786 ms
  Average:      0.2039 ms
  Requests/sec: 48793.5751

  Total data:   43.87 MiB
  Size/request: 46 B
  Size/sec:     2.14 MiB

Response time histogram:
  0.079 ms [1]      |
  0.635 ms [998138] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  1.192 ms [1488]   |
  1.748 ms [239]    |
  2.305 ms [86]     |
  2.861 ms [32]     |
  3.418 ms [7]      |
  3.975 ms [3]      |
  4.531 ms [3]      |
  5.088 ms [0]      |
  5.644 ms [3]      |

Response time distribution:
  10.00% in 0.1616 ms
  25.00% in 0.1748 ms
  50.00% in 0.1924 ms
  75.00% in 0.2183 ms
  90.00% in 0.2532 ms
  95.00% in 0.2828 ms
  99.00% in 0.3812 ms
  99.90% in 0.8123 ms
  99.99% in 1.9750 ms


bench script
https://gist.github.com/LesnyRumcajs/402f3528cbca69ae68f0b11987517ba6

Change checklist

  • I have performed a self-review of my own code,
  • I have made corresponding changes to the documentation. All new code adheres to the team's documentation standards,
  • I have added tests that prove my fix is effective or that my feature works (if possible),
  • I have made sure the CHANGELOG is up-to-date. All user-facing changes should be reflected in this document.

Outside contributions

  • I have read and agree to the CONTRIBUTING document.
  • I have read and agree to the AI Policy document. I understand that failure to comply with the guidelines will lead to rejection of the pull request.

Summary by CodeRabbit

  • New Features

    • RPC HTTP response compression is now configurable via FOREST_RPC_COMPRESS_MIN_BODY_SIZE (default 1 KiB; negative disables).
  • Bug Fixes

    • Very small JSON-RPC responses (e.g., eth_chainId) are no longer gzip-compressed.
  • Documentation

    • Added reference for the compression env var and updated the environment-variables table layout.
  • Chores

    • Updated dictionary word list to include the token "unparsable".

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 21, 2026

Walkthrough

Adds a configurable HTTP response compression layer for the RPC server controlled by FOREST_RPC_COMPRESS_MIN_BODY_SIZE (default 1024). Compression can be disabled with negative values; values above u16::MAX are clamped. Documentation, changelog, and a dictionary entry were updated.

Changes

Cohort / File(s) Summary
RPC Compression Implementation
src/rpc/compression_layer.rs, src/rpc/mod.rs
New conditional compression middleware implemented as a Tower Layer/Service. Parses FOREST_RPC_COMPRESS_MIN_BODY_SIZE (default, disable, clamp, warn on unparsable) and applies compression only when responses exceed the configured threshold. Unit tests added for parsing and gzip behavior.
Environment docs
docs/docs/users/reference/env_variables.md
Added FOREST_RPC_COMPRESS_MIN_BODY_SIZE entry with default, range [-1, 65535], clamping, and disable semantics; reformatted environment variables table layout.
Changelog
CHANGELOG.md
Added unreleased entry describing RPC HTTP response compression behavior, default threshold 1 KiB, and note that negative disables compression.
Dictionary
.config/forest.dic
Incremented dictionary count and added new token unparsable.

Sequence Diagram(s)

sequenceDiagram
  actor Client
  participant HTTP_Server as "RPC HTTP Server"
  participant CompressionLayer as "CompressionLayer\n(predicate + SizeAbove)"
  participant Handler as "RPC Handler / Service"
  Client->>HTTP_Server: Send request (may include Accept-Encoding)
  HTTP_Server->>CompressionLayer: Pass request/response pipeline
  CompressionLayer->>Handler: Invoke inner service (request)
  Handler-->>CompressionLayer: Produce Response (body)
  CompressionLayer->>CompressionLayer: Check body size vs COMPRESS_MIN_BODY_SIZE
  alt compress?
    CompressionLayer->>Client: Return gzip-compressed Response
  else no compress
    CompressionLayer->>Client: Return uncompressed Response
  end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested reviewers

  • akaladarshi
  • hanabi1224
  • elmattic
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately captures the main objective: making HTTP response compression for RPC responses optional and configurable, which is the core focus across all changed files.
Docstring Coverage ✅ Passed Docstring coverage is 83.33% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch optional-compression-rpc
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch optional-compression-rpc

Comment @coderabbitai help to get the list of available commands and usage tips.

@LesnyRumcajs LesnyRumcajs force-pushed the optional-compression-rpc branch from 9b7dac0 to 997cadb Compare April 21, 2026 15:32
@LesnyRumcajs LesnyRumcajs force-pushed the optional-compression-rpc branch from 997cadb to f9443cf Compare April 21, 2026 15:36
@LesnyRumcajs LesnyRumcajs marked this pull request as ready for review April 21, 2026 15:48
@LesnyRumcajs LesnyRumcajs requested a review from a team as a code owner April 21, 2026 15:48
@LesnyRumcajs LesnyRumcajs requested review from akaladarshi and hanabi1224 and removed request for a team April 21, 2026 15:48
@LesnyRumcajs LesnyRumcajs added the RPC requires calibnet RPC checks to run on CI label Apr 21, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/rpc/compression_layer.rs`:
- Around line 53-70: The parsing should accept integers outside i32 so
out-of-range inputs are handled (negative disables, too-large clamps) instead of
falling back to DEFAULT; change the parse from raw.parse::<i32>() to a wider
signed type (e.g., raw.parse::<i128>()), use a matching i128 max derived from
u16::MAX, then handle: on parse error log and return Some(DEFAULT) as before, if
parsed < 0 return None, if parsed > max emit the clamp warning and clamp to max,
and finally return Some(parsed.min(max) as u16). Update uses of parsed and max
in the warning messages accordingly (symbols: COMPRESS_MIN_BODY_SIZE_VAR,
DEFAULT, raw, parsed, max).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 9982f8d0-3821-4cd0-8788-7aca10cd3043

📥 Commits

Reviewing files that changed from the base of the PR and between 799007a and f9443cf.

📒 Files selected for processing (5)
  • .config/forest.dic
  • CHANGELOG.md
  • docs/docs/users/reference/env_variables.md
  • src/rpc/compression_layer.rs
  • src/rpc/mod.rs

Comment thread src/rpc/compression_layer.rs Outdated
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docs/docs/users/reference/env_variables.md`:
- Line 23: Update the documentation entry for FOREST_RPC_MAX_CONNECTIONS to
indicate it accepts 0 by changing the type description from "positive integer"
to "non-negative integer (0 allowed)" or similar wording that notes "0 disables
or allows no connections"; keep existing default and example values but clarify
the semantics of 0 so users know it's intentionally valid. Reference the
variable name FOREST_RPC_MAX_CONNECTIONS and ensure the table cell currently
reading "positive integer" is replaced with the new wording.
- Line 24: Update the doc for FOREST_RPC_COMPRESS_MIN_BODY_SIZE to remove the
contradiction by stating that the accepted values are integers in the range [-1,
65535] where -1 (exactly -1) disables compression and any value >65535 is
clamped to 65535; ensure the description for FOREST_RPC_COMPRESS_MIN_BODY_SIZE
explicitly says "set to -1 to disable compression" (not "any negative value")
and retains the existing default and example values (1024 default, 2048 example)
so readers know the exact behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 236db9cd-362c-401a-854c-ccea8e377459

📥 Commits

Reviewing files that changed from the base of the PR and between f9443cf and 8b9c010.

📒 Files selected for processing (5)
  • .config/forest.dic
  • CHANGELOG.md
  • docs/docs/users/reference/env_variables.md
  • src/rpc/compression_layer.rs
  • src/rpc/mod.rs
✅ Files skipped from review due to trivial changes (2)
  • .config/forest.dic
  • CHANGELOG.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/rpc/mod.rs

Comment thread docs/docs/users/reference/env_variables.md
Comment thread docs/docs/users/reference/env_variables.md
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 21, 2026

Codecov Report

❌ Patch coverage is 93.96552% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.00%. Comparing base (799007a) to head (8b9c010).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/rpc/compression_layer.rs 93.91% 6 Missing and 1 partial ⚠️
Additional details and impacted files
Files with missing lines Coverage Δ
src/rpc/mod.rs 89.20% <100.00%> (ø)
src/rpc/compression_layer.rs 93.91% <93.91%> (ø)

... and 8 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 799007a...8b9c010. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@LesnyRumcajs LesnyRumcajs mentioned this pull request Apr 21, 2026
6 tasks
Copy link
Copy Markdown
Collaborator

@akaladarshi akaladarshi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@LesnyRumcajs LesnyRumcajs added this pull request to the merge queue Apr 21, 2026
Merged via the queue into main with commit 3dcb3e4 Apr 21, 2026
44 checks passed
@LesnyRumcajs LesnyRumcajs deleted the optional-compression-rpc branch April 21, 2026 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

RPC requires calibnet RPC checks to run on CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants