Skip to content

Add CapnProto message size limit#91888

Merged
antonio2368 merged 4 commits intomasterfrom
add-capn-proto-message-size-limit
Dec 12, 2025
Merged

Add CapnProto message size limit#91888
antonio2368 merged 4 commits intomasterfrom
add-capn-proto-message-size-limit

Conversation

@antonio2368
Copy link
Copy Markdown
Member

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Add CapnProto message size limit. It can be changed with format_capn_proto_max_message_size.

Close #90298

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

@antonio2368 antonio2368 force-pushed the add-capn-proto-message-size-limit branch from 355c608 to 74bc682 Compare December 10, 2025 14:36
@antonio2368 antonio2368 requested a review from Copilot December 10, 2025 14:36
@clickhouse-gh
Copy link
Copy Markdown
Contributor

clickhouse-gh bot commented Dec 10, 2025

Workflow [PR], commit [28f472b]

Summary:

job_name test_name status info comment
Integration tests (amd_binary, 4/5) failure
test_merge_tree_s3/test.py::test_merge_canceled_by_s3_errors[node-broken_s3] FAIL cidb, issue
AST fuzzer (amd_tsan) failure
Logical error: Function A with argument with type Nothing and default implementation for Nothing is expected to return result with type Nothing, got B FAIL cidb
BuzzHouse (amd_debug) failure
Logical error: 'Inconsistent AST formatting in Function_arrayElement: the query: FAIL cidb, issue
BuzzHouse (amd_ubsan) failure
UndefinedBehaviorSanitizer: undefined behavior (STID: 4443-51fd) FAIL cidb

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a configurable size limit for CapnProto messages to prevent excessive memory allocation from malformed or corrupted data. The default limit is set to 1 GiB.

Key Changes:

  • Introduces a new setting format_capn_proto_max_message_size with a default value of 1 GiB
  • Implements message size validation in the CapnProto input format reader
  • Adds a test case to verify the size limit enforcement

Reviewed changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/Core/FormatFactorySettings.h Declares the new format_capn_proto_max_message_size setting
src/Formats/FormatSettings.h Adds the max_message_size field to CapnProto format settings with 1 GiB default
src/Formats/FormatFactory.cpp Wires the new setting into the format settings initialization
src/Processors/Formats/Impl/CapnProtoRowInputFormat.h Adds max_message_size member variable to store the limit
src/Processors/Formats/Impl/CapnProtoRowInputFormat.cpp Implements the message size check and throws an error when exceeded
src/Core/SettingsChangesHistory.cpp Records the new setting in version 25.12 settings history
tests/queries/0_stateless/03756_capn_proto_message_size_limit.sql Adds test case for the size limit enforcement

@clickhouse-gh clickhouse-gh bot added the pr-improvement Pull request with some product improvements label Dec 10, 2025
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@Avogar Avogar self-assigned this Dec 10, 2025
@antonio2368 antonio2368 added this pull request to the merge queue Dec 12, 2025
Merged via the queue into master with commit 49461a8 Dec 12, 2025
125 of 130 checks passed
@antonio2368 antonio2368 deleted the add-capn-proto-message-size-limit branch December 12, 2025 13:50
@robot-ch-test-poll2 robot-ch-test-poll2 added the pr-synced-to-cloud The PR is synced to the cloud repo label Dec 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-improvement Pull request with some product improvements pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AddressSanitizer: exceeds maximum supported size with Capnproto input

4 participants