Fix incorrect getSerializedValueSize for ColumnLowCardinality with nullable dictionary#97647
Merged
alexey-milovidov merged 2 commits intomasterfrom Feb 23, 2026
Merged
Conversation
…h nullable dictionary `ColumnLowCardinality` did not override `getSerializedValueSize`, so it used the default `byteSizeAt` which does not account for the null flag byte that `ColumnUnique::serializeValueIntoArena/Memory` writes for nullable dictionaries. For example, for `LowCardinality(Nullable(Int64))` with a NULL value, `byteSizeAt` returned 8 but the actual serialized size is 1 (just the null flag). This caused `IColumnHelper::serializeValueIntoArenaWithNull` to allocate the wrong amount of space (too much), leaving uninitialized bytes in the serialized key. When a subsequent key column (e.g. a `ColumnString`) tried to deserialize from the ReadBuffer, it read garbage as the string size, triggering a logical error exception in ASan builds or `CANNOT_ALLOCATE_MEMORY` in release builds. The fix overrides `getSerializedValueSize` in both `ColumnUnique` (to account for the null flag byte) and `ColumnLowCardinality` (to delegate to the dictionary). https://s3.amazonaws.com/clickhouse-test-reports/json.html?REF=master&sha=b053840ef38b3b36bc7fb44fa6d5fb129571b2cd&name_0=MasterCI&name_1=AST+fuzzer+%28arm_asan%29 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
clickhouse-gh
bot
added
pr-critical-bugfix
pr-must-backport
| CREATE TABLE t_rollup_lc_nullable (value Nullable(Tuple(LowCardinality(Nullable(Int64))))) ENGINE = Memory; | ||
| INSERT INTO t_rollup_lc_nullable VALUES ((NULL)); | ||
|
|
||
| SELECT 1 FROM t_rollup_lc_nullable GROUP BY value, 'foo' WITH ROLLUP; |
Member
Author
There was a problem hiding this comment.
Yes, it reproduces the issue.
Member
Author
|
The change looks good to me. |
nihalzp
approved these changes
Feb 23, 2026
Comment on lines
+479
to
+480
| if (!nested_size) | ||
| return std::nullopt; |
Member
There was a problem hiding this comment.
Currently, the types that ColumnUnique supports cannot return std::nullopt. I guess this might be better because if in the future ColumnUnique expands supported types, this will not break.
robot-ch-test-poll
added
the
pr-synced-to-cloud
This was referenced Feb 23, 2026
robot-ch-test-poll1
added
the
pr-must-backport-synced
robot-clickhouse
added a commit
that referenced
this pull request
Feb 23, 2026
Cherry pick #97647 to 25.12: Fix incorrect getSerializedValueSize for ColumnLowCardinality with nullable dictionary
robot-clickhouse
added a commit
that referenced
this pull request
Feb 23, 2026
…lumnLowCardinality with nullable dictionary
robot-clickhouse
added a commit
that referenced
this pull request
Feb 23, 2026
Cherry pick #97647 to 26.1: Fix incorrect getSerializedValueSize for ColumnLowCardinality with nullable dictionary
robot-clickhouse
added a commit
that referenced
this pull request
Feb 23, 2026
…umnLowCardinality with nullable dictionary
robot-clickhouse-ci-1
added
the
pr-backports-created
nihalzp
added a commit
that referenced
this pull request
Feb 23, 2026
Backport #97647 to 26.1: Fix incorrect getSerializedValueSize for ColumnLowCardinality with nullable dictionary
Algunenano
pushed a commit
to Algunenano/ClickHouse
that referenced
this pull request
Feb 24, 2026
…erialized-value-size Fix incorrect getSerializedValueSize for ColumnLowCardinality with nullable dictionary
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ColumnLowCardinalitydid not overridegetSerializedValueSize, defaulting tobyteSizeAtwhich doesn't account for the null flag byte written byColumnUnique::serializeValueIntoArena/Memoryfor nullable dictionariesLowCardinality(Nullable(Int64))with a NULL value,byteSizeAtreturned 8 but the actual serialized size is 1 (just the null flag), leaving 7 uninitialized bytes in the arenaROLLUP/CUBEwhenLowCardinality(Nullable(...))appeared inside aNullable(Tuple(...))GROUP BY keygetSerializedValueSizein bothColumnUnique(to account for the null flag byte) andColumnLowCardinality(to delegate to the dictionary)CI report: https://s3.amazonaws.com/clickhouse-test-reports/json.html?REF=master&sha=b053840ef38b3b36bc7fb44fa6d5fb129571b2cd&name_0=MasterCI&name_1=AST+fuzzer+%28arm_asan%29
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):
Fix logical error exception during
GROUP BY ... WITH ROLLUP/CUBEwhen keys includeLowCardinality(Nullable(...))insideNullable(Tuple(...)).Documentation entry for user-facing changes
🤖 Generated with Claude Code