Skip to content

[core] Support BlobRef in internal blob serializer#7689

Closed
leaves12138 wants to merge 5 commits into
apache:masterfrom
leaves12138:codex/blob-serializer-blobref
Closed

[core] Support BlobRef in internal blob serializer#7689
leaves12138 wants to merge 5 commits into
apache:masterfrom
leaves12138:codex/blob-serializer-blobref

Conversation

@leaves12138
Copy link
Copy Markdown
Contributor

What

  • support serializing both BlobData and BlobRef in BlobSerializer
  • serialize BlobRef as descriptor bytes without materializing blob payload
  • make internal binary blob write/read paths consistently use BlobSerializer
  • reject BlobStream in BlobSerializer because it cannot be safely materialized here

Why

BlobSerializer and internal binary writers previously called blob.toData() unconditionally. For BlobRef this forces loading the full blob payload, which defeats the lazy reference semantics and can be prohibitively expensive.

Tests

  • mvn -pl paimon-common -Pfast-build -Dtest=BlobSerializerTest,BinaryRowTest test

@leaves12138 leaves12138 changed the title Support BlobRef in internal blob serializer [core] Support BlobRef in internal blob serializer Apr 23, 2026
@leaves12138 leaves12138 marked this pull request as ready for review April 23, 2026 07:18
Copy link
Copy Markdown
Contributor

@JingsongLi JingsongLi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not using isBlobDescriptor to deserialize BlobDescriptor?

@leaves12138
Copy link
Copy Markdown
Contributor Author

Why not using isBlobDescriptor to deserialize BlobDescriptor?

Reasonable

@leaves12138 leaves12138 requested a review from JingsongLi April 24, 2026 05:20
@leaves12138
Copy link
Copy Markdown
Contributor Author

Don't fix this until we meet it in reality

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants