Skip to content

feat(blob view): support blob-view.resolve.enabled to preserve blob view references at read time#414

Merged
lszskye merged 1 commit into
alibaba:mainfrom
SteNicholas:PAIMON-413
Jul 13, 2026
Merged

feat(blob view): support blob-view.resolve.enabled to preserve blob view references at read time#414
lszskye merged 1 commit into
alibaba:mainfrom
SteNicholas:PAIMON-413

Conversation

@SteNicholas

@SteNicholas SteNicholas commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Purpose

Linked issue: close #413

Align with CoreOptions.BLOB_VIEW_RESOLVE_ENABLED in apache/paimon. This PR adds the blob-view.resolve.enabled option (default true). When set to false, reading a table with blob-view-field passes the serialized BlobViewStruct bytes through without resolving them from upstream tables: DataEvolutionSplitRead::WrapWithBlobViewResolverIfNeeded returns the inner reader as-is, skipping the prescan/resolver entirely and no longer requiring blob-view-upstream-warehouse. This enables forwarding blob view references from one blob-view table to another instead of materializing them. Since read-context options are merged on top of table schema options, the option can be disabled per read while the table keeps the default, which is the primary usage pattern.

Tests

  • UT CoreOptionsTest.TestDefaultValue: the option defaults to true.
  • UT CoreOptionsTest.TestFromMap: blob-view.resolve.enabled=false is parsed into CoreOptions::BlobViewResolveEnabled().
  • IT BlobTableInteTest.TestForwardBlobViewReference: end-to-end forward scenario aligned with BlobTableTest#testForwardBlobViewReference in apache/paimon — the source blob-view table keeps the default true without an upstream warehouse, so its default read fails while a read with the option dynamically disabled returns the serialized BlobViewStruct bytes byte-identically; the preserved references are then written into a second blob-view table, which still stores the original upstream references (verified via another resolve-disabled read) and resolves them to the actual upstream blob values on a default read.

API and Format

Adds the public option constant Options::BLOB_VIEW_RESOLVE_ENABLED in include/paimon/defs.h and the getter CoreOptions::BlobViewResolveEnabled(). No storage format or protocol change.

Documentation

The option is documented in the include/paimon/defs.h doc comment, following the existing blob-view options.

Generative AI tooling

Generated-by: Claude Code (Claude Fable 5)

🤖 Generated with Claude Code

Copilot AI review requested due to automatic review settings July 13, 2026 03:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new read-time toggle to control whether blob-view-field values are resolved against upstream tables, aligning paimon-cpp with Apache Paimon’s CoreOptions.BLOB_VIEW_RESOLVE_ENABLED. When disabled, the read path preserves serialized BlobViewStruct bytes and skips upstream-catalog access (and the blob-view-upstream-warehouse requirement), enabling blob-view forwarding workflows.

Changes:

  • Introduce public option key Options::BLOB_VIEW_RESOLVE_ENABLED (blob-view.resolve.enabled, default true) and plumb it into CoreOptions via CoreOptions::BlobViewResolveEnabled().
  • Update DataEvolutionSplitRead::WrapWithBlobViewResolverIfNeeded to bypass blob-view prescan/resolution when resolution is disabled.
  • Add unit and integration tests validating default/parsing behavior and pass-through semantics vs. enforced resolution behavior.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
include/paimon/defs.h Documents and declares the new public option constant for blob-view resolve behavior.
src/paimon/common/defs.cpp Defines the new option key string constant.
src/paimon/core/core_options.h Exposes CoreOptions::BlobViewResolveEnabled() getter.
src/paimon/core/core_options.cpp Adds option storage, parsing (default true), and getter implementation.
src/paimon/core/core_options_test.cpp Verifies default value and parsing from options map.
src/paimon/core/operation/data_evolution_split_read.cpp Skips blob-view resolver wrapping when resolution is disabled.
test/inte/blob_table_inte_test.cpp Integration test ensuring byte-identical passthrough when disabled and failure on missing upstream warehouse when re-enabled per-read.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…iew references at read time

Align with CoreOptions.BLOB_VIEW_RESOLVE_ENABLED in apache/paimon: the
option defaults to true; when set to false the read passes the serialized
BlobViewStruct bytes through without resolving them from upstream tables,
skips the prescan/resolver entirely, and no longer requires
blob-view-upstream-warehouse, so blob view values can be forwarded to
another blob-view table.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@lszskye lszskye left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@lszskye
lszskye merged commit 3d886ba into alibaba:main Jul 13, 2026
9 checks passed
@SteNicholas
SteNicholas deleted the PAIMON-413 branch July 20, 2026 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Support blob-view.resolve.enabled to preserve BlobViewStruct references at read time

3 participants