feat(blob view): support blob-view.resolve.enabled to preserve blob view references at read time#414
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new read-time toggle to control whether blob-view-field values are resolved against upstream tables, aligning paimon-cpp with Apache Paimon’s CoreOptions.BLOB_VIEW_RESOLVE_ENABLED. When disabled, the read path preserves serialized BlobViewStruct bytes and skips upstream-catalog access (and the blob-view-upstream-warehouse requirement), enabling blob-view forwarding workflows.
Changes:
- Introduce public option key
Options::BLOB_VIEW_RESOLVE_ENABLED(blob-view.resolve.enabled, defaulttrue) and plumb it intoCoreOptionsviaCoreOptions::BlobViewResolveEnabled(). - Update
DataEvolutionSplitRead::WrapWithBlobViewResolverIfNeededto bypass blob-view prescan/resolution when resolution is disabled. - Add unit and integration tests validating default/parsing behavior and pass-through semantics vs. enforced resolution behavior.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
include/paimon/defs.h |
Documents and declares the new public option constant for blob-view resolve behavior. |
src/paimon/common/defs.cpp |
Defines the new option key string constant. |
src/paimon/core/core_options.h |
Exposes CoreOptions::BlobViewResolveEnabled() getter. |
src/paimon/core/core_options.cpp |
Adds option storage, parsing (default true), and getter implementation. |
src/paimon/core/core_options_test.cpp |
Verifies default value and parsing from options map. |
src/paimon/core/operation/data_evolution_split_read.cpp |
Skips blob-view resolver wrapping when resolution is disabled. |
test/inte/blob_table_inte_test.cpp |
Integration test ensuring byte-identical passthrough when disabled and failure on missing upstream warehouse when re-enabled per-read. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…iew references at read time Align with CoreOptions.BLOB_VIEW_RESOLVE_ENABLED in apache/paimon: the option defaults to true; when set to false the read passes the serialized BlobViewStruct bytes through without resolving them from upstream tables, skips the prescan/resolver entirely, and no longer requires blob-view-upstream-warehouse, so blob view values can be forwarded to another blob-view table. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Linked issue: close #413
Align with
CoreOptions.BLOB_VIEW_RESOLVE_ENABLEDin apache/paimon. This PR adds theblob-view.resolve.enabledoption (defaulttrue). When set tofalse, reading a table withblob-view-fieldpasses the serializedBlobViewStructbytes through without resolving them from upstream tables:DataEvolutionSplitRead::WrapWithBlobViewResolverIfNeededreturns the inner reader as-is, skipping the prescan/resolver entirely and no longer requiringblob-view-upstream-warehouse. This enables forwarding blob view references from one blob-view table to another instead of materializing them. Since read-context options are merged on top of table schema options, the option can be disabled per read while the table keeps the default, which is the primary usage pattern.Tests
CoreOptionsTest.TestDefaultValue: the option defaults totrue.CoreOptionsTest.TestFromMap:blob-view.resolve.enabled=falseis parsed intoCoreOptions::BlobViewResolveEnabled().BlobTableInteTest.TestForwardBlobViewReference: end-to-end forward scenario aligned withBlobTableTest#testForwardBlobViewReferencein apache/paimon — the source blob-view table keeps the defaulttruewithout an upstream warehouse, so its default read fails while a read with the option dynamically disabled returns the serializedBlobViewStructbytes byte-identically; the preserved references are then written into a second blob-view table, which still stores the original upstream references (verified via another resolve-disabled read) and resolves them to the actual upstream blob values on a default read.API and Format
Adds the public option constant
Options::BLOB_VIEW_RESOLVE_ENABLEDininclude/paimon/defs.hand the getterCoreOptions::BlobViewResolveEnabled(). No storage format or protocol change.Documentation
The option is documented in the
include/paimon/defs.hdoc comment, following the existing blob-view options.Generative AI tooling
Generated-by: Claude Code (Claude Fable 5)
🤖 Generated with Claude Code