Skip to content

[rest] Propagate dependency read context in headers - #8823

Merged
leaves12138 merged 2 commits into
apache:masterfrom
leaves12138:codex/dependency-read-via-header
Jul 29, 2026
Merged

[rest] Propagate dependency read context in headers#8823
leaves12138 merged 2 commits into
apache:masterfrom
leaves12138:codex/dependency-read-via-header

Conversation

@leaves12138

Copy link
Copy Markdown
Contributor

Purpose

This replaces the broad grant-based approach from #8634 with a minimal,
backward-compatible mechanism for REST servers to receive the table that
initiated a runtime dependency read.

The dependency target is already identified by the existing table or data-token
endpoint path. This change adds an optional X-Paimon-Read-Via request header
containing the URL-encoded JSON Identifier of the outermost source table.

Changes

  • Keep the existing Catalog, RESTCatalog, and RESTApi method signatures,
    endpoints, request/response DTOs, and OpenAPI specification unchanged.
  • Derive a dependency-scoped CatalogContext only when a REST-backed table reads
    another table.
  • Propagate that context through Blob View resolution and
    blob-descriptor.source-table.
  • Preserve the outermost source table across nested dependency reads.
  • Reuse the existing header.* support so both target getTable and data-token
    requests carry the context.

Old clients send no header, and old servers can ignore the optional header.

Security

X-Paimon-Read-Via is client-provided context, not authorization proof. The
server remains responsible for authenticating the caller and deciding whether
and how to use the source/target relationship.

Validation

  • mvn -pl paimon-core -am -Pfast-build -DfailIfNoTests=false -DwildcardSuites=none -Dtest=CatalogEnvironmentTest,BlobDescriptorReaderFactoryTest,MockRESTCatalogTest test
    (132 tests passed)
  • mvn -pl paimon-core -am -Pfast-build -DfailIfNoTests=false -DwildcardSuites=none '-Dtest=BlobTableTest#testBlobViewE2E+testForwardBlobViewReference' test
    (2 tests passed)
  • mvn -pl paimon-core -am -DskipTests compile
  • git diff --check

@leaves12138
leaves12138 marked this pull request as ready for review July 23, 2026 08:37
@JingsongLi

Copy link
Copy Markdown
Contributor

cc @sundapeng to take a review.

@sundapeng

Copy link
Copy Markdown
Member

+1 from my side.

I think this is a lightweight and useful way to track runtime dependency reads. It keeps the outermost source table for nested reads, reuses the existing header.* mechanism, and keeps backward compatibility.

One minor suggestion: could we check the request paths in the test and make sure both the target table request and the data-token request contain X-Paimon-Read-Via?

@leaves12138

Copy link
Copy Markdown
Contributor Author

@sundapeng Thanks, good point. Addressed in 4f4e32f. The mock REST server now records headers by request path, and the test independently asserts that both the target table endpoint and the target data-token endpoint carry X-Paimon-Read-Via. Validation: the focused test passed, all 123 MockRESTCatalogTest tests passed on JDK 11, and the non-fast JDK 8 test-compile/checkstyle/spotless/enforcer checks passed.

@leaves12138
leaves12138 merged commit 1a67a4e into apache:master Jul 29, 2026
13 checks passed
}

Options dependencyOptions = new Options(options.toMap());
dependencyOptions.set(METASTORE, RESTCatalogFactory.IDENTIFIER);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dependencyReadContext() currently unconditionally sets metastore to the built-in REST Catalog identifier:

 dependencyOptions.set(METASTORE, RESTCatalogFactory.IDENTIFIER);

This may break custom REST Catalog implementations whose loaders extend RESTCatalogLoader.
For example, customer uses a custom REST Catalog factory:

metastore=paimon-test

Its Catalog loader extends RESTCatalogLoader, so it is recognized as a REST Catalog by the current logic. However, after metastore is overwritten with rest, the following call:

CatalogFactory.createCatalog(dependencyContext)

will create the built-in RESTCatalog instead of the original custom TestRestCatalog.

Could we preserve an existing metastore identifier and only fall back to the built-in REST identifier when it is absent? for example:

 if (!dependencyOptions.containsKey(METASTORE)) {
     dependencyOptions.set(
             METASTORE,
             RESTCatalogFactory.IDENTIFIER);
 }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants