[rest] Propagate dependency read context in headers - #8823
Conversation
|
cc @sundapeng to take a review. |
|
+1 from my side. I think this is a lightweight and useful way to track runtime dependency reads. It keeps the outermost source table for nested reads, reuses the existing One minor suggestion: could we check the request paths in the test and make sure both the target table request and the data-token request contain |
|
@sundapeng Thanks, good point. Addressed in 4f4e32f. The mock REST server now records headers by request path, and the test independently asserts that both the target table endpoint and the target data-token endpoint carry X-Paimon-Read-Via. Validation: the focused test passed, all 123 MockRESTCatalogTest tests passed on JDK 11, and the non-fast JDK 8 test-compile/checkstyle/spotless/enforcer checks passed. |
| } | ||
|
|
||
| Options dependencyOptions = new Options(options.toMap()); | ||
| dependencyOptions.set(METASTORE, RESTCatalogFactory.IDENTIFIER); |
There was a problem hiding this comment.
dependencyReadContext() currently unconditionally sets metastore to the built-in REST Catalog identifier:
dependencyOptions.set(METASTORE, RESTCatalogFactory.IDENTIFIER);This may break custom REST Catalog implementations whose loaders extend RESTCatalogLoader.
For example, customer uses a custom REST Catalog factory:
metastore=paimon-testIts Catalog loader extends RESTCatalogLoader, so it is recognized as a REST Catalog by the current logic. However, after metastore is overwritten with rest, the following call:
CatalogFactory.createCatalog(dependencyContext)will create the built-in RESTCatalog instead of the original custom TestRestCatalog.
Could we preserve an existing metastore identifier and only fall back to the built-in REST identifier when it is absent? for example:
if (!dependencyOptions.containsKey(METASTORE)) {
dependencyOptions.set(
METASTORE,
RESTCatalogFactory.IDENTIFIER);
}
Purpose
This replaces the broad grant-based approach from #8634 with a minimal,
backward-compatible mechanism for REST servers to receive the table that
initiated a runtime dependency read.
The dependency target is already identified by the existing table or data-token
endpoint path. This change adds an optional
X-Paimon-Read-Viarequest headercontaining the URL-encoded JSON
Identifierof the outermost source table.Changes
endpoints, request/response DTOs, and OpenAPI specification unchanged.
another table.
blob-descriptor.source-table.header.*support so both targetgetTableand data-tokenrequests carry the context.
Old clients send no header, and old servers can ignore the optional header.
Security
X-Paimon-Read-Viais client-provided context, not authorization proof. Theserver remains responsible for authenticating the caller and deciding whether
and how to use the source/target relationship.
Validation
mvn -pl paimon-core -am -Pfast-build -DfailIfNoTests=false -DwildcardSuites=none -Dtest=CatalogEnvironmentTest,BlobDescriptorReaderFactoryTest,MockRESTCatalogTest test(132 tests passed)
mvn -pl paimon-core -am -Pfast-build -DfailIfNoTests=false -DwildcardSuites=none '-Dtest=BlobTableTest#testBlobViewE2E+testForwardBlobViewReference' test(2 tests passed)
mvn -pl paimon-core -am -DskipTests compilegit diff --check