[Cherry-pick to branch-1.3] [#12851] fix(iceberg-rest): Share the managed memory catalog in auxiliary mode (#12852) - #12923
Merged
Conversation
…iary mode (#12852) ### What changes were proposed in this pull request? - Expose the Iceberg backend owned by `IcebergCatalogOperations`. - Reuse that backend when the auxiliary Iceberg REST service serves a memory catalog. - Resolve the REST `default_catalog` alias to the configured Gravitino catalog. - Prevent the REST wrapper from closing the borrowed catalog. - Add regression tests covering namespace, table, and view creation for both explicit catalog names and the default catalog alias. ### Why are the changes needed? The auxiliary Iceberg REST service currently creates a second InMemoryCatalog. REST creates are written to that instance, while metadata import uses the instance owned by Gravitino CatalogManager. Because in-memory state is local to each instance, the import cannot find the newly created object and leaves Iceberg and Gravitino metadata inconsistent. Fix: #12851 ### Does this PR introduce _any_ user-facing change? Yes. Namespace, table, and view creation through the Iceberg REST API now succeeds with a memory backend, and the created objects are visible through the Gravitino metadata API. No public API or configuration property is added or removed. ### How was this patch tested? - `./gradlew spotlessApply` - `./gradlew :iceberg:iceberg-common:test --tests org.apache.gravitino.iceberg.common.ops.TestIcebergCatalogWrapper` - `./gradlew :catalogs:catalog-lakehouse-iceberg:compileJava` - `./gradlew :iceberg:iceberg-rest-server:test --tests org.apache.gravitino.iceberg.service.TestIcebergCatalogWrapperManagerForREST` - `./gradlew :iceberg:iceberg-rest-server:test --tests org.apache.gravitino.iceberg.service.TestIcebergCatalogWrapperManagerForREST --tests org.apache.gravitino.iceberg.service.dispatcher.TestIcebergNamespaceHookDispatcher --tests org.apache.gravitino.iceberg.service.dispatcher.TestIcebergTableHookDispatcher --tests org.apache.gravitino.iceberg.service.dispatcher.TestIcebergViewHookDispatcher` - `./gradlew :catalogs:catalog-lakehouse-iceberg:test --tests org.apache.gravitino.catalog.lakehouse.iceberg.integration.test.CatalogIcebergRestIT -PskipTests -PtestMode=embedded -PjdbcBackend=h2 -PskipDockerTests=false -PskipWeb=true` # Conflicts: # catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/IcebergCatalogOperations.java # catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/TestIcebergCatalogOperations.java # core/src/main/java/org/apache/gravitino/catalog/CatalogManager.java # iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/utils/IcebergCatalogUtil.java # iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/provider/DynamicIcebergConfigProvider.java # iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/provider/TestDynamicIcebergConfigProvider.java
github-actions
Bot
added
cherry-pick
roryqi
removed
the
cherry-pick-conflict
roryqi
changed the title
Author
Code Coverage Report
Files |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-pick Information:
branch-1.3Resolution:
branch-1.3authentication and credential handling.Validation:
./gradlew spotlessApply./gradlew :iceberg:iceberg-common:test --tests org.apache.gravitino.iceberg.common.utils.TestIcebergCatalogUtil --tests org.apache.gravitino.iceberg.common.ops.TestIcebergCatalogWrapper./gradlew :catalogs:catalog-lakehouse-iceberg:test --tests org.apache.gravitino.catalog.lakehouse.iceberg.TestIcebergCatalogOperations --tests org.apache.gravitino.catalog.lakehouse.iceberg.TestIcebergSchema./gradlew :iceberg:iceberg-rest-server:test --tests org.apache.gravitino.iceberg.service.provider.TestDynamicIcebergConfigProvider --tests org.apache.gravitino.iceberg.service.TestIcebergCatalogWrapperManagerForREST./gradlew :core:test --tests org.apache.gravitino.catalog.TestCatalogManagergit diff --check