[#9048] feat(iceberg-rest-catalog): Add cache for scan planning.

[sunxiaojian]

What changes were proposed in this pull request?

Support scan planning endpoint for Iceberg REST server

Why are the changes needed?

Fix: #(9048)

Does this PR introduce any user-facing change?

N/A

How was this patch tested?

org.apache.gravitino.iceberg.service.rest.TestScanPlanCache

[sunxiaojian]

Waiting me fix it.

[sunxiaojian]

Waiting me fix it.

Done, Upgraded Iceberg to version 1.10.0 and upgraded the dependent Hadoop to Hadoop 3.

[FANNG1]

Thanks @sunxiaojian , it's an amazing feature, the overall architecture looks good to me, would you like to split this PR into three part to make each PR more simple to review?

1. upgrade Iceberg version
2. support scan planning interface
3. scan planning cache support

[sunxiaojian]

Thanks @sunxiaojian , it's an amazing feature, the overall architecture looks good to me, would you like to split this PR into three part to make each PR more simple to review?

1. upgrade Iceberg version
2. support scan planning interface
3. scan planning cache support

ok, I'll handle it.

[sunxiaojian]

Thanks @sunxiaojian , it's an amazing feature, the overall architecture looks good to me, would you like to split this PR into three part to make each PR more simple to review?

1. upgrade Iceberg version
2. support scan planning interface
3. scan planning cache support

ok, I'll handle it.

1. upgrade Iceberg version #9049
2. support scan planning interface #9050
3. Current PR.

Because the PRs are interdependent, they need to be merged in order. After each PR is merged, the remaining PRs need to merge in the main branch.

[sunxiaojian]

@FANNG1 I have already rebased, PTAL.

[Copilot]

Pull request overview

This PR implements a caching mechanism for Iceberg REST scan planning to improve query performance by caching scan plan results. The implementation adds a new ScanPlanCache class using Caffeine cache with configurable capacity and expiration, integrates it into the catalog wrapper, and adds extensive tests for the scan planning endpoint.

Key changes:

• New scan plan cache using Caffeine with automatic expiration and cleanup
• Configuration options for cache capacity and expiration time
• Integration with the Iceberg REST catalog wrapper for scan planning operations

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 18 comments.

Show a summary per file

File	Description
iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/ScanPlanCache.java	New cache implementation with Caffeine backend, custom cache key based on table identifier and scan parameters
iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/CatalogWrapperForREST.java	Integration of scan plan cache into planTableScan method (but with critical bug - cache checked after expensive operation)
iceberg/iceberg-common/src/main/java/org/apache/gravitino/iceberg/common/IcebergConfig.java	Added configuration entries for cache capacity and expiration
catalogs/catalog-common/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/IcebergConstants.java	Added constant definitions for cache configuration keys
iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/TestPlanTableScan.java	New test file with tests for scan planning endpoint (missing actual cache tests)
docs/iceberg-rest-service.md	Documentation for scan plan cache configuration and usage

[sunxiaojian]

@FANNG1 fixed.

[FANNG1]

seems we couldn't meet VERSION_1_1_0, how about using VERSION_1_2_0?

[FANNG1]

@sunxiaojian please take a look this comment

[sunxiaojian]

@FANNG1 fixed

[FANNG1]

@FANNG1 fixed

Thx, just minor comments, PTAL

[sunxiaojian]

@FANNG1 fixed

Thx, just minor comments, PTAL

@FANNG1 fixed, but you need to help me retry the CI, thanks

[FANNG1]

@FANNG1 fixed

Thx, just minor comments, PTAL

@FANNG1 fixed, but you need to help me retry the CI, thanks

could you update the version from 1.1 to 1.2?

  public static final ConfigEntry<String> SCAN_PLAN_CACHE_IMPL =
      new ConfigBuilder(IcebergConstants.SCAN_PLAN_CACHE_IMPL)
          .doc("The implementation of the scan plan cache")
          .version(ConfigConstants.VERSION_1_1_0)

[FANNG1]

@sunxiaojian do you have time to take a look at the comment? If not, could I add a commit based on your work?

[sunxiaojian]

@sunxiaojian do you have time to take a look at the comment? If not, could I add a commit based on your work?

fixed

[FANNG1]

@sunxiaojian , merged to main, thanks for your effort!