[feature](fe) Add byte-weighted external metadata caches

[CalvinKirs]

What problem does this PR solve?

Issue Number: None

Related PR: #66533

Problem Summary:

External connector metadata caches are bounded only by entry count. A small number of large file listings, partition maps, table metadata objects, or manifest values can therefore consume excessive FE heap while remaining below the configured entry capacity.

This change adds an optional byte-weighted cache path. When max-weight is configured, the cache requires a typed estimator and uses Caffeine maximumWeight; otherwise it preserves the existing maximumSize(capacity) behavior.

The production integrations cover:

• Hive file-listing values.
• Iceberg table, partition, and manifest values.
• Paimon derived partition-view values.

Large values precompute their estimated retained bytes when they are constructed, so the Caffeine weighing callback is O(1). Paimon's legacy count-bounded path does not build or traverse the weighted wrapper, so its default behavior and estimation cost remain unchanged.

Opt-in JMH/JOL benchmarks compare cached weight lookup, construction-time estimation, and JOL retained-graph traversal. The Paimon benchmark covers 10,000 and 100,000 partitions.

Release note

External metadata caches can opt into byte-based limits with meta.cache.<engine>.<entry>.max-weight properties. Paimon partition views use meta.cache.paimon.partition_view.max-weight.

Check List (For Author)

• Test

  • Regression test
  • Unit Test
  • Manual test (add detailed scripts or steps below)
  • No need to test or manual test. Explain why:
    • This is a refactor/code format and no logic has been changed.
    • Previous test can cover this change.
    • No code files have been changed.
    • Other reason

  Unit tests:

  • ./run-fe-ut.sh --run org.apache.doris.common.util.ParseUtilTest,org.apache.doris.common.CacheFactoryTest,org.apache.doris.datasource.metacache.CacheSpecTest,org.apache.doris.datasource.metacache.MetaCacheEntryTest,org.apache.doris.datasource.metacache.AbstractExternalMetaCacheTest
  • ./run-fe-ut.sh --run org.apache.doris.connector.cache.ConnectorMetadataCacheTest,org.apache.doris.connector.paimon.PaimonPartitionViewSizeEstimatorTest,org.apache.doris.connector.paimon.PaimonConnectorValidatePropertiesTest

  Build and benchmark compilation:

  • DISABLE_BUILD_UI=ON ./build.sh --fe
  • mvn -Pbenchmark -pl fe-benchmark -am test-compile -DskipTests

  The JMH benchmarks were compiled but not executed in this PR update.

• Behavior changed:

  • No.
  • Yes. Configured caches can use byte-weighted eviction; count-based capacity remains the default.

• Does this need documentation?

  • No.
  • Yes.

Check List (For Reviewer who merge this PR)

• Confirm the release note
• Confirm test cases
• Confirm document
• Add branch pick label

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?