Skip to content

[fix][broker] Use effective offload policies for extra configs#25781

Open
coderzc wants to merge 4 commits into
apache:masterfrom
coderzc:codex/fix-offload-extra-config-merge-2
Open

[fix][broker] Use effective offload policies for extra configs#25781
coderzc wants to merge 4 commits into
apache:masterfrom
coderzc:codex/fix-offload-extra-config-merge-2

Conversation

@coderzc
Copy link
Copy Markdown
Member

@coderzc coderzc commented May 15, 2026

Motivation

#25736 made broker-level managedLedgerOffloadExtraConfig* entries available to OffloadPoliciesImpl.mergeConfiguration(...). However, managedLedgerExtraConfigurations defaults to an empty map, so a namespace or topic offload policy with no extra config still stops fallback to broker-level extra config.

This can produce runtime offload policies that inherit the broker driver/bucket/region but lose broker-level extra config such as a tiered-storage bucket prefix. The namespace offloaded-ledger scanner also used raw namespace offload policies directly instead of the effective merged policy used by topic loading.

Modifications

  • Merge managedLedgerExtraConfigurations as a map from broker -> namespace -> topic.
  • Treat null or empty higher-level extra config maps as unset so broker-level entries are inherited.
  • Preserve higher-level key overrides, including an explicit empty-string value to clear a broker-level key.
  • Use effective merged namespace offload policies when scanning offloaded ledgers.
  • Add regression coverage for empty namespace-level extra config inheriting broker-level extra config and for scan using the merged policy.

Verifying this change

  • ./gradlew :pulsar-common:test --tests org.apache.pulsar.common.policies.data.OffloadPoliciesTest
  • ./gradlew :pulsar-broker:test --tests org.apache.pulsar.broker.admin.AdminApiOffloadTest.testScanOffloadedLedgersUsesMergedNamespaceOffloadPolicies

@coderzc coderzc changed the title [fix][broker] Inherit broker offload extra configs from empty policies [fix][broker] Use effective offload policies for extra configs May 15, 2026
@coderzc coderzc added type/bug The PR fixed a bug or issue reported a bug area/broker labels May 15, 2026
@coderzc coderzc self-assigned this May 15, 2026
@coderzc coderzc requested a review from codelipenghui May 15, 2026 10:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/broker release/4.0.11 release/4.2.2 type/bug The PR fixed a bug or issue reported a bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant