[improve][broker] PIP-473: transaction metadata data layer

[merlimat]

Summary

Foundation data layer for PIP-473 (metadata-driven transactions). Introduces typed records + path/index constants + a TxnMetadataStore façade over MetadataStore, ready for the v5 TC to plug into in a follow-up.

• New records under pulsar-broker/.../transaction/metadata/:
  • TxnHeader — header at /txn/<txnId>, the CAS linearization point for endTxn
  • TxnOp — op-log entry written under /txn-op/<txnId> with SequenceKeysDeltas
  • TxnEvent — per-segment / per-(segment, sub) notification event
  • Versioned<T> — (value, version) pair returned from reads so callers can CAS
  • TxnState, TxnOpKind — Jackson-friendly enums (serialized by name)
• TxnPaths — path templates, index names, key formatters
• TxnMetadataStore façade — header CRUD with CAS, op append with kind-specific secondary indexes, deadline / final-state range queries (via scanByIndex), and segment / subscription event publish + subscribe (via subscribeSequence)
• Bundled metadata-layer fix: the __seq_counter__ sidecar that backs SequenceKeysDeltas on non-native stores was leaking through scanChildren and the scanByIndex fallback. Filter it out in AbstractMetadataStore's defaults plus the Memory and RocksDB native overrides. Oxia uses native sequence keys (no sidecar) and is unaffected.

No runtime path consumes this yet — the v5 TC PR will wire it up.

Test plan

• pulsar-broker:test --tests TxnMetadataStoreTest — 7 tests on Memory: header CAS lifecycle, write/ack op append + index scan, deadline range scan (filters terminal + out-of-range), final-state range scan, segment + subscription event publish + subscribe
• pulsar-metadata:test --tests MetadataStoreScanChildrenTest — added skipsSequenceCounterSidecars across ZooKeeper / Memory / RocksDB / Oxia / MockZooKeeper
• pulsar-metadata:test --tests SequenceKeysTest + MetadataStoreSecondaryIndexTest — pre-existing suites still green after the abstract-layer change
• Checkstyle clean on pulsar-broker + pulsar-metadata

[lhotari]

Claude Code review findings to check:

  1. [INTENT MISMATCH] Raw segment URIs are embedded into MetadataStore paths without encoding — TxnPaths.segmentEventsParent / subscriptionEventsParent in
  pulsar-broker/.../transaction/metadata/TxnPaths.java:90-102
    - PIP-473 segment names take the form segment://tenant/ns/topic/<hexStart>-<hexEnd>-<segmentId> (see existing comment at MLPendingAckStore.java:534). Concatenated into a metadata path,
  that produces e.g. /txn-segment-events/segment://tenant/ns/topic/... — which contains // (empty path component) and :.
    - ZooKeeper's PathUtils.validatePath rejects this; Oxia, while not slash-validating, would interpret the embedded / as path levels and break the sidecar / sequence-key semantics.
    - The existing convention in transaction code (MLPendingAckStore.java:533, TopicTransactionBuffer.java:868) is Codec.encode(...). This data layer breaks that convention without saying so.
    - Recommended: Either encode segment (and subscription) in TxnPaths helpers, or add a @throws/precondition comment documenting that callers must pre-encode. Currently the tests use
  "segment://A" only because Memory happens to tolerate it — that's a misleading green light for the v5 TC PR.
  2. [BUG] isSequenceCounterChild filter uses a permissive endsWith predicate — pulsar-metadata/.../AbstractMetadataStore.java:608-612, 651-655, 776-782
    - A user record whose final path segment ends with the literal __seq_counter__ would be silently filtered from scanChildren / scanByIndex. Low likelihood in practice, but easy to harden.
    - Since sequenceCounterPath(prefix) = prefix + "__seq_counter__" is always a sibling of the prefix at the same parent level, the sidecar name is always <prefix-leaf>__seq_counter__. A
  stricter check would require the sidecar to be co-located with a known prefix, or at minimum scope by an internal marker char not allowed in user keys.
    - Acceptable as-is if you're confident no user key will collide; worth a one-line comment justifying the looseness.

[lhotari]

LGTM, just check the local Claude Code review findings I added a comment