Search before reporting
Read release policy
User environment
- Broker version:
5.0.0-M1-SNAPSHOT, built from master at commit af54f64e18
- Broker OS / hardware:
Linux 6.8.0-84-generic #84-Ubuntu SMP PREEMPT_DYNAMIC x86_64 (Ubuntu 24.04)
- Broker Java version:
openjdk 21.0.12 2026-07-21 (OpenJDK 64-Bit Server VM, 21.0.12+8-1-24.04-Ubuntu)
- Client library type: Java (
pulsar-admin / PulsarAdmin)
- Client library version: same build,
5.0.0-M1-SNAPSHOT
- Client OS / hardware and Java version: as above, same machine
This is a code-level defect in the broker rather than an environment-specific one — it reproduces on any broker whose replicated cluster name contains a ., independently of OS, JVM or client language, since the affected parsing is broker-side. It is confirmed by a broker test (see Reproducing the issue).
Issue Description
Cluster names are allowed to contain . — NamedEntity.NAMED_ENTITY_PATTERN is ^[-=:.\w]*$, and the comment above it states this applies to "property, namespace, cluster and topic names".
AbstractReplicator builds replicator cursor/subscription names as <replicatorPrefix>.<remoteCluster> in getReplicatorName, but recovers the cluster by splitting on . and taking the last segment (pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractReplicator.java:474):
public static String getRemoteCluster(String remoteCursor) {
String[] split = remoteCursor.split("\\.");
return split[split.length - 1];
}
The two are therefore not inverses. For a cluster named remote.east, the cursor pulsar.repl.remote.east resolves to east.
Expected: the cluster name recovered from a replicator cursor is the one that was used to build it, so replicator subscriptions behave the same whether or not the cluster name contains a dot.
Actual: two distinct failures.
1. Every admin operation addressed at a replicator subscription fails.
Six call sites use this parse, then look the replicator up under the wrong cluster name and get null (a seventh, the orphan-cursor sweep, is covered below):
| Call site |
Operation |
Result |
PersistentTopicsBase.java:1982 |
clear-backlog on a replicator subscription |
404 Replicator not found |
PersistentTopicsBase.java:2037 |
skip messages |
404 Replicator not found |
PersistentTopicsBase.java:4213 |
expire messages (by position) |
404 Replicator not found |
PersistentTopicsBase.java:4329 |
expire messages (by timestamp) |
404 Replicator not found |
PersistentTopicsBase.java:4767 |
getReplicatorReference — peek, replicator stats |
404 Replicator not found |
NamespacesBase.java:2304 |
namespace-wide clear-backlog for a replicator subscription |
wrong subscription targeted |
Each of these guards with subName.startsWith(replicatorPrefix) on the line immediately above, so the subscription is recognised as a replicator's — only the cluster it belongs to is derived wrongly.
2. The orphan-cursor sweep misidentifies live replicators.
PersistentTopic.removeOrphanReplicationCursors() (PersistentTopic.java:559) runs inside initialize(), i.e. on every topic load:
getRemoteCluster("pulsar.repl.remote.east") returns east (PersistentTopic.java:564).
east is not among the topic's replication clusters, so a live, correctly-configured replicator is declared orphaned and removeReplicator("east") is called (PersistentTopic.java:569).
removeReplicator rebuilds the name as pulsar.repl.east and calls asyncDeleteCursor on it (PersistentTopic.java:2496-2500). No such cursor exists, so ManagedLedgerImpl.java:1092-1095 fails the callback with CursorNotFoundException, and the initialize() chain fails at PersistentTopic.java:509.
The cursor survives only because the reconstructed name is wrong too — the sweep fully intends to delete a cursor that is not orphaned. #22890 documents that wrongly removing a replicator cursor loses the entire replication backlog, so this sits on a code path already known to be destructive when it misfires. The sweep was introduced by #19972 / #22890.
I believe this is a bug rather than intended behaviour because getReplicatorName and getRemoteCluster are documented and used as inverses of one another, and dots are explicitly legal in cluster names.
Error messages
Logged on every load of an affected topic:
WARN org.apache.pulsar.broker.service.persistent.PersistentTopic - Remove the orphan replicator because the cluster does not exist remoteCluster=east
ERROR org.apache.pulsar.broker.service.persistent.PersistentTopic - Failed to delete cursor name=pulsar.repl.east
org.apache.bookkeeper.mledger.ManagedLedgerException$CursorNotFoundException: ManagedCursor not found: pulsar.repl.east
Admin side, for a replicator subscription that plainly exists in topics stats:
Replicator not found
Reason: HTTP 404 Not Found
Reproducing the issue
Any cluster whose name contains a . and participates in geo-replication reproduces this. It reproduces on Pulsar standalone.
- Register a cluster with a dot in its name, e.g.
remote.east, and add it to the tenant's allowed clusters.
- Set it as a replication cluster on a namespace, and create a topic there.
- Confirm the replicator exists —
pulsar-admin topics stats persistent://tenant/ns/topic lists a remote.east replicator, and pulsar-admin topics subscriptions ... shows pulsar.repl.remote.east.
- Address any admin operation at it:
pulsar-admin topics skip -s pulsar.repl.remote.east -n 1 persistent://tenant/ns/topic
Observed: Replicator not found (HTTP 404).
Expected: the operation applies to the remote.east replicator.
Reloading the topic (unload, or restart the broker) additionally logs the orphan-sweep warning and the failed cursor deletion shown above.
The same steps with a cluster named without a dot, e.g. remote, all succeed — which isolates the cause to the name parsing.
The orphan-sweep half also reproduces as a broker test: create a topic with a live replicator cursor pulsar.repl.remote.east for cluster remote.east, call PersistentTopic#initialize(), and observe the Remove the orphan replicator warning for remoteCluster=east.
Additional information
The fix is to make getRemoteCluster the exact inverse of getReplicatorName by stripping the known replicatorPrefix rather than splitting on .. All seven call sites already guard with startsWith(replicatorPrefix) immediately above, so the prefix is in scope at every one and no plumbing is needed beyond passing it in.
Fixing the parsing is preferable to rejecting dotted cluster names: deployments may already use them, and turning those into a validation error would break working clusters on upgrade.
Per pip/README.md, this is a bug-fix and should not need a PIP: no public API, configuration, metric, or wire-format change, and no *ClassName configuration selects a Replicator implementation, so AbstractReplicator is not a plugin/SPI extension point.
Related: #22890 and #19972 introduced the orphan-cursor sweep.
Are you willing to submit a PR?
Search before reporting
Read release policy
User environment
5.0.0-M1-SNAPSHOT, built frommasterat commitaf54f64e18Linux 6.8.0-84-generic #84-Ubuntu SMP PREEMPT_DYNAMIC x86_64(Ubuntu 24.04)openjdk 21.0.12 2026-07-21(OpenJDK 64-Bit Server VM, 21.0.12+8-1-24.04-Ubuntu)pulsar-admin/PulsarAdmin)5.0.0-M1-SNAPSHOTThis is a code-level defect in the broker rather than an environment-specific one — it reproduces on any broker whose replicated cluster name contains a
., independently of OS, JVM or client language, since the affected parsing is broker-side. It is confirmed by a broker test (see Reproducing the issue).Issue Description
Cluster names are allowed to contain
.—NamedEntity.NAMED_ENTITY_PATTERNis^[-=:.\w]*$, and the comment above it states this applies to "property, namespace, cluster and topic names".AbstractReplicatorbuilds replicator cursor/subscription names as<replicatorPrefix>.<remoteCluster>ingetReplicatorName, but recovers the cluster by splitting on.and taking the last segment (pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractReplicator.java:474):The two are therefore not inverses. For a cluster named
remote.east, the cursorpulsar.repl.remote.eastresolves toeast.Expected: the cluster name recovered from a replicator cursor is the one that was used to build it, so replicator subscriptions behave the same whether or not the cluster name contains a dot.
Actual: two distinct failures.
1. Every admin operation addressed at a replicator subscription fails.
Six call sites use this parse, then look the replicator up under the wrong cluster name and get
null(a seventh, the orphan-cursor sweep, is covered below):PersistentTopicsBase.java:1982Replicator not foundPersistentTopicsBase.java:2037Replicator not foundPersistentTopicsBase.java:4213Replicator not foundPersistentTopicsBase.java:4329Replicator not foundPersistentTopicsBase.java:4767getReplicatorReference— peek, replicator statsReplicator not foundNamespacesBase.java:2304Each of these guards with
subName.startsWith(replicatorPrefix)on the line immediately above, so the subscription is recognised as a replicator's — only the cluster it belongs to is derived wrongly.2. The orphan-cursor sweep misidentifies live replicators.
PersistentTopic.removeOrphanReplicationCursors()(PersistentTopic.java:559) runs insideinitialize(), i.e. on every topic load:getRemoteCluster("pulsar.repl.remote.east")returnseast(PersistentTopic.java:564).eastis not among the topic's replication clusters, so a live, correctly-configured replicator is declared orphaned andremoveReplicator("east")is called (PersistentTopic.java:569).removeReplicatorrebuilds the name aspulsar.repl.eastand callsasyncDeleteCursoron it (PersistentTopic.java:2496-2500). No such cursor exists, soManagedLedgerImpl.java:1092-1095fails the callback withCursorNotFoundException, and theinitialize()chain fails atPersistentTopic.java:509.The cursor survives only because the reconstructed name is wrong too — the sweep fully intends to delete a cursor that is not orphaned. #22890 documents that wrongly removing a replicator cursor loses the entire replication backlog, so this sits on a code path already known to be destructive when it misfires. The sweep was introduced by #19972 / #22890.
I believe this is a bug rather than intended behaviour because
getReplicatorNameandgetRemoteClusterare documented and used as inverses of one another, and dots are explicitly legal in cluster names.Error messages
Logged on every load of an affected topic:
Admin side, for a replicator subscription that plainly exists in
topics stats:Reproducing the issue
Any cluster whose name contains a
.and participates in geo-replication reproduces this. It reproduces on Pulsar standalone.remote.east, and add it to the tenant's allowed clusters.pulsar-admin topics stats persistent://tenant/ns/topiclists aremote.eastreplicator, andpulsar-admin topics subscriptions ...showspulsar.repl.remote.east.Observed:
Replicator not found(HTTP 404).Expected: the operation applies to the
remote.eastreplicator.Reloading the topic (unload, or restart the broker) additionally logs the orphan-sweep warning and the failed cursor deletion shown above.
The same steps with a cluster named without a dot, e.g.
remote, all succeed — which isolates the cause to the name parsing.The orphan-sweep half also reproduces as a broker test: create a topic with a live replicator cursor
pulsar.repl.remote.eastfor clusterremote.east, callPersistentTopic#initialize(), and observe theRemove the orphan replicatorwarning forremoteCluster=east.Additional information
The fix is to make
getRemoteClusterthe exact inverse ofgetReplicatorNameby stripping the knownreplicatorPrefixrather than splitting on.. All seven call sites already guard withstartsWith(replicatorPrefix)immediately above, so the prefix is in scope at every one and no plumbing is needed beyond passing it in.Fixing the parsing is preferable to rejecting dotted cluster names: deployments may already use them, and turning those into a validation error would break working clusters on upgrade.
Per
pip/README.md, this is a bug-fix and should not need a PIP: no public API, configuration, metric, or wire-format change, and no*ClassNameconfiguration selects aReplicatorimplementation, soAbstractReplicatoris not a plugin/SPI extension point.Related: #22890 and #19972 introduced the orphan-cursor sweep.
Are you willing to submit a PR?