Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Broker] Fix NPE in PersistentTopic.checkSubscriptionTypesEnable #12961

Merged
merged 1 commit into from
Nov 26, 2021

Conversation

RobertIndie
Copy link
Member

@RobertIndie RobertIndie commented Nov 24, 2021

Motivation

The NPE may appear in PersistentTopic.checkSubscriptionTypesEnable. The topicPolicies.getSubscriptionTypesEnabled() may be null.

The problem occurred when upgrading from pulsar version 2.7 to 2.8. We added SubscriptionTypesEnable in 2.8: #9401. When we initialize a topic's topicPolicy in pulasr 2.7, and then get the topic's topicPolicy in version 2.8, the SubscriptionTypesEnable will be null. This leads to the problem.

Here are steps to reproduce:

  • Start broker 2.7
  • Create a topic and init the topic policy
  • Upgrade broker to 2.8
  • We will get the SubscriptionTypesEnable with the value of null
➜  pulsar-281 bin/pulsar-admin topics get-subscription-types-enabled  my-topic
null
  • When we consume from that topic, the issue occurs
Caused by: java.lang.NullPointerException
	at org.apache.pulsar.broker.service.persistent.PersistentTopic.checkSubscriptionTypesEnable(PersistentTopic.java:3206) ~[io.streamnative-pulsar-broker-2.8.1.21.jar:2.8.1.21]
	at org.apache.pulsar.broker.service.persistent.PersistentTopic.subscribe(PersistentTopic.java:688) ~[io.streamnative-pulsar-broker-2.8.1.21.jar:2.8.1.21]
	at org.apache.pulsar.broker.service.ServerCnx.lambda$null$13(ServerCnx.java:1029) ~[io.streamnative-pulsar-broker-2.8.1.21.jar:2.8.1.21]
	at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1106) ~[?:?]
	... 31 more

Modifications

  • Use CollectionUtils.isEmpty to determine if the list is empty to fix the problem of throwing NPE.

Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API: (no)
  • The schema: (no)
  • The default values of configurations: (no)
  • The wire protocol: (no)
  • The rest endpoints: (no)
  • The admin cli options: (no)
  • Anything that affects deployment: (no)

Documentation

  • no-need-doc

This is just a small bug fix.

Signed-off-by: Zike Yang <ar@armail.top>
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Nov 24, 2021
@@ -3218,7 +3219,7 @@ public boolean checkSubscriptionTypesEnable(SubType subType) throws Exception {
if (topicPolicies == null) {
return checkNsAndBrokerSubscriptionTypesEnable(topicName, subType);
} else {
if (topicPolicies.getSubscriptionTypesEnabled().isEmpty()) {
if (CollectionUtils.isEmpty(topicPolicies.getSubscriptionTypesEnabled())) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

topicPolicies.getSubscriptionTypesEnabled() default value is empty list, when can it become null?

@Builder.Default
private List<SubType> subscriptionTypesEnabled = new ArrayList<>();

In addition, it is recommended to combine the two judgments as below:

if (topicPolicies == null || CollectionUtils.isEmpty(topicPolicies.getSubscriptionTypesEnabled())) {
    return checkNsAndBrokerSubscriptionTypesEnable(topicName, subType);
} else {
    return topicPolicies.getSubscriptionTypesEnabled().contains(subType);
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

topicPolicies.getSubscriptionTypesEnabled() default value is empty list.
+1

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem occurred when upgrading from pulsar version 2.7 to 2.8. I have added the root cause of the problem and the reproduce steps to the PR description. Please check. @yuruguo @gaozhangmin

In addition, it is recommended to combine the two judgments as below:

if (topicPolicies == null || CollectionUtils.isEmpty(topicPolicies.getSubscriptionTypesEnabled())) {
    return checkNsAndBrokerSubscriptionTypesEnable(topicName, subType);
} else {
    return topicPolicies.getSubscriptionTypesEnabled().contains(subType);
}

@yuruguo I don't think we need to. The null check of topicPolicy has been done before.

Copy link
Contributor

@codelipenghui codelipenghui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@RobertIndie Could you please add a test to avoid the regression?

@RobertIndie
Copy link
Member Author

@RobertIndie Could you please add a test to avoid the regression?

The problem occurs between different versions of brokers and it seems difficult to add tests.

@codelipenghui codelipenghui merged commit 877bf3a into apache:master Nov 26, 2021
codelipenghui pushed a commit that referenced this pull request Nov 26, 2021
The NPE may appear in `PersistentTopic.checkSubscriptionTypesEnable`.   The `topicPolicies.getSubscriptionTypesEnabled()` may be null.

The problem occurred when upgrading from pulsar version 2.7 to 2.8. We added `SubscriptionTypesEnable` in 2.8: #9401. When we initialize a topic's topicPolicy in pulasr 2.7, and then get the topic's topicPolicy in version 2.8, the `SubscriptionTypesEnable` will be null. This leads to the problem.

Here are steps to reproduce:
* Start broker 2.7
* Create a topic and init the topic policy
* Upgrade broker to 2.8
* We will get the `SubscriptionTypesEnable` with the value of null
```sh
➜  pulsar-281 bin/pulsar-admin topics get-subscription-types-enabled  my-topic
null
```
* When we consume from that topic, the issue occurs
```
Caused by: java.lang.NullPointerException
	at org.apache.pulsar.broker.service.persistent.PersistentTopic.checkSubscriptionTypesEnable(PersistentTopic.java:3206) ~[io.streamnative-pulsar-broker-2.8.1.21.jar:2.8.1.21]
	at org.apache.pulsar.broker.service.persistent.PersistentTopic.subscribe(PersistentTopic.java:688) ~[io.streamnative-pulsar-broker-2.8.1.21.jar:2.8.1.21]
	at org.apache.pulsar.broker.service.ServerCnx.lambda$null$13(ServerCnx.java:1029) ~[io.streamnative-pulsar-broker-2.8.1.21.jar:2.8.1.21]
	at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1106) ~[?:?]
	... 31 more
```

* Use `CollectionUtils.isEmpty` to determine if the list is empty to fix the problem of throwing NPE.

(cherry picked from commit 877bf3a)
@codelipenghui codelipenghui added the cherry-picked/branch-2.8 Archived: 2.8 is end of life label Nov 26, 2021
eolivelli pushed a commit to eolivelli/pulsar that referenced this pull request Nov 29, 2021
### Motivation

The NPE may appear in `PersistentTopic.checkSubscriptionTypesEnable`.   The `topicPolicies.getSubscriptionTypesEnabled()` may be null.

The problem occurred when upgrading from pulsar version 2.7 to 2.8. We added `SubscriptionTypesEnable` in 2.8: apache#9401. When we initialize a topic's topicPolicy in pulasr 2.7, and then get the topic's topicPolicy in version 2.8, the `SubscriptionTypesEnable` will be null. This leads to the problem.

Here are steps to reproduce:
* Start broker 2.7
* Create a topic and init the topic policy
* Upgrade broker to 2.8
* We will get the `SubscriptionTypesEnable` with the value of null
```sh
➜  pulsar-281 bin/pulsar-admin topics get-subscription-types-enabled  my-topic
null
```
* When we consume from that topic, the issue occurs
```
Caused by: java.lang.NullPointerException
	at org.apache.pulsar.broker.service.persistent.PersistentTopic.checkSubscriptionTypesEnable(PersistentTopic.java:3206) ~[io.streamnative-pulsar-broker-2.8.1.21.jar:2.8.1.21]
	at org.apache.pulsar.broker.service.persistent.PersistentTopic.subscribe(PersistentTopic.java:688) ~[io.streamnative-pulsar-broker-2.8.1.21.jar:2.8.1.21]
	at org.apache.pulsar.broker.service.ServerCnx.lambda$null$13(ServerCnx.java:1029) ~[io.streamnative-pulsar-broker-2.8.1.21.jar:2.8.1.21]
	at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1106) ~[?:?]
	... 31 more
```

### Modifications

* Use `CollectionUtils.isEmpty` to determine if the list is empty to fix the problem of throwing NPE.
fxbing pushed a commit to fxbing/pulsar that referenced this pull request Dec 19, 2021
### Motivation

The NPE may appear in `PersistentTopic.checkSubscriptionTypesEnable`.   The `topicPolicies.getSubscriptionTypesEnabled()` may be null.

The problem occurred when upgrading from pulsar version 2.7 to 2.8. We added `SubscriptionTypesEnable` in 2.8: apache#9401. When we initialize a topic's topicPolicy in pulasr 2.7, and then get the topic's topicPolicy in version 2.8, the `SubscriptionTypesEnable` will be null. This leads to the problem.

Here are steps to reproduce:
* Start broker 2.7
* Create a topic and init the topic policy
* Upgrade broker to 2.8
* We will get the `SubscriptionTypesEnable` with the value of null
```sh
➜  pulsar-281 bin/pulsar-admin topics get-subscription-types-enabled  my-topic
null
```
* When we consume from that topic, the issue occurs
```
Caused by: java.lang.NullPointerException
	at org.apache.pulsar.broker.service.persistent.PersistentTopic.checkSubscriptionTypesEnable(PersistentTopic.java:3206) ~[io.streamnative-pulsar-broker-2.8.1.21.jar:2.8.1.21]
	at org.apache.pulsar.broker.service.persistent.PersistentTopic.subscribe(PersistentTopic.java:688) ~[io.streamnative-pulsar-broker-2.8.1.21.jar:2.8.1.21]
	at org.apache.pulsar.broker.service.ServerCnx.lambda$null$13(ServerCnx.java:1029) ~[io.streamnative-pulsar-broker-2.8.1.21.jar:2.8.1.21]
	at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1106) ~[?:?]
	... 31 more
```

### Modifications

* Use `CollectionUtils.isEmpty` to determine if the list is empty to fix the problem of throwing NPE.
codelipenghui pushed a commit that referenced this pull request Dec 21, 2021
The NPE may appear in `PersistentTopic.checkSubscriptionTypesEnable`.   The `topicPolicies.getSubscriptionTypesEnabled()` may be null.

The problem occurred when upgrading from pulsar version 2.7 to 2.8. We added `SubscriptionTypesEnable` in 2.8: #9401. When we initialize a topic's topicPolicy in pulasr 2.7, and then get the topic's topicPolicy in version 2.8, the `SubscriptionTypesEnable` will be null. This leads to the problem.

Here are steps to reproduce:
* Start broker 2.7
* Create a topic and init the topic policy
* Upgrade broker to 2.8
* We will get the `SubscriptionTypesEnable` with the value of null
```sh
➜  pulsar-281 bin/pulsar-admin topics get-subscription-types-enabled  my-topic
null
```
* When we consume from that topic, the issue occurs
```
Caused by: java.lang.NullPointerException
	at org.apache.pulsar.broker.service.persistent.PersistentTopic.checkSubscriptionTypesEnable(PersistentTopic.java:3206) ~[io.streamnative-pulsar-broker-2.8.1.21.jar:2.8.1.21]
	at org.apache.pulsar.broker.service.persistent.PersistentTopic.subscribe(PersistentTopic.java:688) ~[io.streamnative-pulsar-broker-2.8.1.21.jar:2.8.1.21]
	at org.apache.pulsar.broker.service.ServerCnx.lambda$null$13(ServerCnx.java:1029) ~[io.streamnative-pulsar-broker-2.8.1.21.jar:2.8.1.21]
	at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:1106) ~[?:?]
	... 31 more
```

* Use `CollectionUtils.isEmpty` to determine if the list is empty to fix the problem of throwing NPE.

(cherry picked from commit 877bf3a)
@codelipenghui codelipenghui added the cherry-picked/branch-2.9 Archived: 2.9 is end of life label Dec 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/broker cherry-picked/branch-2.8 Archived: 2.8 is end of life cherry-picked/branch-2.9 Archived: 2.9 is end of life doc-not-needed Your PR changes do not impact docs release/2.8.3 release/2.9.2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants