Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix panic on discard message for Go Functions #8776

Merged
merged 8 commits into from Dec 3, 2020

Conversation

wolfstudy
Copy link
Member

@wolfstudy wolfstudy commented Dec 2, 2020

Signed-off-by: xiaolong.ran rxl@apache.org

Fixes #8769

Motivation

When we use pulsar-admin or pulsarctl to create Go Functions, we need to specify the input topic, but in many cases, we will directly specify the topic name, for example: input-topic instead of: persistent://publlic/default/input-topic.

In the consumer of Go Function, we have the following map structure to map the relationship between topic and consumer, as follows:

consumers         map[string]pulsar.Consumer

When assigning a value to the map structure, we use the value directly passed in from pulsar-admin or pulsarctl as the key of the map, but when using it, we use the full path form (eg:persistent:/ /publlic/default/input-topic), so the corresponding key cannot be found, resulting in the following panic:

2020/12/02 11:38:47.845  [info] Created producer cnx=127.0.0.1:54594 -> 127.0.0.1:6650 topic=persistent://public/default/out-topic producer_name=standalone-0-61 producerID=1
===============topic name================
topic name: in-topic
11:38:47.846 [pulsar-io-51-6] INFO  org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:54594] Subscribing on topic persistent://public/default/in-topic / public/default/go_func
11:38:47.847 [pulsar-io-51-6] INFO  org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/in-topic-public%2Fdefault%2Fgo_func] Rewind from 13:10 to 13:0
11:38:47.847 [pulsar-io-51-6] INFO  org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/default/in-topic] There are no replicated subscriptions on the topic
11:38:47.847 [pulsar-io-51-6] INFO  org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/default/in-topic][public/default/go_func] Created new subscription for 1
11:38:47.847 [pulsar-io-51-6] INFO  org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:54594] Created subscription on topic persistent://public/default/in-topic / public/default/go_func
2020/12/02 11:38:47.847 asm_amd64.s:1373: [info] Connected consumer subscription=public/default/go_func consumerID=1 name=jrppw topic=persistent://public/default/in-topic
2020/12/02 11:38:47.847 asm_amd64.s:1373: [info] Created consumer name=jrppw topic=persistent://public/default/in-topic subscription=public/default/go_func consumerID=1
2020/12/02 11:38:47.847 log.go:46: [info] Serving InstanceCommunication on port 54013
2020/12/02 11:38:47.848 instance.go:402: [error] the logAppender is nil, if you want to use it, please specify `--log-topic` at startup.
2020/12/02 11:38:47.848 contextFunc.go:30: [info] attempting to discard input
++++++++++++++++++++
ack input msg topic: persistent://public/default/in-topic
++++++++++++++++++++++++
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x46a1547]

goroutine 1 [running]:
github.com/apache/pulsar/pulsar-function-go/pf.(*goInstance).ackInputMessage(0xc000248000, 0x4a3d420, 0xc00040e000)
	/Users/wolf4j/github.com/apache/pulsar/pulsar-function-go/pf/instance.go:367 +0x1d7
github.com/apache/pulsar/pulsar-function-go/pf.(*goInstance).processResult(0xc000248000, 0x4a3d420, 0xc00040e000, 0x0, 0x0, 0x0)
	/Users/wolf4j/github.com/apache/pulsar/pulsar-function-go/pf/instance.go:355 +0x80
github.com/apache/pulsar/pulsar-function-go/pf.(*goInstance).startFunction(0xc000248000, 0x4a23740, 0xc00020e0c0, 0x0, 0x0)
	/Users/wolf4j/github.com/apache/pulsar/pulsar-function-go/pf/instance.go:177 +0x584
github.com/apache/pulsar/pulsar-function-go/pf.Start(0x4818b00, 0x4938eb0)
	/Users/wolf4j/github.com/apache/pulsar/pulsar-function-go/pf/function.go:171 +0x6b
main.main()
	/Users/wolf4j/github.com/apache/pulsar/pulsar-function-go/examples/contextFunc/contextFunc.go:35 +0x39
11:38:47.855 [pulsar-io-51-6] INFO  org.apache.pulsar.broker.service.ServerCnx - Closed connection from /127.0.0.1:54594

Modifications

In this pull request, we define a TopicName structure to parse the topic name passed in by pulsar-admin or pulsarctl. If it is the default, use the full path method for splicing to ensure that the map is used key is the same.

Signed-off-by: xiaolong.ran <rxl@apache.org>
@flowchartsman
Copy link
Contributor

LGTM

Signed-off-by: xiaolong.ran <rxl@apache.org>
Signed-off-by: xiaolong.ran <rxl@apache.org>
@wolfstudy
Copy link
Member Author

/pulsarbot run-failure-checks

@wolfstudy wolfstudy merged commit f9ad058 into apache:master Dec 3, 2020
RobertIndie pushed a commit to RobertIndie/pulsar that referenced this pull request Dec 8, 2020
Signed-off-by: xiaolong.ran <rxl@apache.org>


Fixes apache#8769

### Motivation

When we use pulsar-admin or pulsarctl to create Go Functions, we need to specify the input topic, but in many cases, we will directly specify the topic name, for example: `input-topic` instead of: `persistent://publlic/default/input-topic`.

In the consumer of Go Function, we have the following map structure to map the relationship between topic and consumer, as follows:

```
consumers         map[string]pulsar.Consumer
```

When assigning a value to the map structure, we use the value directly passed in from pulsar-admin or pulsarctl as the key of the map, but when using it, we use the full path form (eg:persistent:/ /publlic/default/input-topic), so the corresponding key cannot be found, resulting in the following panic:

```
2020/12/02 11:38:47.845  [info] Created producer cnx=127.0.0.1:54594 -> 127.0.0.1:6650 topic=persistent://public/default/out-topic producer_name=standalone-0-61 producerID=1
===============topic name================
topic name: in-topic
11:38:47.846 [pulsar-io-51-6] INFO  org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:54594] Subscribing on topic persistent://public/default/in-topic / public/default/go_func
11:38:47.847 [pulsar-io-51-6] INFO  org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/in-topic-public%2Fdefault%2Fgo_func] Rewind from 13:10 to 13:0
11:38:47.847 [pulsar-io-51-6] INFO  org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/default/in-topic] There are no replicated subscriptions on the topic
11:38:47.847 [pulsar-io-51-6] INFO  org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://public/default/in-topic][public/default/go_func] Created new subscription for 1
11:38:47.847 [pulsar-io-51-6] INFO  org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:54594] Created subscription on topic persistent://public/default/in-topic / public/default/go_func
2020/12/02 11:38:47.847 asm_amd64.s:1373: [info] Connected consumer subscription=public/default/go_func consumerID=1 name=jrppw topic=persistent://public/default/in-topic
2020/12/02 11:38:47.847 asm_amd64.s:1373: [info] Created consumer name=jrppw topic=persistent://public/default/in-topic subscription=public/default/go_func consumerID=1
2020/12/02 11:38:47.847 log.go:46: [info] Serving InstanceCommunication on port 54013
2020/12/02 11:38:47.848 instance.go:402: [error] the logAppender is nil, if you want to use it, please specify `--log-topic` at startup.
2020/12/02 11:38:47.848 contextFunc.go:30: [info] attempting to discard input
++++++++++++++++++++
ack input msg topic: persistent://public/default/in-topic
++++++++++++++++++++++++
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x46a1547]

goroutine 1 [running]:
github.com/apache/pulsar/pulsar-function-go/pf.(*goInstance).ackInputMessage(0xc000248000, 0x4a3d420, 0xc00040e000)
	/Users/wolf4j/github.com/apache/pulsar/pulsar-function-go/pf/instance.go:367 +0x1d7
github.com/apache/pulsar/pulsar-function-go/pf.(*goInstance).processResult(0xc000248000, 0x4a3d420, 0xc00040e000, 0x0, 0x0, 0x0)
	/Users/wolf4j/github.com/apache/pulsar/pulsar-function-go/pf/instance.go:355 +0x80
github.com/apache/pulsar/pulsar-function-go/pf.(*goInstance).startFunction(0xc000248000, 0x4a23740, 0xc00020e0c0, 0x0, 0x0)
	/Users/wolf4j/github.com/apache/pulsar/pulsar-function-go/pf/instance.go:177 +0x584
github.com/apache/pulsar/pulsar-function-go/pf.Start(0x4818b00, 0x4938eb0)
	/Users/wolf4j/github.com/apache/pulsar/pulsar-function-go/pf/function.go:171 +0x6b
main.main()
	/Users/wolf4j/github.com/apache/pulsar/pulsar-function-go/examples/contextFunc/contextFunc.go:35 +0x39
11:38:47.855 [pulsar-io-51-6] INFO  org.apache.pulsar.broker.service.ServerCnx - Closed connection from /127.0.0.1:54594
```

### Modifications

In this pull request, we define a `TopicName` structure to parse the topic name passed in by `pulsar-admin` or `pulsarctl`. If it is the default, use the full path method for splicing to ensure that the map is used key is the same.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Go functions]: Panic on discard message
4 participants