Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pulsar broker start failure when set functionsWorkerEnabled=true #2328

Closed
codelipenghui opened this issue Aug 7, 2018 · 6 comments
Closed

Pulsar broker start failure when set functionsWorkerEnabled=true #2328

codelipenghui opened this issue Aug 7, 2018 · 6 comments
Assignees
@sijie
Labels
area/function deprecated/question Questions should happened in GitHub Discussions

Comments

@codelipenghui
Copy link
Contributor

codelipenghui commented Aug 7, 2018 •
edited
Loading

When i set functionsWorkerEnabled=true

$ bin/pulsar broker

I got exception like this:

17:40:27.945 [main] ERROR org.apache.pulsar.functions.worker.SchedulerManager - Failed to create producer to function assignment topic persistent://public/functions/assignments
org.apache.pulsar.client.api.PulsarClientException$BrokerMetadataException: Namespace missing local cluster name in clusters list: local_cluster=pulsar-qa ns=public/functions clusters=[standalone]
	at org.apache.pulsar.client.impl.ClientCnx.getPulsarClientException(ClientCnx.java:798) ~[org.apache.pulsar-pulsar-client-original-2.2.0-incubating-SNAPSHOT.jar:2.2.0-incubating-SNAPSHOT]
	at org.apache.pulsar.client.impl.ClientCnx.handlePartitionResponse(ClientCnx.java:409) ~[org.apache.pulsar-pulsar-client-original-2.2.0-incubating-SNAPSHOT.jar:2.2.0-incubating-SNAPSHOT]

My broker conf:

# Name of the cluster to which this broker belongs to
clusterName=pulsar-qa
@codelipenghui codelipenghui changed the title Pulsar broker start failure Pulsar broker start failure when set functionsWorkerEnabled=true Aug 7, 2018
@codelipenghui
Copy link
Contributor Author

codelipenghui commented Aug 7, 2018 •
edited
Loading

Found the reason for this issue.

In zookeeper data of path named " /admin/policies/public/functions" has "replication_clusters":["standalone"]

When i set it to my cluster name "pulsar-qa". I can start the broker success.

@codelipenghui
Copy link
Contributor Author

To change config pulsarFunctionsCluster in functions_worker.yml

@sijie
Copy link
Member

sijie commented Aug 7, 2018

@codelipenghui that's correct. the functions_worker.yml was shipped for standalone deployment. It has to be updated for cluster deployment. Let me improve the documentation on how to enable function worker in cluster mode in #2327

@sijie sijie self-assigned this Aug 7, 2018
@sijie sijie added deprecated/question Questions should happened in GitHub Discussions area/function labels Aug 7, 2018
@sijie
Copy link
Member

sijie commented Aug 7, 2018

Closed the issue. Since the problem was addressed by updating functions_worker.yml, #2327 will be used for tracking the documentation updates.

@sijie sijie closed this as completed Aug 7, 2018
@lelabo-m
Copy link
Contributor

lelabo-m commented Mar 5, 2019

Seems to be a mistake in the PR referenced.
I think the PR fixing this issue is #2335.

@rocketraman
Copy link
Member

rocketraman commented Oct 7, 2019 •
edited
Loading

I have this same issue with running Pulsar on Kubernetes. I see that the script gen-yml-from-env.py is supposed to update functions_worker.yml, but I think the script is missing the value pulsarFunctionsCluster in INT_KEYS. I created a followup issue #5325.

@rocketraman rocketraman mentioned this issue Oct 7, 2019
Closed
@sijie sijie mentioned this issue Dec 27, 2019
Closed
@wangjialing218 wangjialing218 mentioned this issue May 12, 2021
Merged
codelipenghui pushed a commit that referenced this issue May 18, 2021
@wangjialing218
Get function cluster from broker config when start function worker wi…
3bdc776 
…th broker (#10552)

### Motivation
When start function worker with broker, we need to set pulsarFunctionsCluster in functions_worker.yml, other wise it will fail to start.   #2328
If we run broker in k8s, we should set correct config map to change the default value in functions_worker.yml.
In this mode, pulsarFunctionsCluster should always be same with the cluster name in broker.conf, so we can get the setting directly from broker.conf.

### Modifications

get function cluster from broker config when start function worker with broker
yangl pushed a commit to yangl/pulsar that referenced this issue Jun 23, 2021
@wangjialing218 @yangl
Get function cluster from broker config when start function worker wi…
2c6178a 
…th broker (apache#10552)

### Motivation
When start function worker with broker, we need to set pulsarFunctionsCluster in functions_worker.yml, other wise it will fail to start.   apache#2328
If we run broker in k8s, we should set correct config map to change the default value in functions_worker.yml.
In this mode, pulsarFunctionsCluster should always be same with the cluster name in broker.conf, so we can get the setting directly from broker.conf.

### Modifications

get function cluster from broker config when start function worker with broker
bharanic-dev pushed a commit to bharanic-dev/pulsar that referenced this issue Mar 18, 2022
@wangjialing218 @ciaocloud
Get function cluster from broker config when start function worker wi…
7c602e2 
…th broker (apache#10552)

### Motivation
When start function worker with broker, we need to set pulsarFunctionsCluster in functions_worker.yml, other wise it will fail to start.   apache#2328
If we run broker in k8s, we should set correct config map to change the default value in functions_worker.yml.
In this mode, pulsarFunctionsCluster should always be same with the cluster name in broker.conf, so we can get the setting directly from broker.conf.

### Modifications

get function cluster from broker config when start function worker with broker
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sijie sijie

Labels
area/function deprecated/question Questions should happened in GitHub Discussions
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@rocketraman @sijie @lelabo-m @codelipenghui