Skip to content

Commit

Permalink
Get function cluster from broker config when start function worker wi…
Browse files Browse the repository at this point in the history
…th broker (#10552)

### Motivation
When start function worker with broker, we need to set pulsarFunctionsCluster in functions_worker.yml, other wise it will fail to start.   #2328
If we run broker in k8s, we should set correct config map to change the default value in functions_worker.yml.
In this mode, pulsarFunctionsCluster should always be same with the cluster name in broker.conf, so we can get the setting directly from broker.conf.

### Modifications

get function cluster from broker config when start function worker with broker
  • Loading branch information
wangjialing218 committed May 18, 2021
1 parent 5e2440b commit 3bdc776
Show file tree
Hide file tree
Showing 3 changed files with 1 addition and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -1531,6 +1531,7 @@ public static WorkerConfig initializeWorkerConfigFromBrokerConfig(ServiceConfigu
String hostname = ServiceConfigurationUtils.getDefaultOrConfiguredAddress(
ServiceConfigurationUtils.getAppliedAdvertisedAddress(brokerConfig));
workerConfig.setWorkerHostname(hostname);
workerConfig.setPulsarFunctionsCluster(brokerConfig.getClusterName());
// inherit broker authorization setting
workerConfig.setAuthenticationEnabled(brokerConfig.isAuthenticationEnabled());
workerConfig.setAuthenticationProviders(brokerConfig.getAuthenticationProviders());
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@
package org.apache.pulsar.functions.worker;

import static org.testng.Assert.assertEquals;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.google.common.collect.Sets;
Expand Down Expand Up @@ -121,7 +120,6 @@ void setup() throws Exception {
tempDirectories[i] = PulsarFunctionTestTemporaryDirectory.create(getClass().getSimpleName());
tempDirectories[i].useTemporaryDirectoriesForWorkerConfig(workerConfig);
workerConfig.setPulsarFunctionsNamespace("public/functions");
workerConfig.setPulsarFunctionsCluster("my-cluster");
workerConfig.setSchedulerClassName(
org.apache.pulsar.functions.worker.scheduler.RoundRobinScheduler.class.getName());
workerConfig.setFunctionRuntimeFactoryClassName(ThreadRuntimeFactory.class.getName());
Expand Down
1 change: 0 additions & 1 deletion site2/docs/functions-worker.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@ In this mode, most of the settings are already inherited from your broker config
Pay attention to the following required settings when configuring functions-worker in this mode.

- `numFunctionPackageReplicas`: The number of replicas to store function packages. The default value is `1`, which is good for standalone deployment. For production deployment, to ensure high availability, set it to be larger than `2`.
- `pulsarFunctionsCluster`: Set the value to your Pulsar cluster name (same as the `clusterName` setting in the broker configuration).
- `initializedDlogMetadata`: Whether to initialize distributed log metadata in runtime. If it is set to `true`, you must ensure that it has been initialized by `bin/pulsarinitialize-cluster-metadata` command.

If authentication is enabled on the BookKeeper cluster, configure the following BookKeeper authentication settings.
Expand Down

0 comments on commit 3bdc776

Please sign in to comment.