we couldn't do any operation for pulsar

[hbgongen]

we made the cluster of pulsar and put the pulsar-manager in another machine in docker,and we can't use any operation such as add namespace or add topic and so on.so maybe i need some configuration of pulsar?

[ChangWinde]

@hbgongen Do you have workarounds？I meet something the same as yours. I can add a topic, add a namespace .etc but I can't delete a certain topic or namespace and can't create a new tenant!

[tuteng]

Please first try check pulsar-manager.log:

second:

./bin/pulsar-admin clusters list
./bin/pulsar-admin clusters get clusster-name (eg, standalone)

The following results are correct:

./bin/pulsar-admin clusters get standalone
{
  "serviceUrl" : "http://127.0.0.1:8080",
  "brokerServiceUrl" : "pulsar://127.0.0.1:6650"
}

Your network should be connected to serviceUrl's network.
@ChangWinde

[ChangWinde]

@tuteng I do what you said and the result following
./bin/pulsar-admin clusters list
pulsar-cluster

./bin/pulsar-admin clusters get pulsar-cluster
{
"serviceUrl" : "http://pulsar.cluster.com:8080",
"serviceUrlTls" : "https://pulsar.cluster.com:8443",
"brokerServiceUrl" : "pulsar://pulsar.cluster.com:6650",
"brokerServiceUrlTls" : "pulsar+ssl://pulsar.cluster.com:6651"
}

It seems nothing wrong with it. What should I check next step?

[tuteng]

Is this http://pulsar.cluster.com your broker's domain name and can you access it?
Please test it use the command:

curl -v http://pulsar.cluster.com:8080/metrics/

If these two domain names are set incorrectly, please use the ./bin/pulsar-admin clusters update command to update them to be correct.

./bin/pulsar-admin clusters update --broker-url http://broker-ip:6605 --url http://broker-ip:8080 pulsar-cluster

@ChangWinde

[ChangWinde]

it seems to work! thanks for your help!

[ChangWinde]

@tuteng I have another question. I find a label 'BOOKIE' in managing cluster, but I can't find any data about bookies. Is it unfinished or I did something wrong

[tuteng]

Yes, we will add the management feature of the bookie in the future.

[jiazhai]

@tuteng how about provide some doc for this kind of FAQ? We could use this issue to track the documentation?

[rodrigodevelms]

Same error here:
My docker file:

version: "3.7"
services:
  pulsar:
    image: apachepulsar/pulsar
    hostname: pulsar
    container_name: pulsar
    restart: unless-stopped
    command: bin/pulsar standalone
    ports:
      - "6650:6650"
      - "8080:8080"
    volumes:
      - ./apache-pulsar/service-data:/pulsar/data
    networks:
      - pulsarnet

  pulsar-manager:
    image:  apachepulsar/pulsar-manager:v0.1.0
    hostname: pulsar-manager
    container_name: pulsar-manager
    restart: unless-stopped
    environment:
      REDIRECT_HOST: "http://127.0.0.1"
      REDIRECT_PORT: "9527"
      DRIVER_CLASS_NAME: "org.postgresql.Driver"
      URL: "jdbc:postgresql://127.0.0.1:5432/pulsar_manager"
      USERNAME: "pulsar"
      PASSWORD: "pulsar"
      USER: "root"
      LOG_LEVEL: DEBUG
    ports:
      - "9527:9527"
    depends_on:
      - pulsar
    networks:
      - pulsarnet
    volumes:
      - ./apache-pulsar/manager-data:/data

volumes:
  apache-pulsar:

networks:
  pulsarnet:
    driver: bridge

My initial setup

ServiceUrl does not accept

http: // localhost: 8080

or

http://127.0.0.1:8080

When I enter the "Cluster" tab
nothing is shown. And in the log I have the following message:

2020-06-08 12:33:48.382 DEBUG 42 --- [pool-2-thread-1] o.a.http.impl.execchain.MainClientExec   : Connection can be kept alive indefinitely
2020-06-08 12:33:48.382 DEBUG 42 --- [pool-2-thread-1] h.i.c.PoolingHttpClientConnectionManager : Connection [id: 101][route: {}->http://pulsar:8080] can be kept alive indefinitely
2020-06-08 12:33:48.382 DEBUG 42 --- [pool-2-thread-1] h.i.c.DefaultManagedHttpClientConnection : http-outgoing-101: set socket timeout to 0
2020-06-08 12:33:48.382 DEBUG 42 --- [pool-2-thread-1] h.i.c.PoolingHttpClientConnectionManager : Connection released: [id: 101][route: {}->http://pulsar:8080][total kept alive: 1; route allocated: 1 of 10; total allocated: 1 of 100]
2020-06-08 12:33:48.383  INFO 42 --- [pool-2-thread-1] o.a.p.m.s.impl.BrokerStatsServiceImpl    : Start collecting stats from env Aries / cluster standalone @ http://localhost:8080
2020-06-08 12:33:48.383 DEBUG 42 --- [pool-2-thread-1] o.a.h.client.protocol.RequestAddCookies  : CookieSpec selected: default
2020-06-08 12:33:48.383 DEBUG 42 --- [pool-2-thread-1] o.a.h.client.protocol.RequestAuthCache   : Auth cache not set in the context
2020-06-08 12:33:48.383 DEBUG 42 --- [pool-2-thread-1] h.i.c.PoolingHttpClientConnectionManager : Connection request: [route: {}->http://localhost:8080][total kept alive: 1; route allocated: 0 of 10; total allocated: 1 of 100]
2020-06-08 12:33:48.383 DEBUG 42 --- [pool-2-thread-1] h.i.c.PoolingHttpClientConnectionManager : Connection leased: [id: 102][route: {}->http://localhost:8080][total kept alive: 1; route allocated: 1 of 10; total allocated: 2 of 100]
2020-06-08 12:33:48.383 DEBUG 42 --- [pool-2-thread-1] o.a.http.impl.execchain.MainClientExec   : Opening connection {}->http://localhost:8080
2020-06-08 12:33:48.383 DEBUG 42 --- [pool-2-thread-1] .i.c.DefaultHttpClientConnectionOperator : Connecting to localhost/127.0.0.1:8080
2020-06-08 12:33:48.384 DEBUG 42 --- [pool-2-thread-1] h.i.c.DefaultManagedHttpClientConnection : http-outgoing-102: Shutdown connection
2020-06-08 12:33:48.384 DEBUG 42 --- [pool-2-thread-1] o.a.http.impl.execchain.MainClientExec   : Connection discarded
2020-06-08 12:33:48.384 DEBUG 42 --- [pool-2-thread-1] h.i.c.PoolingHttpClientConnectionManager : Connection released: [id: 102][route: {}->http://localhost:8080][total kept alive: 1; route allocated: 0 of 10; total allocated: 1 of 100]
2020-06-08 12:33:48.384 ERROR 42 --- [pool-2-thread-1] o.apache.pulsar.manager.utils.HttpUtil   : http request exception:Connect to localhost:8080 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
2020-06-08 12:33:48.384 DEBUG 42 --- [pool-2-thread-1] o.a.h.client.protocol.RequestAddCookies  : CookieSpec selected: default
2020-06-08 12:33:48.384 DEBUG 42 --- [pool-2-thread-1] o.a.h.client.protocol.RequestAuthCache   : Auth cache not set in the context
2020-06-08 12:33:48.384 DEBUG 42 --- [pool-2-thread-1] h.i.c.PoolingHttpClientConnectionManager : Connection request: [route: {}->http://localhost:8080][total kept alive: 1; route allocated: 0 of 10; total allocated: 1 of 100]
2020-06-08 12:33:48.384 DEBUG 42 --- [pool-2-thread-1] h.i.c.PoolingHttpClientConnectionManager : Connection leased: [id: 103][route: {}->http://localhost:8080][total kept alive: 1; route allocated: 1 of 10; total allocated: 2 of 100]
2020-06-08 12:33:48.384 DEBUG 42 --- [pool-2-thread-1] o.a.http.impl.execchain.MainClientExec   : Opening connection {}->http://localhost:8080
2020-06-08 12:33:48.384 DEBUG 42 --- [pool-2-thread-1] .i.c.DefaultHttpClientConnectionOperator : Connecting to localhost/127.0.0.1:8080
2020-06-08 12:33:48.384 DEBUG 42 --- [pool-2-thread-1] h.i.c.DefaultManagedHttpClientConnection : http-outgoing-103: Shutdown connection
2020-06-08 12:33:48.384 DEBUG 42 --- [pool-2-thread-1] o.a.http.impl.execchain.MainClientExec   : Connection discarded
2020-06-08 12:33:48.385 DEBUG 42 --- [pool-2-thread-1] h.i.c.PoolingHttpClientConnectionManager : Connection released: [id: 103][route: {}->http://localhost:8080][total kept alive: 1; route allocated: 0 of 10; total allocated: 1 of 100]
2020-06-08 12:33:48.385 ERROR 42 --- [pool-2-thread-1] o.apache.pulsar.manager.utils.HttpUtil   : http request exception:Connect to localhost:8080 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
2020-06-08 12:33:48.386 ERROR 42 --- [pool-2-thread-1] o.s.s.s.TaskUtils$LoggingErrorHandler    : Unexpected error occurred in scheduled task.

java.lang.NullPointerException: null
	at org.apache.pulsar.manager.service.impl.BrokersServiceImpl.getBrokersList(BrokersServiceImpl.java:55) ~[pulsar-manager.jar:na]
	at org.apache.pulsar.manager.service.impl.BrokerStatsServiceImpl.collectStatsToDB(BrokerStatsServiceImpl.java:150) ~[pulsar-manager.jar:na]
	at org.apache.pulsar.manager.service.impl.BrokerStatsServiceImpl.lambda$scheduleCollectStats$2(BrokerStatsServiceImpl.java:139) ~[pulsar-manager.jar:na]
	at java.util.HashMap.forEach(HashMap.java:1289) ~[na:1.8.0_212]
	at org.apache.pulsar.manager.service.impl.BrokerStatsServiceImpl.scheduleCollectStats(BrokerStatsServiceImpl.java:136) ~[pulsar-manager.jar:na]
	at sun.reflect.GeneratedMethodAccessor129.invoke(Unknown Source) ~[na:na]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_212]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_212]
	at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) ~[spring-context-5.0.6.RELEASE.jar:5.0.6.RELEASE]
	at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.0.6.RELEASE.jar:5.0.6.RELEASE]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_212]
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_212]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_212]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_212]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_212]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_212]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]

2020-06-08 12:33:49.664 DEBUG 42 --- [SimpleHostRoutingFilter.connectionManagerTimer] h.i.c.PoolingHttpClientConnectionManager : Closing expired connections
2020-06-08 12:33:54.664 DEBUG 42 --- [SimpleHostRoutingFilter.connectionManagerTimer] h.i.c.PoolingHttpClientConnectionManager : Closing expired connections

I did Tuteng's steps:

root@pulsar:/pulsar# ./bin/pulsar-admin clusters list
"standalone"

root@pulsar:/pulsar# ./bin/pulsar-admin clusters get standalone
{
  "serviceUrl" : "http://localhost:8080",
  "brokerServiceUrl" : "http://localhost:6605"
}

./bin/pulsar-admin clusters update --broker-url http://localhost:6605 --url http://localhost:8080 standalone

It works. But if I restart the container, the error comes back, and nothing else I do will solve it. I have to do

docker system prune -a
volume prune docker

reinstall everything
My /etc/hosts

127.0.0.1   localhost
127.0.0.1   dell
# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

I don't know what I can do anymore.

[tuteng]

You may update the address of the service.

./bin/pulsar-admin clusters update --broker-url http://pulsar-ip:6605 --url http://pulsar-ip:8080 standalone

@rodrigodevelms

[rodrigodevelms]

But every time I reset the container, or restart the computer, I have to make this command. I tried on docker-compose.yml to add the line

command:>
      bash -c "bin/pulsar standalone &&
      bin/pulsar-admin clusters update --broker-url http://pulsar:6605 --url http://pulsar:8080 standalone ".

But it does not work.

[tuteng]

If you start a cluster, it is configured by this command during initialization. http://pulsar.apache.org/docs/en/deploy-bare-metal/

bin/pulsar initialize-cluster-metadata \
  --cluster pulsar-cluster-1 \
  --zookeeper zk1.us-west.example.com:2181 \
  --configuration-store zk1.us-west.example.com:2181 \
  --web-service-url http://pulsar.us-west.example.com:8080 \
  --web-service-url-tls https://pulsar.us-west.example.com:8443 \
  --broker-service-url pulsar://pulsar.us-west.example.com:6650 \
  --broker-service-url-tls pulsar+ssl://pulsar.us-west.example.com:6651

If you start a standalone environment, you can configure the broadcast address advertisedAddress in the configuration file conf/standaone.conf

Then:

./bin/pulsar-admin clusters get cluster-name

In a word, our aim is to enable pulsar-manager service to connect the pulsar cluster, whether through the domain name or IP address @rodrigodevelms

[davidlanouette]

This issue is a dumpster fire. It appears to be 2 separate issues (that have similar causes - not configuring the cluster correctly). And both seem to have been answered/solved by @tuteng.

I would suggest that we create a new ticket to update the manager to provide better info, instead of hanging. Then close this ticket.