Pulsar SQL Fails Out of the Box with Default Docker Deployment v2.5.0

[junquero]

Describe the bug
Run pulsar docker

To Reproduce
Steps to reproduce the behavior (commands in bold):

1. docker volume create pulsar_conf
docker volume create pulsar_data
docker run --name pulsar -it -p 6650:6650 -p 7080:8080 -v pulsar_data:/pulsar/data -v pulsar_conf:/pulsar/conf apachepulsar/pulsar-all:2.5.0 bin/pulsar standalone

2. docker exec -it pulsar /bin/bash
   2.1 # bin/pulsar sql-worker run

3. docker exec -it pulsar /bin/bash
   3.1 # bin/pulsar sql
   3.2 presto> show catalogs;
   3.3 presto> show schemas in pulsar;
   Query 20200305_223404_00001_pikhq failed: Failed to get schemas from pulsar: Cannot cast org.glassfish.jersey.inject.hk2.Hk2InjectionManagerFactory to org.glassfish.jersey.internal.inject.InjectionManagerFactory
   3.4 presto> show tables in pulsar."public/default";
   Query is gone (server restarted?)

In log file of pulsar sql worker:
Caused by: java.lang.ClassCastException: Cannot cast org.glassfish.jersey.inject.hk2.Hk2InjectionManagerFactory to org.glassfish.jersey.internal.inject.InjectionManagerFactory

Expected behavior
I expected it to work as described in Query data with Pulsar SQL

Screenshots
Not applicable.

Desktop (please complete the following information):

• OS: Windows 10. Docker Desktop 2.2.0.3 (Engine 19.03.5) Linux containers (with Hyper-V).

Additional context
Some minutes later, the pulsar broker crashes. In the log:

22:35:43.832 [registration-service-provider-scheduler] INFO org.apache.bookkeeper.stream.storage.impl.cluster.ClusterControllerLeaderImpl - Cluster topology is changed - new cluster : Versioned(value=[], version=12)
22:35:43.832 [Curator-LeaderSelector-0] INFO org.apache.bookkeeper.stream.storage.impl.cluster.ClusterControllerLeaderImpl - No servers is alive yet. Backoff 200ms and retry.

[junquero]

NOTE: this comment is not correct. The issue described is because volumes mounted in the container had data from pulsar 2.5.0.

---

Tested also with latest right now. It behaves diferently:

show schemas does work right, but show tables never ended because the broker did crash

1. presto> show schemas in pulsar;

    Schema

---

information_schema
public/default
public/functions
sample/etx
sample/standalone/ns1
(5 rows)

Query 20200305_235341_00001_qunvt, FINISHED, 1 node
Splits: 19 total, 19 done (100.00%)
0:05 [5 rows, 104B] [0 rows/s, 19B/s]

presto> show tables in pulsar."public/default";

Query 20200305_235405_00002_qunvt, RUNNING, 1 node, 19 splits
1:25 [ 0 rows, 0B] [ 0 rows/s, 0B/s] [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 0%

 STAGES   ROWS  ROWS/s  BYTES  BYTES/s  QUEUED    RUN   DONE

0.........R 0 0 0B 0B 0 1 0
1.......R 0 0 0B 0B 0 17 0
2.....R 0 0 0B 0B 0 1 0

here the query hangs and the broker crashes.

[junquero]

I edited the description of the issue to add two volumes creation commands before running the container. Also, note that 7080 to 8080 port mapping is used to avoid potential conflicts with other services like jboss/wildfly/tomcat/.... using port 8080 in the docker host.

[junquero]

I have to say that with a clean environment, it has worked with latest version. I just removed previous volumes (that were created and populated with data with a pulsar 2.5.0 container), and it worked. I have tried again with 2.5.0 with a clean volume, and the Cannot cast error is still there. Therefore, in summary, it doesn't work in 2.5.0, and it works in "latest".

Can this fix be backported to 2.5.0?

[codelipenghui]

@junquero Did you find the root cause? I also met this problem with the last master branch but 2.5.1 works well.

[codelipenghui]

The problem is very interesting. When I downgrade the maven version from 3.6.3 to 3.6.1, it works.

[junquero]

No, I didn't find the root cause. I was just playing with Pulsar SQL, but I'm not using it. Sorry.

[frejonb]

I have the same problem in 2.5.0

[sijie]

@frejonb The 2.5.0 image has a problem. Can you try the latest 2.5.2 image or 2.6.0 image?