[BUG][DOCKER] docker compose doesn't work #4804

mannharleen · 2021-02-19T05:22:37Z

**For better global communication, Please describe it in English. If you feel the description in English is not clear, then you can append description in Chinese(just for Mandarin(CN)), thx! **
Describe the bug

Out of the box - docker compose doesn't work

To Reproduce
Steps to reproduce the behavior, for example:

Follow the instruction at https://dolphinscheduler.apache.org/en-us/docs/1.3.4/user_doc/docker-deployment.html

Expected behavior
Master and Worker should show up under "Monitor" tab

Screenshots

zookeeper is working fine:

Which version of Dolphin Scheduler:
-1.3.4

Additional context
Logs:

dolphinscheduler-zookeeper     | 2021-02-19 11:43:38,838 [myid:1] - WARN  [NIOWorkerThread-4:NIOServerCnxn@373] - Close of session 0x0
dolphinscheduler-zookeeper     | java.io.IOException: Len error 1195725856
dolphinscheduler-zookeeper     |        at org.apache.zookeeper.server.NIOServerCnxn.readLength(NIOServerCnxn.java:541)
dolphinscheduler-zookeeper     |        at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:332)
dolphinscheduler-zookeeper     |        at org.apache.zookeeper.server.NIOServerCnxnFactory$IOWorkRequest.doWork(NIOServerCnxnFactory.java:522)
dolphinscheduler-zookeeper     |        at org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:154)
dolphinscheduler-zookeeper     |        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
dolphinscheduler-zookeeper     |        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
dolphinscheduler-zookeeper     |        at java.base/java.lang.Thread.run(Thread.java:834)
dolphinscheduler-zookeeper     | 2021-02-19 11:43:40,234 [myid:1] - WARN  [NIOWorkerThread-2:NIOServerCnxn@373] - Close of session 0x0
dolphinscheduler-zookeeper     | java.io.IOException: Len error 1195725856
dolphinscheduler-zookeeper     |        at org.apache.zookeeper.server.NIOServerCnxn.readLength(NIOServerCnxn.java:541)
dolphinscheduler-zookeeper     |        at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:332)
dolphinscheduler-zookeeper     |        at org.apache.zookeeper.server.NIOServerCnxnFactory$IOWorkRequest.doWork(NIOServerCnxnFactory.java:522)
dolphinscheduler-zookeeper     |        at org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:154)
dolphinscheduler-zookeeper     |        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
dolphinscheduler-zookeeper     |        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
dolphinscheduler-zookeeper     |        at java.base/java.lang.Thread.run(Thread.java:834)
dolphinscheduler-zookeeper     | 2021-02-19 11:43:45,268 [myid:1] - WARN  [NIOWorkerThread-1:NIOServerCnxn@373] - Close of session 0x0
dolphinscheduler-zookeeper     | java.io.IOException: Len error 1195725856
dolphinscheduler-zookeeper     |        at org.apache.zookeeper.server.NIOServerCnxn.readLength(NIOServerCnxn.java:541)
dolphinscheduler-zookeeper     |        at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:332)
dolphinscheduler-zookeeper     |        at org.apache.zookeeper.server.NIOServerCnxnFactory$IOWorkRequest.doWork(NIOServerCnxnFactory.java:522)
dolphinscheduler-zookeeper     |        at org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:154)
dolphinscheduler-zookeeper     |        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
dolphinscheduler-zookeeper     |        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
dolphinscheduler-zookeeper     |        at java.base/java.lang.Thread.run(Thread.java:834)
dolphinscheduler-zookeeper     | 2021-02-19 13:07:23,703 [myid:1] - INFO  [NIOWorkerThread-3:NIOServerCnxn@507] - Processing ruok command from /172.22.0.4:45996
dolphinscheduler-zookeeper     | 2021-02-19 13:07:23,707 [myid:1] - INFO  [NIOWorkerThread-4:NIOServerCnxn@507] - Processing srvr command from /172.22.0.4:45998
dolphinscheduler-zookeeper     | 2021-02-19 13:07:23,711 [myid:1] - INFO  [NIOWorkerThread-2:NIOServerCnxn@507] - Processing wchs command from /172.22.0.4:46000
dolphinscheduler-zookeeper     | 2021-02-19 13:07:23,714 [myid:1] - INFO  [NIOWorkerThread-1:NIOServerCnxn@507] - Processing cons command from /172.22.0.4:46002

dolphinscheduler-worker        | init env variables
dolphinscheduler-worker        | generate app config
dolphinscheduler-worker        | generate nginx config
dolphinscheduler-worker        | connect remote zookeeper
dolphinscheduler-worker        | test postgresql service
dolphinscheduler-worker        | connect postgresql service
dolphinscheduler-worker        | import sql data
dolphinscheduler-worker        |  _ _   |_  _ _|_. ___ _ |    _
dolphinscheduler-worker        | | | |\/|_)(_| | |_\  |_)||_|_\
dolphinscheduler-worker        |      /               |
dolphinscheduler-worker        |                         3.2.0
dolphinscheduler-worker        | 11:17:57.356 [main] INFO ru.yandex.clickhouse.ClickHouseDriver - Driver registered
dolphinscheduler-worker        | 11:17:58.118 [main] INFO com.alibaba.druid.pool.DruidDataSource - {dataSource-1} inited
dolphinscheduler-worker        | 11:17:58.258 [main] DEBUG com.alibaba.druid.pool.PreparedStatementPool - stmt enter cache
dolphinscheduler-worker        | 11:17:58.286 [main] INFO org.apache.dolphinscheduler.dao.upgrade.DolphinSchedulerManager - The database has been initialized. Skip the initialization step
dolphinscheduler-worker        | 11:17:58.286 [main] INFO org.apache.dolphinscheduler.dao.upgrade.shell.CreateDolphinScheduler - init DolphinScheduler finished
dolphinscheduler-worker        | 11:17:58.347 [main] DEBUG com.alibaba.druid.pool.PreparedStatementPool - stmt enter cache
dolphinscheduler-worker        | 11:17:58.363 [main] DEBUG com.alibaba.druid.pool.PreparedStatementPool - stmt enter cache
dolphinscheduler-worker        | 11:17:58.364 [main] INFO org.apache.dolphinscheduler.dao.upgrade.shell.CreateDolphinScheduler - upgrade DolphinScheduler finished
dolphinscheduler-worker        | 11:17:58.364 [main] INFO org.apache.dolphinscheduler.dao.upgrade.shell.CreateDolphinScheduler - create DolphinScheduler success
dolphinscheduler-worker        | start worker-server
dolphinscheduler-worker        | Begin stop worker-server......
dolphinscheduler-worker        | no worker-server to stop
dolphinscheduler-worker        | End stop worker-server.
dolphinscheduler-worker        | Begin start worker-server......
dolphinscheduler-worker        | starting worker-server, logging to /opt/dolphinscheduler/bin/../logs/dolphinscheduler-worker-server-94a7255fe9f0.out
dolphinscheduler-worker        | nohup /usr/lib/jvm/java-1.8-openjdk/bin/java -Dlogging.config=classpath:logback-worker.xml -Ddruid.mysql.usePingMethod=false -server -Xms2g -Xmx2g -Xmn1g -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=128m  -Xss512k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:LargePageSizeInBytes=128m -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError  -XX:HeapDumpPath=dump.hprof -classpath /opt/dolphinscheduler/bin/../conf:/opt/dolphinscheduler/bin/../lib/* org.apache.dolphinscheduler.server.worker.WorkerServer > /opt/dolphinscheduler/bin/../logs/dolphinscheduler-worker-server-94a7255fe9f0.out 2>&1 &
dolphinscheduler-worker        | End start worker-server.
dolphinscheduler-worker        | start logger-server
dolphinscheduler-worker        | Begin stop logger-server......
dolphinscheduler-worker        | no logger-server to stop
dolphinscheduler-worker        | End stop logger-server.
dolphinscheduler-worker        | Begin start logger-server......
dolphinscheduler-worker        | starting logger-server, logging to /opt/dolphinscheduler/bin/../logs/dolphinscheduler-logger-server-94a7255fe9f0.out
dolphinscheduler-worker        | nohup /usr/lib/jvm/java-1.8-openjdk/bin/java  -server -Xms1g -Xmx1g -Xmn500m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=128m  -Xss512k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:LargePageSizeInBytes=128m -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError  -XX:HeapDumpPath=dump.hprof -classpath /opt/dolphinscheduler/bin/../conf:/opt/dolphinscheduler/bin/../lib/* org.apache.dolphinscheduler.server.log.LoggerServer > /opt/dolphinscheduler/bin/../logs/dolphinscheduler-logger-server-94a7255fe9f0.out 2>&1 &
dolphinscheduler-worker        | End start logger-server.
dolphinscheduler-worker        | tail begin
dolphinscheduler-worker        | [INFO] 2021-02-18 19:05:52.134 org.apache.dolphinscheduler.service.zk.ZookeeperCachedOperator:[55] - add listener to zk path: /dolphinscheduler

dolphinscheduler-master        | init env variables
dolphinscheduler-master        | generate app config
dolphinscheduler-master        | generate nginx config
dolphinscheduler-master        | connect remote zookeeper
dolphinscheduler-master        | test postgresql service
dolphinscheduler-master        | connect postgresql service
dolphinscheduler-master        | import sql data
dolphinscheduler-master        |  _ _   |_  _ _|_. ___ _ |    _
dolphinscheduler-master        | | | |\/|_)(_| | |_\  |_)||_|_\
dolphinscheduler-master        |      /               |
dolphinscheduler-master        |                         3.2.0
dolphinscheduler-master        | 11:17:56.831 [main] INFO ru.yandex.clickhouse.ClickHouseDriver - Driver registered
dolphinscheduler-master        | 11:17:57.389 [main] INFO com.alibaba.druid.pool.DruidDataSource - {dataSource-1} inited
dolphinscheduler-master        | 11:17:57.530 [main] DEBUG com.alibaba.druid.pool.PreparedStatementPool - stmt enter cache
dolphinscheduler-master        | 11:17:57.664 [main] INFO org.apache.dolphinscheduler.dao.upgrade.DolphinSchedulerManager - The database has been initialized. Skip the initialization step
dolphinscheduler-master        | 11:17:57.665 [main] INFO org.apache.dolphinscheduler.dao.upgrade.shell.CreateDolphinScheduler - init DolphinScheduler finished
dolphinscheduler-master        | 11:17:57.753 [main] DEBUG com.alibaba.druid.pool.PreparedStatementPool - stmt enter cache
dolphinscheduler-master        | 11:17:57.766 [main] DEBUG com.alibaba.druid.pool.PreparedStatementPool - stmt enter cache
dolphinscheduler-master        | 11:17:57.768 [main] INFO org.apache.dolphinscheduler.dao.upgrade.shell.CreateDolphinScheduler - upgrade DolphinScheduler finished
dolphinscheduler-master        | 11:17:57.768 [main] INFO org.apache.dolphinscheduler.dao.upgrade.shell.CreateDolphinScheduler - create DolphinScheduler success
dolphinscheduler-master        | start master-server
dolphinscheduler-master        | Begin stop master-server......
dolphinscheduler-master        | no master-server to stop
dolphinscheduler-master        | End stop master-server.
dolphinscheduler-master        | Begin start master-server......
dolphinscheduler-master        | starting master-server, logging to /opt/dolphinscheduler/bin/../logs/dolphinscheduler-master-server-be835eb5e254.out
dolphinscheduler-master        | nohup /usr/lib/jvm/java-1.8-openjdk/bin/java -Dlogging.config=classpath:logback-master.xml -Ddruid.mysql.usePingMethod=false -server -Xms4g -Xmx4g -Xmn2g -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=128m  -Xss512k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:LargePageSizeInBytes=128m -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError  -XX:HeapDumpPath=dump.hprof -classpath /opt/dolphinscheduler/bin/../conf:/opt/dolphinscheduler/bin/../lib/* org.apache.dolphinscheduler.server.master.MasterServer > /opt/dolphinscheduler/bin/../logs/dolphinscheduler-master-server-be835eb5e254.out 2>&1 &
dolphinscheduler-master        | End start master-server.
dolphinscheduler-master        | tail begin
dolphinscheduler-master        | [INFO] 2021-02-18 19:04:56.075 org.apache.zookeeper.ClientCnxn:[1156] - Client session timed out, have not heard from server in 26750ms for sessionid 0x100028582e10000, closing socket connection and attempting reconnect

Whats going wrong?

The text was updated successfully, but these errors were encountered:

chengshiwen · 2021-02-19T06:36:58Z

@mannharleen The docker image 1.3.4 of dolphinscheduler has problems and has not been updated for a long time. In 1.3.5 there many updates about docker and k8s, please visit the 1.3.5 Release Notes and the latest readme English or Chinese.
Attention: Both the official docker image in https://hub.docker.com/r/apache/dolphinscheduler and the official document in http://dolphinscheduler.apache.org/en-us/docs/1.3.5/user_doc/docker-deployment.html have not been updated due to the Chinese New Year. They will be updated soon.

mannharleen · 2021-02-19T08:35:38Z

Ah I see. I will try with 1.3.5 and close this issue if the same issues don’t crop up again.

Thanks!

chengshiwen · 2021-02-19T09:06:01Z

@mannharleen The docker image 1.3.5 of dolphinscheduler has been pushed to docker hub named apache/dolphinscheduler:latest and apache/dolphinscheduler:1.3.5. You can download the docker-compose.yml 1.3.5 and execute docker pull apache/dolphinscheduler:latest (Note that the image id is a9514c9fe1d4) and docker-compose up -d. Wait a minute to check http://127.0.0.1:12345/dolphinscheduler with username/password admin/dolphinscheduler123.

Attention: You can create a tenant named root and associate the admin user with the root tenant in docker. Next, start the joyful test.

davidzollo · 2021-02-19T09:17:51Z

@mannharleen , hi , we created Slack workspace of Apache DolphinScheduler Community some days ago,
welcome to join through invitation url: https://join.slack.com/t/asf-dolphinscheduler/shared_invite/zt-l8k90ceu-wwUfobaDkJxjzMfZp4y1Ag

mannharleen · 2021-02-19T10:34:20Z

@chengshiwen Still facing an error:

dolphinscheduler-zookeeper     | EndOfStreamException: Unable to read additional data from client, it probably closed the socket: address = /172.22.0.7:55722, session = 0x10007306bcf0001
dolphinscheduler-zookeeper     |        at org.apache.zookeeper.server.NIOServerCnxn.handleFailedRead(NIOServerCnxn.java:163)
dolphinscheduler-zookeeper     |        at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:326)
dolphinscheduler-zookeeper     |        at org.apache.zookeeper.server.NIOServerCnxnFactory$IOWorkRequest.doWork(NIOServerCnxnFactory.java:522)
dolphinscheduler-zookeeper     |        at org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:154)
dolphinscheduler-zookeeper     |        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
dolphinscheduler-zookeeper     |        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
dolphinscheduler-zookeeper     |        at java.base/java.lang.Thread.run(Thread.java:834)

Using the latest image now:

chengshiwen · 2021-02-19T10:39:39Z

@mannharleen Re-try:

docker-compose down -v
check version of docker-compose.yml is 1.3.5, it's better to download it from docker-compose.yml
docker-compose up -d
watch -n 1 docker ps until healthy if watch command exists
docker logs -f dolphinscheduler-api until listening :12345 occurs

chengshiwen · 2021-02-19T11:19:49Z

I notice that you are using Windows. If the problem still exists, please try to run it on Linux/Mac. And tell us your environment, I will try to reproduce the problem, thx

davidzollo · 2021-02-20T08:06:23Z

@mannharleen hi, does zookeeper problem solved ? can I help you ?

mannharleen · 2021-02-20T11:36:43Z

Am really impressed by the fact that you guys (@dailidong and @chengshiwen) are actually following up!!

I didn’t get the chance yet, but will give it all a try again soon.

chengshiwen · 2021-02-24T10:03:42Z

We have tried to reproduce the issue on Windows with Docker Toolbox, but there is no such problem and it runs normally😂
If possible, please provide more environment information to help us troubleshoot the problem, thank you very much!🙏

chengshiwen · 2021-04-02T10:48:38Z

Close this issue, since no updats for a long time and unable to reproduce

chengshiwen · 2021-04-29T15:17:12Z

@mannharleen There is another possible reason for this problem, which is that the memory is too small. To solve this problem, please refer to https://dolphinscheduler.apache.org/en-us/docs/latest/user_doc/docker-deployment.html and check 0. Requirements

mannharleen changed the title ~~docker compose doesn't work~~ [BUG][DOCKER] docker compose doesn't work Feb 19, 2021

chengshiwen mentioned this issue Feb 23, 2021

Docker & K8s Improvement Plan #4837

Closed

71 tasks

lenboo added this to Requirement(需求) in DolphinScheduler Work Plan via automation Feb 24, 2021

lenboo moved this from Requirement(需求) to Doing(正在开发中) in DolphinScheduler Work Plan Feb 24, 2021

chengshiwen closed this as completed Apr 2, 2021

davidzollo removed this from Doing in DolphinScheduler Work Plan Apr 18, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG][DOCKER] docker compose doesn't work #4804

[BUG][DOCKER] docker compose doesn't work #4804

mannharleen commented Feb 19, 2021

chengshiwen commented Feb 19, 2021 •

edited

mannharleen commented Feb 19, 2021

chengshiwen commented Feb 19, 2021 •

edited

davidzollo commented Feb 19, 2021

mannharleen commented Feb 19, 2021

chengshiwen commented Feb 19, 2021 •

edited

chengshiwen commented Feb 19, 2021

davidzollo commented Feb 20, 2021

mannharleen commented Feb 20, 2021

chengshiwen commented Feb 24, 2021 •

edited

chengshiwen commented Apr 2, 2021

chengshiwen commented Apr 29, 2021 •

edited

[BUG][DOCKER] docker compose doesn't work #4804

[BUG][DOCKER] docker compose doesn't work #4804

Comments

mannharleen commented Feb 19, 2021

chengshiwen commented Feb 19, 2021 • edited

mannharleen commented Feb 19, 2021

chengshiwen commented Feb 19, 2021 • edited

davidzollo commented Feb 19, 2021

mannharleen commented Feb 19, 2021

chengshiwen commented Feb 19, 2021 • edited

chengshiwen commented Feb 19, 2021

davidzollo commented Feb 20, 2021

mannharleen commented Feb 20, 2021

chengshiwen commented Feb 24, 2021 • edited

chengshiwen commented Apr 2, 2021

chengshiwen commented Apr 29, 2021 • edited

chengshiwen commented Feb 19, 2021 •

edited

chengshiwen commented Feb 19, 2021 •

edited

chengshiwen commented Feb 19, 2021 •

edited

chengshiwen commented Feb 24, 2021 •

edited

chengshiwen commented Apr 29, 2021 •

edited