[1.3.9-prepare]Add standalone server module to make it easier to develop and run #6022 #6480

CalvinKirs · 2021-10-09T13:02:54Z

No description provided.

…lop and run #6022

CalvinKirs · 2021-10-09T13:04:26Z

...nscheduler-service/src/main/java/org/apache/dolphinscheduler/service/zk/ZookeeperConfig.java

+        if(StringUtils.isNotBlank(System.getProperty("zookeeper.quorum"))){
+            return System.getProperty("zookeeper.quorum");
+        }


Avoid modifying too much, do this temporarily and the dev version is OK.

…lop and run #6022

CalvinKirs · 2021-10-11T15:41:51Z

@kezhenxu94 hi, can you help me review?thx

kezhenxu94 · 2021-10-11T15:42:37Z

@kezhenxu94 hi, can you help me review?thx

Sure. Will take a look soon.

kezhenxu94 · 2021-10-11T15:44:24Z

@CalvinKirs can we just cherry pick the related commits into the target branch?

kezhenxu94 · 2021-10-11T15:47:03Z

Standalone server is mostly for convenient development, so I don't know how much it will benefit to cherry pick this to the 1.3.9 release, I think developers often develop on dev branch, right?

CalvinKirs · 2021-10-11T15:47:55Z

@CalvinKirs can we just cherry pick the related commits into the target branch?

It’s very difficult, our branches are quite different 😭

CalvinKirs · 2021-10-11T15:49:59Z

Standalone server is mostly for convenient development, so I don't know how much it will benefit to cherry pick this to the 1.3.9 release, I think developers often develop on dev branch, right?

I totally agree with you. but the main purpose of doing this is to allow some new users to start easily and quickly. @dailidong

davidzollo · 2021-10-12T11:46:37Z

Standalone server is mostly for convenient development, so I don't know how much it will benefit to cherry pick this to the 1.3.9 release, I think developers often develop on dev branch, right?

I totally agree with you. but the main purpose of doing this is to allow some new users to start easily and quickly. @dailidong

Now there are many deployment steps for DolphinScheduler, especially for new users, what they want to feel most is to quickly experience whether this can meet his needs and how to quickly experience the functions of DolphinScheduler.

kezhenxu94

LGTM in terms of standalone server functionalities, I can see this error logs when running a simple shell task, might not be related to this but might be worthy to take a look.

2021-10-12 20:04:02.210 ERROR 76462 --- [ool-13-thread-2] o.a.d.remote.handler.NettyServerHandler  : process msg Command [type=TASK_EXECUTE_REQUEST, opaque=3, bodyLen=1510] error

java.lang.NullPointerException: null
	at org.apache.dolphinscheduler.server.worker.processor.TaskExecuteProcessor.getTaskLogPath(TaskExecuteProcessor.java:175) ~[classes/:na]
	at org.apache.dolphinscheduler.server.worker.processor.TaskExecuteProcessor.process(TaskExecuteProcessor.java:133) ~[classes/:na]
	at org.apache.dolphinscheduler.remote.handler.NettyServerHandler$1.run(NettyServerHandler.java:134) ~[classes/:na]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_292]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_292]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_292]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_292]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_292]

kezhenxu94 · 2021-10-12T11:51:16Z

...nscheduler-service/src/main/java/org/apache/dolphinscheduler/service/zk/ZookeeperConfig.java

    private int maxWaitTime;

    public String getServerList() {
+        if(StringUtils.isNotBlank(System.getProperty("zookeeper.quorum"))){


Seems it's not formatted

Thx your remind, done

kezhenxu94 · 2021-10-12T11:58:22Z

...ler-standalone-server/src/main/java/org/apache/dolphinscheduler/server/StandaloneServer.java

+    private static void startAlertServer() {
+        AlertServer.getInstance().start();
+    }


Can you cherry-pick this part from the latest dev branch and uncomment line 61 to enable alert server?

Suggested change

private static void startAlertServer() {

AlertServer.getInstance().start();

}

private static void startAlertServer() {

final Path alertPluginPath = Paths.get(

StandaloneServer.class.getProtectionDomain().getCodeSource().getLocation().getPath(),

"../../../dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/pom.xml"

).toAbsolutePath();

if (Files.exists(alertPluginPath)) {

System.setProperty("alert.plugin.binding", alertPluginPath.toString());

System.setProperty("alert.plugin.dir", "");

}

AlertServer.getInstance().start();

}

Well, the settings are totally different between this branch and dev branch?

I can see

alert.type=EMAIL plugin.dir=/Users/xx/your/path/to/plugin/dir

vs

alert.plugin.binding= alert.plugin.dir=

yup, an Email Alert is that must be loaded. An email will be loaded automatically when AlertServer is started

So we don’t need to set up email

sonarqubecloud · 2021-10-12T15:44:19Z

SonarCloud Quality Gate failed.

0 Bugs
0 Vulnerabilities
0 Security Hotspots
2 Code Smells

0.0% Coverage
0.0% Duplication

kezhenxu94

LGTM

lenboo

+1

CalvinKirs added 2 commits October 9, 2021 16:59

test

009109e

[1.3.9-prepare]Add standalone server module to make it easier to deve…

508896f

…lop and run #6022

CalvinKirs commented Oct 9, 2021

View reviewed changes

[1.3.9-prepare]Add standalone server module to make it easier to deve…

bd1ec0b

…lop and run #6022

CalvinKirs added this to the 1.3.9 milestone Oct 11, 2021

CalvinKirs added the enhancement New feature or request label Oct 11, 2021

CalvinKirs added the discussion discussion label Oct 11, 2021

kezhenxu94 approved these changes Oct 12, 2021

View reviewed changes

code format

1761f1a

kezhenxu94 approved these changes Oct 13, 2021

View reviewed changes

lenboo approved these changes Oct 13, 2021

View reviewed changes

lenboo merged commit 7223469 into apache:1.3.9-prepare Oct 13, 2021

-    private static void startAlertServer() {
-        AlertServer.getInstance().start();
-    }
+    private static void startAlertServer() {
+        final Path alertPluginPath = Paths.get(
+                StandaloneServer.class.getProtectionDomain().getCodeSource().getLocation().getPath(),
+                "../../../dolphinscheduler-alert-plugin/dolphinscheduler-alert-email/pom.xml"
+        ).toAbsolutePath();
+        if (Files.exists(alertPluginPath)) {
+            System.setProperty("alert.plugin.binding", alertPluginPath.toString());
+            System.setProperty("alert.plugin.dir", "");
+        }
+        AlertServer.getInstance().start();
+    }

[1.3.9-prepare]Add standalone server module to make it easier to develop and run #6022 #6480

[1.3.9-prepare]Add standalone server module to make it easier to develop and run #6022 #6480

Uh oh!

Conversation

CalvinKirs commented Oct 9, 2021

Uh oh!

CalvinKirs Oct 9, 2021

Choose a reason for hiding this comment

Uh oh!

CalvinKirs commented Oct 11, 2021

Uh oh!

kezhenxu94 commented Oct 11, 2021

Uh oh!

kezhenxu94 commented Oct 11, 2021

Uh oh!

kezhenxu94 commented Oct 11, 2021

Uh oh!

CalvinKirs commented Oct 11, 2021

Uh oh!

CalvinKirs commented Oct 11, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

davidzollo commented Oct 12, 2021

Uh oh!

kezhenxu94 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kezhenxu94 Oct 12, 2021

Choose a reason for hiding this comment

Uh oh!

CalvinKirs Oct 12, 2021

Choose a reason for hiding this comment

Uh oh!

kezhenxu94 Oct 12, 2021

Choose a reason for hiding this comment

Uh oh!

kezhenxu94 Oct 12, 2021

Choose a reason for hiding this comment

Uh oh!

CalvinKirs Oct 12, 2021

Choose a reason for hiding this comment

Uh oh!

CalvinKirs Oct 12, 2021

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud bot commented Oct 12, 2021

Uh oh!

kezhenxu94 left a comment

Choose a reason for hiding this comment

Uh oh!

lenboo left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

CalvinKirs commented Oct 11, 2021 •

edited

Loading

kezhenxu94 left a comment •

edited

Loading