[Feature][Deploy] Add missing default values in dolphinscheduler_env.sh

[EricGao888]

Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

• In the latest code, we have dolphinscheduler_env.sh to set environment variables. However, a few default values are missing.

  dolphinscheduler/script/env/dolphinscheduler_env.sh

  Lines 33 to 36 in 88d2803

  	export SPRING_DATASOURCE_DRIVER_CLASS_NAME
  	export SPRING_DATASOURCE_URL
  	export SPRING_DATASOURCE_USERNAME
  	export SPRING_DATASOURCE_PASSWORD

• Also, we need some examples and docs for Dolphin users about how to config dolphinscheduler_env.sh
• https://github.com/apache/dolphinscheduler/blob/dev/script/env/dolphinscheduler_env.sh

Use case

• BTW, after building the project, we will have ./tools/bin/dolphinscheduler_env.sh and ./bin/env/dolphinscheduler_env.sh. May I ask why we need both of them?

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[github-actions]

Thank you for your feedback, we have received your issue, Please wait patiently for a reply.

• In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
• If you haven't received a reply for a long time, you can join our slack and send your question to channel #troubleshooting

[SbloodyS]

In fact, we can unite dolphinscheduler_env.sh into one.

[EricGao888]

In fact, we can unite dolphinscheduler_env.sh into one.

Another question is, we already have

dolphinscheduler/dolphinscheduler-tools/src/main/resources/application.yaml

Lines 20 to 24 in ebc4253

	datasource:
	driver-class-name: org.postgresql.Driver
	url: jdbc:postgresql://127.0.0.1:5432/dolphinscheduler
	username: root
	password: root

to config db connection, why we need to config again in dolphinscheduler_env.sh? Could we remove duplicate db connection configuration in dolphinscheduler_env.sh?

[EricGao888]

In fact, we can unite dolphinscheduler_env.sh into one.

Another question is, we already have

dolphinscheduler/dolphinscheduler-tools/src/main/resources/application.yaml

Lines 20 to 24 in ebc4253

	datasource:
	driver-class-name: org.postgresql.Driver
	url: jdbc:postgresql://127.0.0.1:5432/dolphinscheduler
	username: root
	password: root

to config db connection, why we need to config again in dolphinscheduler_env.sh? Could we remove duplicate db connection configuration in dolphinscheduler_env.sh?

Okay, I think I got it. DS has been refactored in micro-services fashion. Every module has its own {module_name}/conf/application.yaml after building the project. Therefore, it is definitely more reasonable for users to config bin/env/dolphinscheduler_env.sh so that the configuration will take effect globally.

[EricGao888]

After looking into other modules' start scripts, it seems even config bin/env/dolphinscheduler_env.sh will not take effect globally. In each module's start script except that of worker-server, it sources the dolphinscheduler_env.sh in its own bin folder instead of the global bin/env/dolphinscheduler_env.sh. Also each module's start script uses its own lib, which means if users use mysql driver, they need to add mysql-connector jar into every module's lib folder. Therefore, there are two problems:

• Why worker-sever uses the global bin/env/dolphinscheduler_env.sh but others don't?
• Is there any way more convenient for users to config dolphinscheduler_env.sh or add extra jar, e.g. mysql-connector jar, into lib folder?

[SbloodyS]

After looking into other modules' start scripts, it seems even config bin/env/dolphinscheduler_env.sh will not take effect globally. In each module's start script except that of worker-server, it sources the dolphinscheduler_env.sh in its own bin folder instead of the global bin/env/dolphinscheduler_env.sh. Also each module's start script uses its own lib, which means if users use mysql driver, they need to add mysql-connector jar into every module's lib folder. Therefore, there are two problems:

• Why worker-sever uses the global bin/env/dolphinscheduler_env.sh but others don't?
• Is there any way more convenient for users to config dolphinscheduler_env.sh or add extra jar, e.g. mysql-connector jar, into lib folder?

For dolphinscheduler_env.sh. I think it can be solved in a unified way for the following scenarios.

First of all by adding a variable like --envpath in each modules start.sh.

1. For those users who want to use in docker deployment. Setting the default value of --envpath to source each module's own dolphinscheduler_env.sh when executing bash bin/start.sh in docker container's starting.
2. For those users who want to use in non docker and cluster deployment. Then specify --envpath source ${DOLPHINSCHEDULER_HOME}/bin/env/dolphinscheduler_env.sh in start-all.sh/stop-all.sh/scp-hosts.sh/dolphinscheduler-daemon.sh.
3. For thos users who want to use in user defined deployment. Then users can chose using the way of setting ${DOLPHINSCHEDULER_HOME}/bin/env/dolphinscheduler_env.sh or By using bin/start.sh -- envpath to customize its own path.

If we do this. we should also modify the docs. WDYT? @EricGao888 @caishunfeng @zhongjiajie

[kezhenxu94]

Description

• In the latest code, we have dolphinscheduler_env.sh to set environment variables. However, a few default values are missing.

  dolphinscheduler/script/env/dolphinscheduler_env.sh

  Lines 33 to 36 in 88d2803

  	export SPRING_DATASOURCE_DRIVER_CLASS_NAME
  	export SPRING_DATASOURCE_URL
  	export SPRING_DATASOURCE_USERNAME
  	export SPRING_DATASOURCE_PASSWORD

Because we don't need to set these default, there default values are in application.yaml, we just export them so that they can be passed down to the start up script.

• BTW, after building the project, we will have ./tools/bin/dolphinscheduler_env.sh and ./bin/env/dolphinscheduler_env.sh. May I ask why we need both of them?

./bin/env/dolphinscheduler_env.sh is used when you install all the components in a single machine by using install.sh, but mostly, in production, you will install each component in separate machine, in that case, you just copy master-server to the desired machine and only ./master-server/bin/dolphinscheduler_env.sh is used, you won't be able to use ./bin/env/dolphinscheduler_env.sh in that case

[SbloodyS]

Description

• In the latest code, we have dolphinscheduler_env.sh to set environment variables. However, a few default values are missing.

  dolphinscheduler/script/env/dolphinscheduler_env.sh

  Lines 33 to 36 in 88d2803

  	export SPRING_DATASOURCE_DRIVER_CLASS_NAME
  	export SPRING_DATASOURCE_URL
  	export SPRING_DATASOURCE_USERNAME
  	export SPRING_DATASOURCE_PASSWORD

Because we don't need to set these default, there default values are in application.yaml, we just export them so that they can be passed down to the start up script.

• BTW, after building the project, we will have ./tools/bin/dolphinscheduler_env.sh and ./bin/env/dolphinscheduler_env.sh. May I ask why we need both of them?

./bin/env/dolphinscheduler_env.sh is used when you install all the components in a single machine by using install.sh, but mostly, in production, you will install each component in separate machine, in that case, you just copy master-server to the desired machine and only ./master-server/bin/dolphinscheduler_env.sh is used, you won't be able to use ./bin/env/dolphinscheduler_env.sh in that case

Yes. I got what you mean.

Some configurations are the same in the same cluster. Such as SPRING_DATASOURCE_DRIVER_CLASS_NAME and SPRING_DATASOURCE_URL. This is a little more repetitive configuration work for users. Especially for those users who want to use pseudo-cluster deployment.