Skip to content
Permalink
Browse files
Merge branch 'master' into master
  • Loading branch information
Eights-Li committed Jan 19, 2020
2 parents af3b93d + eac4b8e commit fbc269308d0ff599e6fe0c38cfe3bb3296489631
Showing 41 changed files with 1,139 additions and 155 deletions.
@@ -15,7 +15,7 @@ jobs:
node-version: 8.16.2
- name: Build
run: |
mkdir dist && npm install && npm i docsite && node_modules/docsite/bin/docsite.js build && cp -r baidu_verify_*.html googleff48c3fa45891a20.html sitemap.xml 404.html .asf.yaml index.html en-us zh-cn build img download dist/
mkdir dist && npm install && npm i docsite && node_modules/docsite/bin/docsite.js build && cp -r sitemap.xml 404.html .asf.yaml index.html en-us zh-cn build img download dist/
- name: Deploy
uses: peaceiris/actions-gh-pages@v2.4.0
@@ -0,0 +1,35 @@
.git
.svn
.hg
.zip
.gz
.DS_Store
.idea
.idea/
.idea/*
.target
.target/
**/**/target/**
target/*
*/target
*/target/*
.settings
.nbproject
.classpath
.project
*.iml
*.ipr
*.iws
*.tgz
.*.swp
.vim
.tmp
node_modules
npm-debug.log
.vscode
logs/*
.www
t.*
yarn.lock
package-lock.json
config.gypi

This file was deleted.

@@ -7,7 +7,7 @@ There are two deployment modes for the backend:

## Preparations

Download the latest version of the installation package, download address: [download](https://dist.apache.org/repos/dist/dev/incubator/dolphinscheduler),
Download the latest version of the installation package, download address: [download](https://dolphinscheduler.apache.org/en-us/docs/user_doc/download.html),
download apache-dolphinscheduler-incubating-x.x.x-dolphinscheduler-backend-bin.tar.gz


@@ -78,7 +78,7 @@ Configure SSH secret-free login on deployment machines and other installation ma

```directory
bin : Basic service startup script
DISCLAIMER : DISCLAIMER
DISCLAIMER-WIP : DISCLAIMER-WIP
conf : Project Profile
lib : The project relies on jar packages, including individual module jars and third-party jars
LICENSE : LICENSE
@@ -97,7 +97,7 @@ install.sh : One-click deployment script

- Modify deployment parameters (depending on your server and business situation):

- Modify the parameters in **install.sh** to replace the values required by your business
- Modify the parameters in `install.sh` to replace the values required by your business
- MonitorServerState switch variable, added in version 1.0.3, controls whether to start the self-start script (monitor master, worker status, if off-line will start automatically). The default value of "false" means that the self-start script is not started, and if it needs to start, it is changed to "true".
- 'hdfsStartupSate' switch variable controls whether to start hdfs
The default value of "false" means not to start hdfs
@@ -107,9 +107,9 @@ install.sh : One-click deployment script


## Deployment
Either of the following two methods can be deployed,automated deployment is recommended, and experienced partners can use source deployment as well.
Either of the following two methods can be deployed,binary file deployment is recommended, and experienced partners can use source deployment as well.

### Automated Deployment
### Binary file Deployment

- Install zookeeper tools

@@ -147,17 +147,59 @@ After successful deployment, the log can be viewed and stored in a specified fol

After downloading the release version of the source package, unzip it into the root directory

* Execute the compilation command:
* Build a tar package

```
mvn -U clean package -Prelease -Dmaven.test.skip=true
```
Execute the compilation command:

```
mvn -U clean package -Prelease -Dmaven.test.skip=true
```

* View directory
View directory

After normal compilation, `apache-dolphinscheduler-incubating-${latest.release.version}-dolphinscheduler-backend-bin.tar.gz`
After normal compilation, `apache-dolphinscheduler-incubating-${latest.release.version}-dolphinscheduler-backend-bin.tar.gz`
is generated in the `./dolphinscheduler-dist/dolphinscheduler-backend/target` directory

* OR build a rpm package

The rpm package can be installed on the Linux platform using the rpm command or yum. The rpm package can be used to help Dolphinscheduler better integrate with other management tools, such as ambari, cloudera manager.

Execute the compilation command:

```
mvn -U clean package -Prpmbuild -Dmaven.test.skip=true
```

View directory

After normal compilation, `apache-dolphinscheduler-incubating-${latest.release.version}-1.noarch.rpm`
is generated in the `./dolphinscheduler-dist/target/rpm/apache-dolphinscheduler-incubating/RPMS/noarch/` directory


* Decompress the compiled tar.gz package or use the rpm command to install (the rpm installation method will install dolphinscheduler in the /opt/soft directory) . The dolphinscheduler directory structure is like this:

```
../
├── bin
├── conf
|── DISCLAIMER
|—— install.sh
|—— lib
|—— LICENSE
|—— licenses
|—— NOTICE
|—— script
|—— sql
```


- Install zookeeper tools

`pip install kazoo`

- Switch to deployment user, one-click deployment

`sh install.sh`

### Start-and-stop services commonly used in systems (for service purposes, please refer to System Architecture Design for details)

@@ -8,7 +8,7 @@ The front-end has three deployment modes: automated deployment, manual deploymen

#### Download the installation package

Please download the latest version of the installation package, download address: [download](https://dist.apache.org/repos/dist/dev/incubator/dolphinscheduler)
Please download the latest version of the installation package, download address: [download](https://dolphinscheduler.apache.org/en-us/docs/user_doc/download.html)

After downloading apache-dolphinscheduler-incubating-x.x.x-dolphinscheduler-front-bin.tar.gz,
decompress`tar -zxvf apache-dolphinscheduler-incubating-x.x.x-dolphinscheduler-front-bin.tar.gz ./`and enter the`dolphinscheduler-ui`directory
@@ -22,18 +22,6 @@ Automated deployment is recommended for either of the following two ways

### Automated Deployment

Edit the installation file`vi install-dolphinscheduler-ui.sh` in the` dolphinscheduler-ui` directory

Change the front-end access port and the back-end proxy interface address

```
# Configure the front-end access port
esc_proxy="8888"
# Configure proxy back-end interface
esc_proxy_port="http://192.168.xx.xx:12345"
```

>Front-end automatic deployment based on Linux system `yum` operation, before deployment, please install and update`yum`
under this directory, execute`./install-dolphinscheduler-ui.sh`
File renamed without changes.
File renamed without changes.
@@ -9,11 +9,11 @@

## 3. Download the new version of the installation package

- [github](https://github.com/apache/incubator-dolphinscheduler/releases), download the latest version of the front and back installation packages (backend referred to as dolphinscheduler-backend, front end referred to as dolphinscheduler-ui)
- [download](https://dolphinscheduler.apache.org/en-us/docs/user_doc/download.html), download the latest version of the front and back installation packages (backend referred to as dolphinscheduler-backend, front end referred to as dolphinscheduler-front)
- The following upgrade operations need to be performed in the new version of the directory

## 4. Database upgrade
- Modify the following properties in conf/dao/data_source.properties
- Modify the following properties in conf/application-dao.properties

```
spring.datasource.url
@@ -0,0 +1,11 @@
# How to become Dolphinscheduler committer

Anyone can be a contributor to an Apache project. Being a contributor simply means that you take an interest in the project and contribute in some way, ranging from asking sensible questions (which documents the project and provides feedback to developers) through to providing new features as patches.

If you become a valuable contributor to the project you may well be invited to become a committer. Committer is a term used at the ASF to signify someone who is committed to a particular project. It brings with it the privilege of write access to the project repository and resources.

In Dolphinscheduler(incubating) community, if a committer who have earned even more merit, can be invited to be a part of the Podling Project Management Committee (PPMC).

One thing that is sometimes hard to understand when you are new to the open development process used at the ASF, is that we value the community more than the code. A strong and healthy community will be respectful and be a fun and rewarding place. More importantly, a diverse and healthy community can continue to support the code over the longer term, even as individual companies come and go from the field.

More details could be found [here](https://community.apache.org/contributors/).
@@ -1,3 +1,63 @@
#### how to submit code
#### How to submit

* First from the remote repository *https://github.com/apache/incubator-dolphinscheduler.git* fork a copy of the code into your own repository

* There are currently three branches in the remote repository:
* master normal delivery branch
After the stable release, merge the code from the stable branch into the master.

* dev daily development branch
Every day dev development branch, newly submitted code can pull request to this branch.


* Clone your repository to your local
`git clone https://github.com/apache/incubator-dolphinscheduler.git`

* Add remote repository address, named upstream
`git remote add upstream https://github.com/apache/incubator-dolphinscheduler.git`

* View repository
    `git remote -v`

>At this time, there will be two repositories: origin (your own repository) and upstream (remote repository)
* Get/Update remote repository code
    `git fetch upstream`

* Synchronize remote repository code to local repository

```
git checkout origin/dev
git merge --no-ff upstream/dev
```

If remote branch has a new branch such as `dev-1.0`, you need to synchronize this branch to the local repository

```
git checkout -b dev-1.0 upstream/dev-1.0
git push --set-upstream origin dev-1.0
```

* After modifying the code locally, submit it to your own repository:

`git commit -m 'commit content'`

`git push`

* Submit changes to the remote repository

* On the github page, click New pull request.
<p align = "center">
<img src = "http://geek.analysys.cn/static/upload/221/2019-04-02/90f3abbf-70ef-4334-b8d6-9014c9cf4c7f.png" width = "60%" />
</p>

* Select the modified local branch and the branch you want to merge with the past, Create pull request.
<p align = "center">
<img src = "http://geek.analysys.cn/static/upload/221/2019-04-02/fe7eecfe-2720-4736-951b-b3387cf1ae41.png" width = "60%" />
</p>

* Then the community Committers will do CodeReview, and then he will discuss some details (including design, implementation, performance, etc.) with you. When everyone on the team is satisfied with this modification, the commit will be merged into the dev branch

* Finally, congratulations, you have become an official contributor to dolphinscheduler!


TODO
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -4,7 +4,7 @@

## 1、准备工作

请下载最新版本的安装包,下载地址: [下载](https://dist.apache.org/repos/dist/dev/incubator/dolphinscheduler)
请下载最新版本的安装包,下载地址: [下载](https://dolphinscheduler.apache.org/en-us/docs/user_doc/download.html)

#### 准备一: 基础软件安装(必装项请自行安装)

@@ -90,7 +90,7 @@ install.sh : 一键部署脚本

- 修改部署参数(根据自己服务器及业务情况):

- 修改 **install.sh**中的各参数,替换成自身业务所需的值
- 修改 `install.sh`中的各参数,替换成自身业务所需的值
- monitorServerState 开关变量,在1.0.3版本中增加,控制是否启动自启动脚本(监控master,worker状态,如果掉线会自动启动)
默认值为"false"表示不启动自启动脚本,如果需要启动改为"true"

@@ -102,9 +102,9 @@ install.sh : 一键部署脚本


## 2、部署
以下两种方式任选其一部署即可,推荐自动化部署,有经验的小伙伴也可以使用源码部署
以下两种方式任选其一部署即可,推荐二进制文件部署,有经验的小伙伴也可以使用源码部署

### 2.1 自动部署
### 2.1 二进制文件部署

- 安装zookeeper工具

@@ -141,22 +141,44 @@ install.sh : 一键部署脚本

将源码包release版本下载后,解压进入根目录

* 执行编译命令:
* 编译生成tar包

```
mvn -U clean package -Prelease -Dmaven.test.skip=true
```
tar包的好处是解压即可安装

执行编译命令:

* 查看目录
```
mvn -U clean package -Prelease -Dmaven.test.skip=true
```

正常编译完后,会在 `dolphinscheduler-dist/dolphinscheduler-backend/target`目录下生成
`apache-dolphinscheduler-incubating-${latest.release.version}-dolphinscheduler-backend-bin.tar.gz`
查看目录

正常编译完后,会在 `dolphinscheduler-dist/dolphinscheduler-backend/target`目录下生成
`apache-dolphinscheduler-incubating-${latest.release.version}-dolphinscheduler-backend-bin.tar.gz`


* 或者编译生成rpm包

rpm包可以在linux平台使用rpm命令或者yum命令来安装,rpm包可以用来帮助Dolphinscheduler更方便的集成到其它管理工具,比如Ambari,Cloudera Manager等。

执行编译命令:

```
mvn -U clean package -Prpmbuild -Dmaven.test.skip=true
```

查看目录

正常编译完后,会在 `dolphinscheduler-dist/target/rpm/apache-dolphinscheduler-incubating/RPMS/noarch/`目录下生成
`apache-dolphinscheduler-incubating-${latest.release.version}-1.noarch.rpm`

* 解压编译好的tar.gz包或者使用rpm命令安装后(rpm的安装方式会将dolphinscheduler安装在/opt/soft目录下)dolphinscheduler目录结构如下:

```查看目录
../
├── bin
├── conf
|── DISCLAIMER
|── DISCLAIMER-WIP
|—— install.sh
|—— lib
|—— LICENSE
@@ -165,7 +187,13 @@ install.sh : 一键部署脚本
|—— script
|—— sql
```
- 安装zookeeper工具

`pip install kazoo`

- 切换到部署用户,一键部署

`sh install.sh`

### 2.3 系统常用启停服务(服务用途请具体参见《系统架构设计》小节)

0 comments on commit fbc2693

Please sign in to comment.