Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[release] Prepare for release Linkis-1.0.2 #983

Merged
merged 52 commits into from
Sep 2, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
b2b554b
first commit
Aug 4, 2021
6481e1d
添加once-computation-client的代码
Aug 4, 2021
d56411b
修改flink的版本
Aug 4, 2021
33c0c09
添加onceengine的相关代码
Aug 4, 2021
55ae0c0
flink代码的相关修改
Aug 5, 2021
6cf4246
添加flink引擎的相关的license
Aug 5, 2021
50639ea
添加once引擎client的相关的license
Aug 5, 2021
4435e90
支持并发引擎
Aug 5, 2021
59a6e23
优化flink引擎的相关内容
Aug 5, 2021
8ae1a88
Merge branch 'master' of git.weoa.com:cooperyang/mylinkis
Aug 5, 2021
f86df56
HIVE3.0 编译报错,使用WritableComparable代替具体的实现类接受返回,编译通过
Aug 6, 2021
d0a28ed
移除proxyuser
demonray Aug 9, 2021
e5fef28
remove useless code
demonray Aug 9, 2021
c833d4d
Merge pull request #945 from demonray/dev-1.0.2
peacewong Aug 9, 2021
7ea30a0
#946
lunescode Aug 9, 2021
36607e3
#946
lunescode Aug 9, 2021
9b3c2ad
merged dev-1.0.2
Fuu3214 Aug 10, 2021
1fde192
modified gitignore
Fuu3214 Aug 10, 2021
542dd22
added English language comments
Fuu3214 Aug 10, 2021
e26a62f
added English language comments
Fuu3214 Aug 10, 2021
5816180
fixed a bug that one thread per concurrent-task rather than per concu…
Fuu3214 Aug 10, 2021
45f961b
1. fixed a small bug
Fuu3214 Aug 12, 2021
d1db126
Update AMConfiguration.scala
peacewong Aug 12, 2021
b2578fe
Update YarnSessionClusterDescriptorAdapter.scala
peacewong Aug 12, 2021
1eacd7b
Merge pull request #951 from Fuu3214/dev-1.0.2
peacewong Aug 12, 2021
907d971
Merge pull request #943 from zosimer/dev-1.0.2
peacewong Aug 15, 2021
15e44f9
add configuration for admin password, use admin username as default
FireFoxAhri Aug 15, 2021
c8cec90
Merge pull request #925 from FireFoxAhri/dev-1.1.0-924
peacewong Aug 15, 2021
1f3f9a5
in ecm plugin Decide eurekaPreferIp by judging a host is valid ip , n…
wanghualei Aug 15, 2021
ab494e1
in ecm plugin Decide eurekaPreferIp by judging a host is valid ip , n…
wanghualei Aug 15, 2021
0d8b93f
use EUREKA_PREFER_IP env.
wanghualei Aug 15, 2021
460cc41
Update linkis-env.sh
peacewong Aug 16, 2021
5221230
Merge pull request #950 from lunescode/dev-1.0.2
peacewong Aug 16, 2021
3daab43
Optimize the code compatibility of flink EngineConn, and optimize the…
wushengyeyouya Aug 16, 2021
3867bef
Merge pull request #960 from wushengyeyouya/dev-1.0.2
peacewong Aug 16, 2021
75f92f7
Solve compilation problems
peacewong Aug 16, 2021
fa45b8c
Solve compilation problems
peacewong Aug 16, 2021
b3f0ef8
Merge pull request #961 from peacewong/dev-1.0.2
Aug 16, 2021
f404402
Optimize BML default download user default jvm user
peacewong Aug 17, 2021
0850689
Merge pull request #963 from peacewong/dev-1.0.2
leeebai Aug 17, 2021
917f743
Solve Spark version compatibility issues
peacewong Aug 19, 2021
ba6eb88
Solve Hive version compatibility issues
peacewong Aug 19, 2021
d87a821
Merge pull request #966 from peacewong/dev-1.0.2
Alexkun Aug 19, 2021
ae51cd5
remove instance-label client dependency
peacewong Aug 19, 2021
99efa0a
exclude pentaho-aggdesigner-algorithm module
peacewong Aug 19, 2021
1525eb4
Solve the host name and ip judgment abnormality in the gateway router
peacewong Aug 20, 2021
93e6cb9
Merge pull request #967 from peacewong/dev-1.0.2
Alexkun Aug 20, 2021
5466d57
Optimize computation client module, add some test classes for new cli…
wushengyeyouya Aug 25, 2021
fc0b272
Optimize computation client module, add some test classes for new cli…
wushengyeyouya Aug 25, 2021
057f220
Merge pull request #974 from wushengyeyouya/dev-1.0.2
peacewong Aug 26, 2021
02ee70f
Update FlinkOnceJobTest.java
Aug 30, 2021
92a1608
Merge pull request #981 from WeBankFinTech/Fuu3214-dev-1.0.2
peacewong Aug 30, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
21 changes: 21 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@ linkis-computation-governance/linkis-engineconn/linkis-engineconn-executor/acces
linkis-computation-governance/linkis-engineconn/linkis-engineconn-executor/callback-service/target/
linkis-computation-governance/linkis-engineconn/linkis-engineconn-executor/executor-core/target/
linkis-computation-governance/linkis-engineconn/linkis-engineconn-executor/resource-executor/target/
linkis-computation-governance/linkis-engineconn/linkis-clustered-engineconn/linkis-once-engineconn/target/
linkis-computation-governance/linkis-engineconn/linkis-engineconn-core/target/


linkis-computation-governance/linkis-entrance-client/target/
linkis-computation-governance/linkis-entrance/target/
linkis-computation-governance/linkis-manager/label-common/target/
Expand Down Expand Up @@ -78,6 +82,15 @@ linkis-public-enhancements/linkis-publicService/linkis-script-dev/linkis-storage
linkis-public-enhancements/linkis-publicService/linkis-udf/target/
linkis-public-enhancements/linkis-publicService/linkis-variable/target/
linkis-public-enhancements/linkis-publicService/target/
linkis-public-enhancements/linkis-publicservice/linkis-error-code/linkis-error-code-client/target/
linkis-public-enhancements/linkis-publicservice/linkis-error-code/linkis-error-code-common/target/
linkis-public-enhancements/linkis-publicservice/linkis-error-code/linkis-error-code-server/target/
linkis-public-enhancements/linkis-publicservice/linkis-instance-label/linkis-instance-label-client/target/
linkis-public-enhancements/linkis-publicservice/linkis-instance-label/linkis-instance-label-server/target/
linkis-public-enhancements/linkis-publicservice/linkis-udf/linkis-udf-client/target/
linkis-public-enhancements/linkis-publicservice/linkis-udf/linkis-udf-common/target/
linkis-public-enhancements/linkis-publicservice/linkis-udf/linkis-udf-service/target/



linkis-spring-cloud-services/linkis-service-discovery/linkis-eureka/target/
Expand All @@ -100,10 +113,18 @@ linkis-engineconn-plugins/engineconn-plugins/python/target/
linkis-engineconn-plugins/engineconn-plugins/spark/target/
linkis-engineconn-plugins/engineconn-plugins/pipeline/target/
linkis-engineconn-plugins/engineconn-plugins/shell/target/
linkis-engineconn-plugins/engineconn-plugins/flink/target/

linkis-orchestrator/plugin/linkis-orchestrator-ecm-plugin/target/
linkis-orchestrator/linkis-code-orchestrator/target/
linkis-orchestrator/linkis-computation-orchestrator/target/
linkis-orchestrator/linkis-orchestrator-core/target/

assembly/target
assembly/public-module/target/
assembly-combined-package/assembly-combined/public-module-combined/target/
assembly-combined-package/assembly-combined/target/
assembly-combined-package/target/

linkis-extensions/linkis-io-file-client/target/

4 changes: 2 additions & 2 deletions assembly-combined-package/assembly-combined/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<parent>
<groupId>com.webank.wedatasphere.linkis</groupId>
<artifactId>linkis</artifactId>
<version>1.0.1</version>
<version>1.0.2</version>
</parent>
<modelVersion>4.0.0</modelVersion>

Expand Down Expand Up @@ -75,4 +75,4 @@
</plugins>
</build>

</project>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<parent>
<artifactId>linkis</artifactId>
<groupId>com.webank.wedatasphere.linkis</groupId>
<version>1.0.1</version>
<version>1.0.2</version>
</parent>
<modelVersion>4.0.0</modelVersion>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,12 @@ then
export SERVER_JAVA_OPTS=" -DserviceName=$SERVER_NAME -Xmx$SERVER_HEAP_SIZE -XX:+UseG1GC -Xloggc:$LINKIS_LOG_DIR/${SERVER_NAME}-gc.log $DEBUG_CMD "
fi

#SPRING_ARGS
if [ "$EUREKA_PREFER_IP" == "true" ]; then
SPRING_ARGS="$SPRING_ARGS --eureka.instance.prefer-ip-address=true "
SPRING_ARGS="$SPRING_ARGS --eureka.instance.instance-id=\${spring.cloud.client.ip-address}:\${spring.application.name}:\${server.port}"
fi

## conf dir
export SERVER_CONF_PATH=$LINKIS_CONF_DIR

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,14 @@ fi

export SERVER_CLASS=com.webank.wedatasphere.linkis.gateway.springcloud.LinkisGatewayApplication


#SPRING_ARGS
if [ "$EUREKA_PREFER_IP" == "true" ]; then
SPRING_ARGS="$SPRING_ARGS --eureka.instance.prefer-ip-address=true "
SPRING_ARGS="$SPRING_ARGS --eureka.instance.instance-id=\${spring.cloud.client.ip-address}:\${spring.application.name}:\${server.port} "
fi


## conf dir
export SERVER_CONF_PATH=$LINKIS_CONF_DIR

Expand Down
6 changes: 3 additions & 3 deletions assembly-combined-package/config/linkis-env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ SPARK_CONF_DIR=/appcom/config/spark-config
### You can access it in your browser at the address below:http://${EUREKA_INSTALL_IP}:${EUREKA_PORT}
#EUREKA_INSTALL_IP=127.0.0.1 # Microservices Service Registration Discovery Center
EUREKA_PORT=20303
EUREKA_PREFER_IP=true
export EUREKA_PREFER_IP=false

### Gateway install information
#GATEWAY_INSTALL_IP=127.0.0.1
Expand Down Expand Up @@ -130,7 +130,7 @@ export SERVER_HEAP_SIZE="512M"
##The decompression directory and the installation directory need to be inconsistent
LINKIS_HOME=/appcom/Install/LinkisInstall

LINKIS_VERSION=1.0.1
LINKIS_VERSION=1.0.2

# for install
LINKIS_PUBLIC_MODULE=lib/linkis-commons/public-module
LINKIS_PUBLIC_MODULE=lib/linkis-commons/public-module
2 changes: 1 addition & 1 deletion assembly-combined-package/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<parent>
<groupId>com.webank.wedatasphere.linkis</groupId>
<artifactId>linkis</artifactId>
<version>1.0.1</version>
<version>1.0.2</version>
</parent>
<modelVersion>4.0.0</modelVersion>

Expand Down
2 changes: 1 addition & 1 deletion linkis-commons/linkis-common/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<parent>
<artifactId>linkis</artifactId>
<groupId>com.webank.wedatasphere.linkis</groupId>
<version>1.0.1</version>
<version>1.0.2</version>
</parent>
<modelVersion>4.0.0</modelVersion>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ object Configuration extends Logging {

val CLOUD_CONSOLE_VARIABLE_SPRING_APPLICATION_NAME = CommonVars("wds.linkis.console.variable.application.name", "linkis-ps-publicservice")

//read from env
val EUREKA_PREFER_IP = CommonVars("EUREKA_PREFER_IP", false).getValue

def getGateWayURL(): String = {
val url = GATEWAY_URL.getValue.trim
Expand Down
2 changes: 1 addition & 1 deletion linkis-commons/linkis-hadoop-common/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<parent>
<artifactId>linkis</artifactId>
<groupId>com.webank.wedatasphere.linkis</groupId>
<version>1.0.1</version>
<version>1.0.2</version>
</parent>
<modelVersion>4.0.0</modelVersion>

Expand Down
2 changes: 1 addition & 1 deletion linkis-commons/linkis-httpclient/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<parent>
<artifactId>linkis</artifactId>
<groupId>com.webank.wedatasphere.linkis</groupId>
<version>1.0.1</version>
<version>1.0.2</version>
</parent>
<modelVersion>4.0.0</modelVersion>

Expand Down
2 changes: 1 addition & 1 deletion linkis-commons/linkis-message-scheduler/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<parent>
<artifactId>linkis</artifactId>
<groupId>com.webank.wedatasphere.linkis</groupId>
<version>1.0.1</version>
<version>1.0.2</version>
<relativePath>../../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion linkis-commons/linkis-module/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<parent>
<artifactId>linkis</artifactId>
<groupId>com.webank.wedatasphere.linkis</groupId>
<version>1.0.1</version>
<version>1.0.2</version>
</parent>
<modelVersion>4.0.0</modelVersion>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -139,11 +139,17 @@ private static void addOrUpdateRemoteConfig(Environment env, boolean isUpdateOrN
}

private static void initDWCApplication() {
String hostName = Utils.getComputerName();
boolean eurekaPreferIp = Configuration.EUREKA_PREFER_IP();
if(eurekaPreferIp){
hostName = applicationContext.getEnvironment().getProperty("spring.cloud.client.ip-address");
logger.info("using ip address replace hostname,beacause eureka.instance.prefer-ip-address:" + eurekaPreferIp);
}
serviceInstance = new ServiceInstance();
serviceInstance.setApplicationName(applicationContext.getEnvironment().getProperty("spring.application.name"));
serviceInstance.setInstance(Utils.getComputerName() + ":" + applicationContext.getEnvironment().getProperty("server.port"));
serviceInstance.setInstance(hostName + ":" + applicationContext.getEnvironment().getProperty("server.port"));
LinkisException.setApplicationName(serviceInstance.getApplicationName());
LinkisException.setHostname(Utils.getComputerName());
LinkisException.setHostname(hostName);
LinkisException.setHostPort(Integer.parseInt(applicationContext.getEnvironment().getProperty("server.port")));
}

Expand Down
2 changes: 1 addition & 1 deletion linkis-commons/linkis-mybatis/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<parent>
<artifactId>linkis</artifactId>
<groupId>com.webank.wedatasphere.linkis</groupId>
<version>1.0.1</version>
<version>1.0.2</version>
<relativePath>../../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion linkis-commons/linkis-protocol/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<parent>
<artifactId>linkis</artifactId>
<groupId>com.webank.wedatasphere.linkis</groupId>
<version>1.0.1</version>
<version>1.0.2</version>
</parent>
<modelVersion>4.0.0</modelVersion>

Expand Down
2 changes: 1 addition & 1 deletion linkis-commons/linkis-rpc/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<parent>
<artifactId>linkis</artifactId>
<groupId>com.webank.wedatasphere.linkis</groupId>
<version>1.0.1</version>
<version>1.0.2</version>
</parent>
<modelVersion>4.0.0</modelVersion>

Expand Down
2 changes: 1 addition & 1 deletion linkis-commons/linkis-scheduler/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
<parent>
<artifactId>linkis</artifactId>
<groupId>com.webank.wedatasphere.linkis</groupId>
<version>1.0.1</version>
<version>1.0.2</version>
</parent>
<artifactId>linkis-scheduler</artifactId>

Expand Down
2 changes: 1 addition & 1 deletion linkis-commons/linkis-storage/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<modelVersion>4.0.0</modelVersion>

<parent>
<version>1.0.1</version>
<version>1.0.2</version>
<groupId>com.webank.wedatasphere.linkis</groupId>
<artifactId>linkis</artifactId>
</parent>
Expand Down
2 changes: 1 addition & 1 deletion linkis-commons/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<artifactId>linkis</artifactId>
<groupId>com.webank.wedatasphere.linkis</groupId>
<version>1.0.1</version>
<version>1.0.2</version>
</parent>
<modelVersion>4.0.0</modelVersion>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
<parent>
<groupId>com.webank.wedatasphere.linkis</groupId>
<artifactId>linkis-cli</artifactId>
<version>1.0.1</version>
<version>1.0.2</version>
<relativePath>../pom.xml</relativePath>
</parent>
<artifactId>linkis-cli-application</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
<parent>
<groupId>com.webank.wedatasphere.linkis</groupId>
<artifactId>linkis-cli</artifactId>
<version>1.0.1</version>
<version>1.0.2</version>
<relativePath>../pom.xml</relativePath>
</parent>
<artifactId>linkis-cli-common</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
<parent>
<groupId>com.webank.wedatasphere.linkis</groupId>
<artifactId>linkis-cli</artifactId>
<version>1.0.1</version>
<version>1.0.2</version>
<relativePath>../pom.xml</relativePath>
</parent>
<artifactId>linkis-cli-core</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
<parent>
<artifactId>linkis</artifactId>
<groupId>com.webank.wedatasphere.linkis</groupId>
<version>1.0.1</version>
<version>1.0.2</version>
</parent>
<artifactId>linkis-cli</artifactId>
<packaging>pom</packaging>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
<parent>
<artifactId>linkis</artifactId>
<groupId>com.webank.wedatasphere.linkis</groupId>
<version>1.0.1</version>
<version>1.0.2</version>
</parent>

<artifactId>linkis-computation-client</artifactId>
Expand Down Expand Up @@ -54,6 +54,12 @@
<version>${linkis.version}</version>
</dependency>

<dependency>
<groupId>com.webank.wedatasphere.linkis</groupId>
<artifactId>linkis-bml-client</artifactId>
<version>${linkis.version}</version>
</dependency>

</dependencies>

<build>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.webank.wedatasphere.linkis.computation.client


trait JobListener {

def onJobSubmitted(job: LinkisJob): Unit

def onJobRunning(job: LinkisJob): Unit

def onJobFinished(job: LinkisJob): Unit

def onJobUnknownError(job: LinkisJob, t: Throwable): Unit

}