Releases · apache/linkis

Flink-EngineConn which offers solid support for Flink jobs. Executing, debugging and monitoring Flink SQL or applications are now available, together with SQL-enhancement ability powered by Linkis Orchestrator.
LinkisManagerClient which enables direct access to LinkisManager. Submitting and managing OnceJob rely on this feature.

Abbreviations:

CGS: Computation Governance Services

PES: Public Enhancement Services

MGS: Microservice Governance Services

New Feature

EngineConn

Linkis-936 [CGS-LinkisOnceEngineconn] supports OnceEngineExecutor

EnginePlugin

Linkis-935 [CGS-EngineConnPlugin-Flink] supports Flink EngineConn
Linkis-947 [CGS-EngineConnPlugin-Flink] supports executing Flink SQL and Flink applications
Linkis-948 [CGS-EngineConnPlugin-Flink] multiple-datasource support for Flink EngineConn
Linkis-949 [CGS-EngineConnPlugin-Flink] monitoring Flink Metrics

ComputationClient

Linkis-937 [CGS-LinkisComputationClient] supports OnceEngineExecutor client

Enhancement

Linkis-953 [CGS-LinkisManager] label supports '-' in hostname
Linkis-925 [MGS-LinkisServiceGateway] fix weak password in linkis gateway
Linkis-950 [CGS-LinkisEngineConnManager] support both ip address and hostname for service discovery
Linkis-967 [CGS-LinkisEntrance] remove instance-label client dependency, Solve the host name and ip judgment abnormality in the gateway router, exclude the pom dependency to pentaho-aggdesigner-algorithm jar.
Linkis-963 [PES-LinkisBmlServer] default download user changed to jvm user, and supports to set default download user by configuration.

Bugs Fix

Linkis-938 [CGS-LimkisMnagager] fixes a serial execution bug
Linkis-952 [CGS-LinkisEngineConn] fixes a redundant thread bug
Linkis-943 [CGS-EngineConnPlugin-Hive] fixes a Hive3.0 compilation error
Linkis-961 [CGS-EngineConnPlugin-Flink] fixes a Flink-EnginePlugin compilation bug
Linkis-966 [CGS-EngineConnPlugin-Hive] [CGS-EnginePlugin-Spark] Solve Spark and hive compatibility issue

Credits

The release of Linkis 1.0.2 is inseparable from the contributors of the Linkis community. Thanks to all the community contributors!

Compiled Version

Tencent Cloud:

Linkis-1.0.2 Compiled (.tar.gz)

Assets 2

23 Jul 08:52

sargentti

1.0.1

52970ca

Linkis 1.0.1 release(Not Apache Release)

Linkis-1.0.1 includes all of Project Linkis 1.0.1.
This release mainly fixes performance bugs in high concurrency scenarios, optimizes the resource management module, and optimizes the Spark engine to support the use of FATE (Federated Machine Learning), and fixes multiple discovered bugs.
Abbreviations:
CGS: Computation Governance Services
PES: Public Enhancement Services
MGS: Microservice Governance Services

Enhancement

Linkis-906 [Commons] HTTPClient supports setting contentType when uploading files.
Linkis-897 [Commons] HttpClient supports Response whose content is empty.
Linkis-877 [Commons] HDFS FileSystem object supports caching.
Linkis-862 [Commons] Optimize ShutdownHook to be compatible with windows operating system.
Linkis-889 [Orchestrator] ioClient supports isolating a separate Orchestrator.
Linkis-880 [PES-JobHistory] JobHistory supports recording the original code of the task submitted by the user.
Linkis-879 [PES-InstanceLabel] InstanceLabel module support to modify and remove instance label information.
Linkis-864 [PES-ErrorCode] Optimize the initialization sql of the error code, remove the default value of the ID field.
Linkis-872 [PES-Configuration] Management console configuration supports real-time modification of cached values.
Linkis-844 [PES-Configuration] The management console supports setting memory unit, the default unit is GB.
Linkis-869 [CG-Entrance] Optimize task submission record client IP for link analysis.
Linkis-848 [CG-LimkisMnagager] The tenant label of ECM should be retained to facilitate the restart.
Linkis-863 [EnginePlugin-Spark] Spark EngineConn startup parameters support the setting of native parameters to support the use of Fate, such as setting the python version.

Bugs Fix

Linkis-900 [Commons] Logging is recommended to be optimized, because the user may pass in Null and thus cause an exception.
Linkis-851 [Commons] The version of the jackson-core-asl package is incorrect after linkis1.0 is compiled with the CDH version.
Linkis-896 [PES-UDF] UDFClient has a bug that can only get UDF but cannot get method functions.
Linkis-873 [PES-BML] Optimize the execution order of the BML EngineConnHook to fix the bug of not being able to read resource files.
Linkis-850 [PES-Configuration] Failed to save IDE-JDBC parameter configuration in Linkis1.0 management console.
Linkis-847 [CG-Entrance] When a task contains thousands of SQL, the execution may not be correct.
Linkis-842 [CG-Entrance] Task execution fails when the sql limit setting is greater than 5000.
Linkis-841 [CG-Entrance] Log is not fully recorded in the stress test of 10,000 tasks.
Linkis-840 [CG-Entrance] During the stress test of 10,000 tasks, the task status is in the Inited/Running/Scheduled status and no longer updated.
Linkis-846 [CG-LinkisMnaager] When the parameter of the number of engine is set to 2 in management console, however, 18 engines is started when schedulis is scheduling.
Linkis-845 [CG-LinkisMnaager] The value of concurrent label may be null in ECM management .
Linkis-843 [EnginePlugin-Hive] When killing a complex hive task, Hive MR of Yarn is not simultaneously killed.

Credits

The release of Linkis 1.0.1 is inseparable from the contributors of the Linkis community. Thanks to all the community contributors!

Compiled Version

Tencent Cloud:

Linkis-1.0.1 Compiled (.tar.gz)

Assets 2

29 Jun 15:19

sargentti

1.0.0

da37165

Linkis 1.0.0 release(Not Apache Release)

Linkis-1.0.0 includes all of Project Linkis 1.0.0.

This release mainly brings in Linkis-Orchestrator, optimizing lots of architecture code of 'Computation Governance', adding support for OnceEngineConn, which used for batch jobs and streaming jobs, and simplifies the deployment of Linkis.
Meanwhile, the /docs directory is removed, documents are moved to the Linkis-Doc repository.

New Feature

Orchestrator

Linkis-746 Add the core module of Linkis Orchestrator, and provide top-level architecture and extension interfaces for the Orchestrator module.
Linkis-739 Add the Linkis OLAP scene task Orchestrator module.
Linkis-733 Add linkis-code-orchestrator module to support OLAP job requests.

EngineConn

Linkis-799 Add linkis-once-engineconn module for Linkis1.0 to support for OnceEngineConn, which used for batch jobs and streaming jobs.

Web

Linkis-757 Add Linkis front-end web module, provide Global history, Resource management, Parameter configuration, Global variables and other modules.

Client

Linkis-754 Add Linkis-Cli module, allowing submitting codes through shell command.

Enhancement

Linkis-818 Rename Application EngineConn in Eureka to Linkis-CG-EngineConn to follow the naming convention.
Linkis-807 Enhance the Linkis1.0.0 common module, and provide lots of tools classes for other modules.
Linkis-805 Use Utils.tryCatch to catch exception, instead of try...catch directly.
Linkis-797 Add Chinese and English support for all logs.
Linkis-789 Constant substitution.
Linkis-788 Optimize some magic values in DESUtil class.
Linkis-782 Optimize some magic values in Linkis codes.
Linkis-781 Optimize error code definition.
Linkis-777 Merge publicservice with other publicenhancement microservices, and add the combined packaging method.
Linkis-775 Optimize Linkis computation governance module and make it adapt to Linkis1.0's new architecture.
Linkis-768 Remove all commented code and optimize HttpBmlClient .
Linkis-767 Increase the regularity of the code, remove all @author and @Date at the top of each class.
Linkis-752 Optimize Linkis engineconn module and make it adapt to Linkis1.0's new architecture.
Linkis-751 Optimize Linkis spring cloud services module and make it adapt to Linkis1.0's new architecture.
Linkis-749 Add a new linkis-io-file-client module, modify some comments and add some license.
Linkis-748 Optimize the public-enhancements module and make it adapt to Linkis1.0's new architecture.

Bugs Fix

Linkis-825 Solve the recursive calling problem in SparkPythonExecutor's close method.
Linkis-816 Shutdown all executors instead of only closing the default one for multi-executor EngineConns.
Linkis-815 Fix Orchestrator AsyncExec ResultSet wait didn't get notified bug and remove Orchestrator limit 5000 constraint.
Linkis-804 Fix the bug where "limit 5000" would not be automatically added if the user submit a sql like 'select*'.
Linkis-803 Fix the thread safety issues of SSOUtils, which may cause OOM exception.
Linkis-787 Fix the package conflict problem caused by hive version upgrade.
Linkis-780 Fix the value conversion problem of RM module in capacity scheduler.
Linkis-729 Solve the problem that cookies does not take effect after login, and cookies added to Action is not delivered to gateway by the HttpClient.

Credits

The release of Linkis 1.0.0 is inseparable from the contributors of the Linkis community. Thanks to all the community contributors!

Compiled Version

Tencent Cloud:

Linkis-1.0.0 Compiled (.tar.gz)

Contributors

Date and author

Assets 2

15 Mar 13:51

sargentti

1.0.0-RC1

89f3b4f

Linkis 1.0.0-RC1 release(Not Apache Release)

Linkis 1.0.0-RC1 is an important release that marks the start of the 1.0 version, including all of Project Linkis 1.0.0 RC1.

With the design of supporting full-stack computation/storage engines, this release brings a brand new implemention for the architecture of computation governance, provides a variety of new features and Provides a strong compatibility guarantee for later 1.x version.

New Feature

[Linkis-572] Add the EngineConn management plug-in in the Entrance module.
[Linkis-576] Add a new module EngineConnPlugin for the Linkis1.0 architecture to simplify the user's implementation of a new Linkis underlying computation storage engine.
[Linkis-584] Add a new LinkisManager Common module to collect all commonly used interfaces in LinkisManager.
[Linkis-585] Add a new LinkisManager Monitor module to monitor the status of instances managed by LinkisManager.
[Linkis-587] Add the common module of EngineConn of Linkis1.0 architecture.
[Linkis-591] Add the Common Module for Computation Governance.
[Linkis-600] Add the linkis-manager-service-common module, it can be used by AppManager and ResourceManager.
[Linkis-602] Add the Persistence Module for Linkis Manager to provide the persistence capability.
[Linkis-604] Add the ability of microservice tagging, and support the tagging of each micro-service.
[Linkis-599] Add the Linkis ResourceManager Common Module.
[Linkis-609] Add a universal label module to provide basic support for linkis label capability.
[Linkis-630] Add the launch module of EngineConn of Linkis1.0 architecture.

Enhancement

[Linkis-569] Enhance the processing capacity of Linkis RPC, and backward compatibility with Linkis0.x RPC.
[Linkis-570] Enhance the ability of RM and integrate it into Linkis Manager.
[Linkis-574] Enhance the label management function of Linkis, so it can manage the Label of EngineConnManager and EngineConn.
[Linkis-580] Enhance SparkEngineConn module, adapts to Linkis1.0's new architecture.
[Linkis-583] Enhance ShellEngine, adapts to Linkis1.0's new architecture.
[Linkis-590] Enhance the core module of EngineConn, provide the abstractions and interfaces of the core functions.
[Linkis-592] Enhance JDBCEngine, adapts to Linkis1.0's new architecture.
[Linkis-594] Enhance IOEngine, adapts to Linkis1.0's new architecture.
[Linkis-596] Enhance HiveEngine, adapts to Linkis1.0's new architecture.
[Linkis-597] Enhance PipelineEngine, adapts to Linkis1.0's new architecture.
[Linkis-608] Enhance scritps for starting and shutting down services, optimize the steps of construction.
[Linkis-618] Enhance the Context Service module of Public Enhancements and adapt it with Linkis 1.0.
[Linkis-638] Enhance the encryption function of linkis-common.
[Linkis-643] Enhance the Linkis DataSource Module to adapt to the structure of Linkis 1.0.
[Linkis-645] Enhance the linkis-hadoop module to support the new feature of multi hadoop cluster.
[Linkis-648] Enhance linkis-httpclient module to support PUT and Delete requests.
[Linkis-650] Optimize linkis-mybatis module to integrate with mybatis-plus.
[Linkis-652] Enhance the module of linkis-protocol, adapts to Linkis1.0's new architecture.
[Linkis-654] Enhance the Linkis Gateway Module to support forwarding requests by labels in Linkis 1.0.
[Linkis-655] Enhance the Linkis JobHistory Module to support task cache, label persistent and manager of job history.
[Linkis-656] Optimize the implementation of the serializer of the linkis-rpc module.
[Linkis-660] Enhance the linkis-scheduler module to automatically clean up idle Consumers.
[Linkis-662] Enhance the linkis-user-control Module, adapts to Linkis1.0's new architecture.
[Linkis-663] Enhance the linkis-entrance module, adapts to Linkis1.0's new architecture.
[Linkis-666] Optimize directory hierarchy structures and remove useless directories.
[Linkis-668] Enhance the linkis-storage module to support viewFS and multi hadoop cluster.
[Linkis-674] Enhance the PythonEngineConn, adatps to Linkis1.0 architecture.
[Linkis-675] Optimize directories and files, update the version of all pom files.

Credits

The release of Linkis 1.0.0-RC1 is inseparable from the contributors of the WeDataSphere community. They selflessly contributed their time and made lots of valueable suggestions for the document construction, deployment simplification and user experience perfection. Without their help, Linkis 1.0.0-RC1 can not be released so smoothly. Thank you all Contributors of the community!

Upgrade Wizard

In principle, Linkis 1.0.0-RC1 is backward compatible with all the interfaces of Linkis0.X, so the upgrade has little effect to the upper layer applications.
As a computation middleware, except for the related user configuration data of PublicService, other data in the Linkis database has no need for migration.
For the migration of the user configuration data of PublicService, please refer to: Linkis1.0.0-RC1 Upgrade Document

Cloud Resource

Tencent Cloud:

URL: https://osp-1257653870.cos.ap-guangzhou.myqcloud.com/WeDatasphere/Linkis/1.0.0-RC1/wedatasphere-linkis-1.0.0-RC1-dist.tar.gz

Assets 2

21 Jan 08:29

wForget

0.11.0

c5dbcb1

Linkis 0.11.0 release(Not Apache Release)

0.11.0 is an important version led by MobTech with the help of WeBank.
This is the next release of Linkis 0.10.0 base line. It contains 20 bug fixes, improvements and enhancements since 0.10.0.

Enhancement

[Linkis-363] Elasticsearch Engine Support for Linkis.
[Linkis-380] Presto Engine Support for Linkis.
[Linkis-PR-381] Support Presto resource management
[Linkis-294] Support spark cluster deploy mode.
[Linkis-322] Support custom spark configuration.
[Linkis-353] Obtain yarn queues with permission.
[Linkis-PR-448] Gateway adds routing rules based on datasource.
[Linkis-323] The Spark engine log prints ApplicationID.
[Linkis-PR-348] Support loop replacement in parameter variable replacement.
[Linkis-PR-512] PySpark tasks adapt to Spark3.0.
[Linkis-PR-533] Shell engine supports custom variables.
[Linkis-PR-546] Shell engine supports output result set.

Bug Fix

[Linkis-187] Fix the problem of resource moreThan judgment logic.
[Linkis-320] Fixed an abnormal problem when the name is too long during LDAP authentication.
[Linkis-454] Fix CDH Spark version acquisition problem.
[Linkis-344] Fix the problem that different users of JDBC tasks affect each other.
[Linkis-361] Fix the problem of JDBC engine writing result set.
[Linkis-PR-348] Fix the java.library.path configuration in HiveQLProcessBuilder.
[Linkis-PR-348] Fix the problem of sharing functions to the specified directory.
[Linkis-PR-537] Modify the length of the name field in the linkis_udf_tree table.
[Linkis-489] Fix the problem of user cookie loss after accessing single sign-on.
[Linkis-PR-538] Fix the bug of token verification when the third-party system integrates linkis and uses token authentication.

Credits

The release of Linkis 0.11.0 is inseparable from the contributors of the WeDataSphere community. They selflessly contribute their codes and actively carry out technical exchanges with community partners. With their help, Linkis 0.11.0 can be successfully released. Thank you all Contributors to the community! Rank in no particular order:
wForget: Implement elasticsearch engine and gateway routing plugin , implement other enhancements and fix some bugs.
Yogaflre: Implement presto engine and presto resource management.
77954309: Shell engine supports custom variables.
xfei6868: PySpark tasks adapt to Spark3.0.
hcl3039359: Modify the length of the name field in the linkis_udf_tree table.
linweijiang: Fix the problem of user cookie loss after accessing single sign-on.
cumtcwf: Fix the bug of token verification.
zhangxhmuye: Support shell engine result set output.

0.11.0 是在微众银行的倾力帮助下，由 MobTech 主导完成的一个重要版本。
这是基于 Linkis 0.10.0 的下一个发行版本。自 0.10.0 起，本次版本包含了 20 个Bug修复，改进和增强。

特性增强

[Linkis-363] 新增 ElasticSearch 引擎支持。
[Linkis-380] 新增 Presto 引擎支持。
[Linkis-PR-381] 支持 Presto 资源控制
[Linkis-294] 支持 Spark Cluster 模式。
[Linkis-322] 支持自定义 Spark 配置。
[Linkis-353] 支持获取有权限的Yarn队列。
[Linkis-PR-448] 网关添加基于 DataSource 路由规则
[Linkis-323] Spark引擎日志打印ApplicationID。
[Linkis-PR-348] 在参数变量替换时支持循环替换
[Linkis-PR-512] PySpark 任务适配 Spark3.0。
[Linkis-PR-533] Shell 引擎支持自定义变量
[Linkis-PR-546] Shell 引擎支持输出结果集

Bug修复

[Linkis-187] 修复 Resource moreThan 判断逻辑问题。
[Linkis-320] 修复 LDAP 认证时，名称太长导致异常问题。
[Linkis-454] 修复 CDH Spark 版本获取问题。
[Linkis-344] 修复 JDBC 任务不同用户相互影响的问题。
[Linkis-361] 修复 JDBC 引擎写结果集问题。
[Linkis-PR-348] 修复 HiveQLProcessBuilder 中 java.library.path 配置。
[Linkis-PR-348] 修复共享函数到指定目录问题。
[Linkis-PR-537] 修改 linkis_udf_tree 表中 name 字段长度。
[Linkis-489] 修复接入单点登录后用户 cookie 丢失问题。
[Linkis-PR-538] 修复第三方系统集成 linkis，使用 Token 认证方式时 Token 校验的 Bug。

新贡献者

Linkis 0.11.0 的发布，离不开WeDataSphere社区的贡献者，他们无私地贡献自己的代码，积极地与社区伙伴进行技术交流，有了他们的助力，Linkis 0.11.0 才能顺利地发布，在此感谢各位社区的贡献者! 排名不分先后:
wForget: 实现 ElasticSearch 引擎，实现网关路由插件，其他若干功能实现以及BUG修复。
Yogaflre: 实现 Presto 引擎，Presto 资源控制。
77954309: Shell 引擎支持自定义变量。
xfei6868: PySpark 任务适配 Spark3.0。
hcl3039359: 修改 linkis_udf_tree 表中 name 字段长度。
linweijiang: 修复接入单点登录后用户 cookie 丢失问题。
cumtcwf: 修复 Token 校验的 Bug。
zhangxhmuye: 支持 Shell 引擎结果集输出。

云资源

以下为Linkis安装包资源：
https://osp-1257653870.cos.ap-guangzhou.myqcloud.com/WeDatasphere/Linkis/0.11.0/wedatasphere-linkis-0.11.0-dist.tar.gz

Assets 3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New Feature

EngineConn

EnginePlugin

ComputationClient

Enhancement

Bugs Fix

Credits

Compiled Version

Enhancement

Bugs Fix

Credits

Compiled Version

New Feature

Orchestrator

EngineConn

Web

Client

Enhancement

Bugs Fix

Credits

Compiled Version

Contributors

New Feature

Enhancement

Credits

Upgrade Wizard

Cloud Resource

Enhancement

Bug Fix

Credits

特性增强

Bug修复

新贡献者

云资源

Releases: apache/linkis

Apache Linkis(incubating) Release-1.1.2-Incubating-RC2

Apache Linkis(incubating) Release-1.1.2-Incubating-RC1

Apache Linkis(incubating) Release-1.1.1

Apache Linkis(incubating) Release-1.1.0

Apache Linkis(incubating) Release-1.0.3-Incubating

Linkis 1.0.2 release(Not Apache Release)

New Feature

EngineConn

EnginePlugin

ComputationClient

Enhancement

Bugs Fix

Credits

Compiled Version

Linkis 1.0.1 release(Not Apache Release)

Enhancement

Bugs Fix

Credits

Compiled Version

Linkis 1.0.0 release(Not Apache Release)

New Feature

Orchestrator

EngineConn

Web

Client

Enhancement

Bugs Fix

Credits

Compiled Version

Contributors

Linkis 1.0.0-RC1 release(Not Apache Release)

New Feature

Enhancement

Credits

Upgrade Wizard

Cloud Resource

Linkis 0.11.0 release(Not Apache Release)

Enhancement

Bug Fix

Credits

特性增强

Bug修复

新贡献者

云资源