Skip to content

Commit

Permalink
Merge branch 'apache:master' into master
Browse files Browse the repository at this point in the history
  • Loading branch information
lujx98 committed Sep 28, 2023
2 parents 017f23c + 6e8d44f commit a7a55ff
Show file tree
Hide file tree
Showing 69 changed files with 655 additions and 677 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/nightly-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:
restore-keys: |
${{ needs.global-environment.outputs.GLOBAL_CACHE_PREFIX }}-maven-third-party-
- name: Run SpotBugs
run: ./mvnw clean install spotbugs:check -T1C
run: ./mvnw clean install spotbugs:check -Pcheck -T1C

check-and-report-sonarcloud:
if: github.repository == 'apache/shardingsphere'
Expand Down Expand Up @@ -82,7 +82,7 @@ jobs:
name: Checkstyle - SonarCloud
needs: global-environment
runs-on: ubuntu-latest
timeout-minutes: 45
timeout-minutes: 60
steps:
- uses: actions/checkout@v3
- uses: actions/setup-java@v3
Expand Down
4 changes: 1 addition & 3 deletions distribution/proxy-native/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@

<properties>
<imageName>apache-shardingsphere-proxy-native</imageName>
<mainClass>org.apache.shardingsphere.proxy.Bootstrap</mainClass>
<native.image.repository>apache/shardingsphere-proxy-native</native.image.repository>
</properties>

Expand Down Expand Up @@ -103,8 +102,7 @@
<version>${native-maven-plugin.version}</version>
<extensions>true</extensions>
<configuration>
<fallback>false</fallback>
<verbose>true</verbose>
<mainClass>org.apache.shardingsphere.proxy.Bootstrap</mainClass>
<buildArgs>
<arg>--report-unsupported-elements-at-runtime</arg>
<arg>-J-Xmx7g</arg>
Expand Down
9 changes: 5 additions & 4 deletions docs/document/content/dev-manual/sharding.cn.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,8 @@ chapter = true

### 已知实现

| *配置标识* | *详细说明* | *全限定类名* |
|---------|-------------------|------------------------------------------------------------------------------|
| GROOVY | 使用 Groovy 语法的行表达式 | `org.apache.shardingsphere.infra.expr.groovy.GroovyInlineExpressionParser` |
| LITERAL | 使用标准列表的行表达式 | `org.apache.shardingsphere.infra.expr.literal.LiteralInlineExpressionParser` |
| *配置标识* | *详细说明* | *全限定类名* |
|----------|----------------------------------------------------|--------------------------------------------------------------------------------|
| GROOVY | 使用 Groovy 语法的行表达式 | `org.apache.shardingsphere.infra.expr.groovy.GroovyInlineExpressionParser` |
| LITERAL | 使用标准列表的行表达式 | `org.apache.shardingsphere.infra.expr.literal.LiteralInlineExpressionParser` |
| ESPRESSO | 基于 GraalVM Truffle 的 Espresso 实现的使用 Groovy 语法的行表达式 | `org.apache.shardingsphere.infra.expr.espresso.EspressoInlineExpressionParser` |
9 changes: 5 additions & 4 deletions docs/document/content/dev-manual/sharding.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,8 @@ Row Value Expressions definition

### Implementation classes

| *Configuration Type* | *Description* | *Fully-qualified class name* |
|----------------------|---------------------------------------------------|------------------------------------------------------------------------------|
| GROOVY | Row Value Expressions that uses the Groovy syntax | `org.apache.shardingsphere.infra.expr.groovy.GroovyInlineExpressionParser` |
| LITERAL | Row Value Expressions that uses a standard list | `org.apache.shardingsphere.infra.expr.literal.LiteralInlineExpressionParser` |
| *Configuration Type* | *Description* | *Fully-qualified class name* |
|----------------------|------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------|
| GROOVY | Row Value Expressions that uses the Groovy syntax | `org.apache.shardingsphere.infra.expr.groovy.GroovyInlineExpressionParser` |
| LITERAL | Row Value Expressions that uses a standard list | `org.apache.shardingsphere.infra.expr.literal.LiteralInlineExpressionParser` |
| ESPRESSO | Row Value Expressions that uses the Groovy syntax based on GraalVM Truffle's Espresso implementation | `org.apache.shardingsphere.infra.expr.espresso.EspressoInlineExpressionParser` |
Original file line number Diff line number Diff line change
@@ -1,23 +1,62 @@
+++
title = "行表达式"
weight = 7
weight = 8
+++

## 使用 Groovy 语法的行表达式

在配置中使用 `${ expression }``$->{ expression }` 标识 Groovy 表达式即可。 Groovy 表达式使用的是 Groovy 的语法,Groovy 能够支持的所有操作,行表达式均能够支持。
`${begin..end}` 表示范围区间; `${[unit1, unit2, unit_x]}` 表示枚举值。
行表达式中如果出现连续多个 `${ expression }``$->{ expression }` 表达式,整个表达式最终的结果将会根据每个子表达式的结果进行笛卡尔组合。

类型:GROOVY

可配置属性
用例

- `<GROOVY>t_order_${1..3}` 将被转化为 `t_order_1, t_order_2, t_order_3`
- `<GROOVY>${['online', 'offline']}_table${1..3}` 将被转化为 `online_table1, online_table2, online_table3, offline_table1, offline_table2, offline_table3`

## 使用标准列表的行表达式

`LITERAL` 实现将不对表达式部分做任何符号的转化,从标准列表的输入直接获得标准列表的输出。此有助于解决 GraalVM Native Image 下不便于使用 Groovy 表达式的问题。

类型:LITERAL

可配置属性:
用例:

- `<LITERAL>t_order_1, t_order_2, t_order_3` 将被转化为 `t_order_1, t_order_2, t_order_3`
- `<LITERAL>t_order_${1..3}` 将被转化为 `t_order_${1..3}`

## 基于 GraalVM Truffle 的 Espresso 实现的使用 Groovy 语法的行表达式

此为可选实现,你需要在自有项目的 `pom.xml` 主动声明如下依赖。并且请确保自有项目通过 GraalVM CE 23.0.1 For JDK17 编译。

```xml
<dependencies>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-infra-expr-espresso</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
</dependencies>
```

用户必须通过 GraalVM Updater 安装 Espresso 组件,即在 bash 执行如下命令

```bash
gu install espresso
```

`ESPRESSO` 仍为实验性模块,其允许在 GraalVM Native Image 下通过 GraalVM Truffle 的 Espresso 实现来使用带 Groovy 语法的行表达式。

语法部分与 `GROOVY` 实现规则相同。

类型:ESPRESSO

用例:

- `<ESPRESSO>t_order_${1..3}` 将被转化为 `t_order_1, t_order_2, t_order_3`
- `<ESPRESSO>${['online', 'offline']}_table${1..3}` 将被转化为 `online_table1, online_table2, online_table3, offline_table1, offline_table2, offline_table3`

## 操作步骤

Expand Down Expand Up @@ -52,7 +91,7 @@ rules:
t_order_inline:
type: INLINE
props:
algorithm-expression: <GROOVY>t_order_${order_id % 2}
algorithm-expression: t_order_${order_id % 2}
keyGenerators:
snowflake:
type: SNOWFLAKE
Expand Down
Original file line number Diff line number Diff line change
@@ -1,23 +1,68 @@
+++
title = "Row Value Expressions"
weight = 7
weight = 8
+++

## Row Value Expressions that uses the Groovy syntax

Type: GROOVY

Attributes:
Just use `${ expression }` or `$->{ expression }` in the configuration to identify the row expressions.
The content of row expressions uses Groovy syntax, and all operations supported by Groovy are supported by row expressions.
`${begin..end}` denotes the range interval, `${[unit1, unit2, unit_x]}` denotes the enumeration value.
If there are multiple `${ expression }` or `$->{ expression }` expressions in a row expression, the final result of the
whole expression will be a Cartesian combination based on the result of each sub-expression.

None
Example:

- `<GROOVY>t_order_${1..3}` will be converted to `t_order_1, t_order_2, t_order_3`
- `<GROOVY>${['online', 'offline']}_table${1..3}` will be converted to `online_table1, online_table2, online_table3, offline_table1, offline_table2, offline_table3`

## Row Value Expressions that uses a standard list

The `LITERAL` implementation will not convert any symbols to the expression part, and will directly obtain the output of
the standard list from the input of the standard list.
This helps address the issue that Groovy expressions are inconvenient to use under GraalVM Native Image.

Type: LITERAL

Attributes:
Example:

- `<LITERAL>t_order_1, t_order_2, t_order_3` will be converted to `t_order_1, t_order_2, t_order_3`
- `<LITERAL>t_order_${1..3}` will be converted to `t_order_${1..3}`

## Row Value Expressions that uses the Groovy syntax based on GraalVM Truffle's Espresso implementation

This is an optional implementation, and you need to actively declare the following dependencies in the `pom.xml` of your own project.
And make sure your own project is compiled with GraalVM CE 23.0.1 For JDK17.

```xml
<dependencies>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-infra-expr-espresso</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
</dependencies>
```

The user must install the Espresso component via GraalVM Updater, i.e. execute the following command in bash

```bash
gu install espresso
```

`ESPRESSO` is still an experimental module that allows the use of Row Value Expressions with Groovy syntax under GraalVM
Native Image through the Espresso implementation of GraalVM Truffle.

The syntax part is the same as the `GROOVY` implementation rules.

Type: ESPRESSO

Example:

None
- `<ESPRESSO>t_order_${1..3}` will be converted to `t_order_1, t_order_2, t_order_3`
- `<ESPRESSO>${['online', 'offline']}_table${1..3}` will be converted to `online_table1, online_table2, online_table3, offline_table1, offline_table2, offline_table3`

## Procedure

Expand Down Expand Up @@ -54,7 +99,7 @@ rules:
t_order_inline:
type: INLINE
props:
algorithm-expression: <GROOVY>t_order_${order_id % 2}
algorithm-expression: t_order_${order_id % 2}
keyGenerators:
snowflake:
type: SNOWFLAKE
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,5 +50,7 @@ ShardingSphere 默认情况下仅包含核心 SPI 的实现,在 Git Source 存
- `org.apache.shardingsphere:shardingsphere-transaction-xa-narayana`,基于 Narayana 的 XA 分布式事务管理器
- SQL 翻译
- `org.apache.shardingsphere:shardingsphere-sql-translator-jooq-provider`,使用 JooQ 的 SQL 翻译器
- 行表达式
- `org.apache.shardingsphere:shardingsphere-infra-expr-espresso`,基于 GraalVM Truffle 的 Espresso 实现的使用 Groovy 语法的行表达式

除了以上可选插件外,ShardingSphere 社区开发者还贡献了大量的插件实现,可以在 [ShardingSphere Plugin](https://github.com/apache/shardingsphere-plugin) 仓库中查看插件的使用说明,ShardingSphere Plugin 仓库中的插件会和 ShardingSphere 保持相同的发布节奏,可以在 https://central.sonatype.com/ 进行检索,并安装到 ShardingSphere 中。
Original file line number Diff line number Diff line change
Expand Up @@ -50,5 +50,7 @@ All optional plugins are listed below in the form of `groupId:artifactId`.
- `org.apache.shardingsphere:shardingsphere-transaction-xa-narayana`, XA distributed transaction manager based on Narayana
- SQL translator
- `org.apache.shardingsphere:shardingsphere-sql-translator-jooq-provider`, JooQ SQL translator
- Row Value Expressions definition
- `org.apache.shardingsphere:shardingsphere-infra-expr-espresso`,Row Value Expressions that uses the Groovy syntax based on GraalVM Truffle's Espresso implementation

In addition to the above optional plugins, ShardingSphere community developers have contributed a number of plugin implementations. These plugins can be found in [ShardingSphere Plugins] (https://github.com/apache/shardingsphere-plugin) repository. Plugins in ShardingSphere Plugin repository would remain the same release plan with ShardingSphere, they can be retrieved at https://central.sonatype.com/, and install into ShardingSphere.
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,3 @@ ShardingSphere-Proxy 提供基于 YAML 的配置方式,并使用 DistSQL 进

规则配置部分与 ShardingSphere-JDBC 的 YAML 配置完全一致。
DistSQL 与 YAML 配置能够相互取代。

更多使用细节请参见[使用示例](https://github.com/apache/shardingsphere/tree/master/examples/shardingsphere-proxy-example)
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,3 @@ By configuration, application developers can flexibly use data sharding, readwri

Rule configuration keeps consist with YAML configuration of ShardingSphere-JDBC.
DistSQL and YAML can be replaced each other.

Please refer to [Example](https://github.com/apache/shardingsphere/tree/master/examples/shardingsphere-proxy-example) for more details.
Original file line number Diff line number Diff line change
Expand Up @@ -38,5 +38,7 @@ ShardingSphere 默认情况下仅包含核心 SPI 的实现,在 Git Source 存
- `org.apache.shardingsphere:shardingsphere-transaction-xa-narayana`,基于 Narayana 的 XA 分布式事务管理器
- SQL 翻译
- `org.apache.shardingsphere:shardingsphere-sql-translator-jooq-provider`,使用 JooQ 的 SQL 翻译器
- 行表达式
- `org.apache.shardingsphere:shardingsphere-infra-expr-espresso`,基于 GraalVM Truffle 的 Espresso 实现的使用 Groovy 语法的行表达式

除了以上可选插件外,ShardingSphere 社区开发者还贡献了大量的插件实现,可以在 [ShardingSphere Plugin](https://github.com/apache/shardingsphere-plugin) 仓库中查看插件的使用说明,ShardingSphere Plugin 仓库中的插件会和 ShardingSphere 保持相同的发布节奏,可以在 https://central.sonatype.com/ 进行检索,并安装到 ShardingSphere 中。
Original file line number Diff line number Diff line change
Expand Up @@ -38,5 +38,7 @@ All optional plugins are listed below in the form of `groupId:artifactId`.
- `org.apache.shardingsphere:shardingsphere-transaction-xa-narayana`, XA distributed transaction manager based on Narayana
- SQL translator
- `org.apache.shardingsphere:shardingsphere-sql-translator-jooq-provider`, JooQ SQL translator
- Row Value Expressions definition
- `org.apache.shardingsphere:shardingsphere-infra-expr-espresso`,Row Value Expressions that uses the Groovy syntax based on GraalVM Truffle's Espresso implementation

In addition to the above optional plugins, ShardingSphere community developers have contributed a number of plugin implementations. These plugins can be found in [ShardingSphere Plugins] (https://github.com/apache/shardingsphere-plugin) repository. Plugins in ShardingSphere Plugin repository would remain the same release plan with ShardingSphere, they can be retrieved at https://central.sonatype.com/, and install into ShardingSphere.
In addition to the above optional plugins, ShardingSphere community developers have contributed a number of plugin implementations. These plugins can be found in [ShardingSphere Plugins](https://github.com/apache/shardingsphere-plugin) repository. Plugins in ShardingSphere Plugin repository would remain the same release plan with ShardingSphere, they can be retrieved at https://central.sonatype.com/, and install into ShardingSphere.
Original file line number Diff line number Diff line change
Expand Up @@ -85,8 +85,3 @@ psql -h ${proxy_host} -p ${proxy_port} -U ${proxy_username}
```bash
gsql -r -h ${proxy_host} -p ${proxy_port} -U ${proxy_username} -W ${proxy_password}
```

## 配置示例

完整配置请参考 ShardingSphere 仓库中的示例:
<https://github.com/apache/shardingsphere/tree/master/examples/shardingsphere-proxy-example>
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,3 @@ Connect ShardingSphere-Proxy with openGauss client:
```bash
gsql -r -h ${proxy_host} -p ${proxy_port} -U ${proxy_username} -W ${proxy_password}
```

## Sample

Please refer to samples on ShardingSphere repository for complete configuration:
<https://github.com/apache/shardingsphere/tree/master/examples/shardingsphere-proxy-example>
Original file line number Diff line number Diff line change
Expand Up @@ -83,8 +83,3 @@ ShardingSphere-Proxy 默认端口 `3307`,可以通过环境变量 `-e PORT`
5. 使用客户端连接 ShardingSphere-Proxy

请参考 [ShardingSphere-Proxy 启动手册 - 使用二进制发布包](/cn/user-manual/shardingsphere-proxy/startup/bin/)

## 配置示例

完整配置请参考 ShardingSphere 仓库中的示例:
<https://github.com/apache/shardingsphere/tree/master/examples/shardingsphere-proxy-example>
Original file line number Diff line number Diff line change
Expand Up @@ -83,8 +83,3 @@ Support setting environment variable CGROUP_ MEM_ OPTS: used to set related memo
5. Use Client to connect to ShardingSphere-Proxy

Please refer to [ShardingSphere-Proxy quick start manual - binary distribution packages](/en/user-manual/shardingsphere-proxy/startup/bin/).

## Configuration Example

For full configuration, please refer to the examples given in ShardingSphere library:
<https://github.com/apache/shardingsphere/tree/master/examples/shardingsphere-proxy-example>
Original file line number Diff line number Diff line change
Expand Up @@ -50,16 +50,14 @@ services:

## 前提条件

1. 根据 https://www.graalvm.org/downloads/ 要求安装和配置 JDK 17 对应的 `GraalVM Community Edition``Oracle GraalVM`
。或者使用 `SDKMAN!`
。如果你希望使用携带了 [GraalVM Free Terms and Conditions license](https://www.oracle.com/downloads/licenses/graal-free-license.html)
`Oracle GraalVM`,下面的命令应更改为 `sdk install java 17.0.8-graal`
1. 根据 https://www.graalvm.org/downloads/ 要求安装和配置 JDK 17 对应的 `GraalVM Community Edition`
`GraalVM Community Edition` 的下游发行版。若使用 `SDKMAN!`

```shell
sdk install java 17.0.8-graalce
```

2. 根据 https://www.graalvm.org/latest/reference-manual/native-image/#prerequisites 的要求安装本地工具链。
2. 根据 https://www.graalvm.org/jdk17/reference-manual/native-image/#prerequisites 的要求安装本地工具链。

3. 如果需要构建 Docker Image, 确保 `docker-ce` 已安装。

Expand Down Expand Up @@ -145,7 +143,7 @@ services:

- 如果你不对 Git Source 做任何更改, 上文提及的命令将使用 `oraclelinux:9-slim` 作为 Base Docker Image。
但如果你希望使用 `busybox:glic``gcr.io/distroless/base``scratch` 等更小体积的 Docker Image 作为 Base Docker
Image,你需要根据 https://www.graalvm.org/latest/reference-manual/native-image/guides/build-static-executables/ 的要求,
Image,你需要根据 https://www.graalvm.org/jdk17/reference-manual/native-image/guides/build-static-executables/ 的要求,
做为 `pom.xml``native profile` 添加 `-H:+StaticExecutableWithDynamicLibC``jvmArgs` 等操作。
另请注意,某些第三方依赖将需要在 `Dockerfile` 安装更多系统库,例如 `libdl`
因此请确保根据你的使用情况调整 `distribution/proxy-native` 下的 `pom.xml``Dockerfile` 的内容。
Expand All @@ -155,7 +153,7 @@ services:
- 针对 GraalVM Native Image 形态的 ShardingSphere Proxy,其提供的可观察性的能力与
https://shardingsphere.apache.org/document/current/cn/user-manual/shardingsphere-proxy/observability/ 并不一致。

- 你可以使用 https://www.graalvm.org/latest/tools/ 提供的一系列命令行工具或可视化工具观察 GraalVM Native Image 的内部行为,
- 你可以使用 https://www.graalvm.org/jdk17/tools/ 提供的一系列命令行工具或可视化工具观察 GraalVM Native Image 的内部行为,
并根据其要求使用 VSCode 完成调试工作。如果你正在使用 IntelliJ IDEA 并且希望调试生成的 GraalVM Native Image,你可以关注
https://blog.jetbrains.com/idea/2022/06/intellij-idea-2022-2-eap-5/#Experimental_GraalVM_Native_Debugger_for_Java
及其后继。如果你使用的不是 Linux,则无法对 GraalVM Native Image 进行 Debug,请关注尚未关闭的
Expand Down
Loading

0 comments on commit a7a55ff

Please sign in to comment.