Skip to content
Permalink
Browse files
[IOTDB-1559] Refactor the IT framework (#4371)
  • Loading branch information
irvine0109 committed Nov 30, 2021
1 parent 3d48a88 commit 4f8015ce528782f8ed59f276c56af91659267c5b
Show file tree
Hide file tree
Showing 180 changed files with 4,380 additions and 3,107 deletions.
@@ -0,0 +1,50 @@
name: Cluster Test

on:
push:
branches:
- master
- 'rel/*'
paths-ignore:
- 'docs/**'
pull_request:
branches:
- master
- 'rel/*'
paths-ignore:
- 'docs/**'
# allow manually run the action:
workflow_dispatch:

env:
MAVEN_OPTS: -Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false -Dmaven.wagon.http.retryHandler.class=standard -Dmaven.wagon.http.retryHandler.count=3

jobs:
unix:
strategy:
fail-fast: false
max-parallel: 20
matrix:
java: [ 8 ]
os: [ ubuntu-latest ]
runs-on: ${{ matrix.os}}

steps:
- uses: actions/checkout@v2
- name: Set up JDK ${{ matrix.java }}
uses: actions/setup-java@v1
with:
java-version: ${{ matrix.java }}
- name: Cache Maven packages
uses: actions/cache@v2
with:
path: ~/.m2
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2-
- name: Check Apache Rat
run: mvn -B apache-rat:check -P site -P code-coverage
- name: IT/UT Test
shell: bash
# we do not compile client-cpp for saving time, it is tested in client.yml
# we can skip influxdb-protocol because it has been tested separately in influxdb-protocol.yml
run: mvn -B clean verify -Dsession.test.skip=true -Diotdb.test.skip=true -Dcluster.test.skip=true -Dtsfile.test.skip=true -pl integration -am -PCluster
@@ -53,4 +53,4 @@ jobs:
shell: bash
# we do not compile client-cpp for saving time, it is tested in client.yml
# we can skip influxdb-protocol because it has been tested separately in influxdb-protocol.yml
run: mvn -B clean post-integration-test -Dtest.port.closed=true -P '!testcontainer,!influxdb-protocol'
run: mvn -B clean verify -Dtest.port.closed=true -P '!testcontainer,!influxdb-protocol'
@@ -109,5 +109,5 @@ jobs:
shell: bash
if: ${{ matrix.it_task == 'others'}}
# we can skip influxdb-protocol because it has been tested separately in influxdb-protocol.yml
run: source ~/.bash_profile && mvn -B clean integration-test -Dtest.port.closed=true -Diotdb.test.skip=true -Dcluster.test.skip=true -P '!influxdb-protocol'
run: source ~/.bash_profile && mvn -B clean verify -Dtest.port.closed=true -Diotdb.test.skip=true -Dcluster.test.skip=true -P '!influxdb-protocol'

@@ -0,0 +1,161 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# Integration Test refactoring tutorial

- Step 0. Prerequisites

- **IT Location has been moved**; all Integration Tests have been moved to the integration module.
- **Test case MUST label**; the test classification label is the junit's `category`, which determines the test environment or process in which the test case is tested.
- **Code related to the test environment MAY need to be refactored**; this determines whether the current test environment can correctly test the test case. The corresponding statement needs to be refactored.



- Step 1. Labeling test cases

- Add the appropriate `Category` before the test case class or test case method, which can collect any desired test category labels.

- The `Category` of the following three test categories are all real and effective,

```java
@Category({LocalStandaloneTest.class, ClusterTest.class, RemoteTest.class})
public class IoTDBAliasIT {
......
}
@Category({LocalStandaloneTest.class, ClusterTest.class})
public class IoTDBAlignByDeviceIT {
......
}
@Category({LocalStandaloneTest.class})
public class IoTDBArithmeticIT {
......
}
```

- You can also add `Category` at the test method level.

```java
@Category({LocalStandaloneTest.class})
public class IoTDBExampleIT {
// This case can ONLY test in environment of local.
@Test
public void theStandaloneCase() {
......
}
// This case can test in environment of local, cluster and remote.
@Test
@Category({ClusterTest.class, RemoteTest.class})
public void theAllEnvCase() {
......
}
}
```

- At present, all test cases must at least add the `Category` of the stand-alone test, namely `LocalStandaloneTest.class`.



- Step 2. Environmental code refactoring

- If the test case needs to be tested in the Cluster or Remote environment, the environment-related code MUST be refactored accordingly. If it is only tested in the LocalStandalone environment, modifications are only recommended. (Not all test cases can be tested in the Cluster or Remote environment because statements that are limited by some functions, such as local file operations, cannot be refactored.)

| | LocalStandalone | Cluster | Remote |
| :------------------------- | :-------------: | :---------: | :---------: |
| setUp and tearDown | Recommend | Must | Must |
| getConnection | Recommend | Must | Must |
| change config | Recommend | Must | Not support |
| Local file operation | Won't change | Not support | Not support |
| Local descriptor operation | Won't change | Not support | Not support |
| restart operation | Won't change | Not support | Not support |



- The `setUp` and `tearDown` methods must be refactored in the Cluster and Remote environment

```java
@Category({LocalStandaloneTest.class, ClusterTest.class, RemoteTest.class})
public class IoTDBAliasIT {
@BeforeClass
public static void setUp() throws Exception {
// EnvironmentUtils.closeStatMonitor(); // orginal setup code
// EnvironmentUtils.envSetUp(); // orginal setup code
EnvFactory.getEnv().initBeforeClass(); // new initBeforeClass code
insertData();
}
@AfterClass
public static void tearDown() throws Exception {
// EnvironmentUtils.cleanEnv(); // orginal code
EnvFactory.getEnv().cleanAfterClass(); // new cleanAfterClass code
}
}
```



- The `getConnection` must be refactored in Cluster and Remote environments

```java
private static void insertData() throws ClassNotFoundException {
// Class.forName(Config.JDBC_DRIVER_NAME); // orginal connection code
// try (Connection connection =
// DriverManager.getConnection(
// Config.IOTDB_URL_PREFIX + "127.0.0.1:6667/", "root", "root");
try (Connection connection = EnvFactory.getEnv().getConnection(); // new code
Statement statement = connection.createStatement()) {
for (String sql : sqls) {
statement.execute(sql);
}
} catch (Exception e) {
e.printStackTrace();
}
}
```

- The method of changing the IoTDB configuration must be refactored in the Cluster environment. (As the remote environment cannot change the configuration remotely at present, the test cases that change the configuration will not support testing in the remote environment)
- In the Cluster environment, as the configuration cannot be changed dynamically, only the configuration changes before the environment init are effective.
- The refactoring has included most of the configuration changes, which can be changed through the method of `ConfigFactory.getConfig()`.

```java
@Category({LocalStandaloneTest.class, ClusterTest.class})
public class IoTDBCompleteIT {
private int prevVirtualStorageGroupNum;
@Before
public void setUp() {
prevVirtualStorageGroupNum =
IoTDBDescriptor.getInstance().getConfig().getVirtualStorageGroupNum();
// IoTDBDescriptor.getInstance().getConfig().setVirtualStorageGroupNum(16); // orginal code
ConfigFactory.getConfig().setVirtualStorageGroupNum(16); // new code
EnvFactory.getEnv().initBeforeClass();
}
```
- If the configuration item has not been included in the method of `ConfigFactory.getConfig()`, it needs to be defined in the BaseConfig.java interface file and implemented in StandaloneEnvConfig.java and ClusterEnvConfig.java, respectively. This part is not very common. For specific, please refer to the realized part.
@@ -0,0 +1,163 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# IoTDB社区Integration Test改造说明

- 步骤0. 前提须知
- **位置已移动**;所有的Integration Test已被移动至单独的integration模块。
- **测试用例必须打分类标签**`Category` 即测试分类标签,决定了该测试用例在哪套测试环境或流程中被测试。
- **涉及测试环境的代码可能要重构**;决定了该测试用例是否能被当前测试环境正确测试,需要根据相应的环境重构相应的代码。



- 步骤1. 测试用例打标签

- 在测试用例类或者测试用例方法前加上合适的`Category`,可以是任意期望的测试分类标签的集合。

- 真实样例,下面三个测试类的`Category`都是真实有效的,

```java
@Category({LocalStandaloneTest.class, ClusterTest.class, RemoteTest.class})
public class IoTDBAliasIT {
......
}
@Category({LocalStandaloneTest.class, ClusterTest.class})
public class IoTDBAlignByDeviceIT {
......
}
@Category({LocalStandaloneTest.class})
public class IoTDBArithmeticIT {
......
}
```

- 甚至,你还可以在测试方法级别加`Category`

```java
@Category({LocalStandaloneTest.class})
public class IoTDBExampleIT {
// This case can ONLY test in environment of local.
@Test
public void theStandaloneCase() {
......
}
// This case can test in environment of local, cluster and remote.
@Test
@Category({ClusterTest.class, RemoteTest.class})
public void theAllEnvCase() {
......
}
}
```

- 目前,所有测试用例至少要加上单机测试的测试分类,即`LocalStandaloneTest.class`



- 步骤2. 环境代码重构

- 如果测试用例需要在Cluster或者Remote环境下被测试,那么必须对环境相关的代码作相应重构,如果是仅在LocalStandalone环境下测试,则只推荐修改。(不是所有的测试用例可以在Cluster或者Remote环境下被测试,因为受限于部分功能的语句比如本地文件操作,这些代码不能被重构。)

| | LocalStandalone | Cluster | Remote |
| :------------------------- | :-------------: | :---------: | :---------: |
| setUp and tearDown | Recommend | Must | Must |
| getConnection | Recommend | Must | Must |
| change config | Recommend | Must | Not support |
| Local file operation | Won't change | Not support | Not support |
| Local descriptor operation | Won't change | Not support | Not support |
| restart operation | Won't change | Not support | Not support |



- `setUp``tearDown` 方法内的重构,在Cluster和Remote环境下是必须更改的

```java
@Category({LocalStandaloneTest.class, ClusterTest.class, RemoteTest.class})
public class IoTDBAliasIT {
@BeforeClass
public static void setUp() throws Exception {
// EnvironmentUtils.closeStatMonitor(); // orginal setup code
// EnvironmentUtils.envSetUp(); // orginal setup code
EnvFactory.getEnv().initBeforeClass(); // new initBeforeClass code
insertData();
}
@AfterClass
public static void tearDown() throws Exception {
// EnvironmentUtils.cleanEnv(); // orginal code
EnvFactory.getEnv().cleanAfterClass(); // new cleanAfterClass code
}
}
```



- `getConnection` 的重构,在Cluster和Remote环境下是必须更改

```java
private static void insertData() throws ClassNotFoundException {
// Class.forName(Config.JDBC_DRIVER_NAME); // orginal connection code
// try (Connection connection =
// DriverManager.getConnection(
// Config.IOTDB_URL_PREFIX + "127.0.0.1:6667/", "root", "root");
try (Connection connection = EnvFactory.getEnv().getConnection(); // new code
Statement statement = connection.createStatement()) {
for (String sql : sqls) {
statement.execute(sql);
}
} catch (Exception e) {
e.printStackTrace();
}
}
```



- 更改配置的方法,在Cluster环境下是必须重构的。(由于目前Remote环境无法远程更改配置,更改配置的测试用例将不支持Remote环境下测试)

- 在Cluster环境下,由于无法动态更改配置,只有环境init前的配置更改才有效。
- 重构已包含了大部分的配置更改,通过`ConfigFactory.getConfig()` 的方法可以进行链式更改。

```java
@Category({LocalStandaloneTest.class, ClusterTest.class})
public class IoTDBCompleteIT {
private int prevVirtualStorageGroupNum;
@Before
public void setUp() {
prevVirtualStorageGroupNum =
IoTDBDescriptor.getInstance().getConfig().getVirtualStorageGroupNum();
// IoTDBDescriptor.getInstance().getConfig().setVirtualStorageGroupNum(16); // orginal code
ConfigFactory.getConfig().setVirtualStorageGroupNum(16); // new code
EnvFactory.getEnv().initBeforeClass();
}
```
- 若配置项尚未在`ConfigFactory.getConfig()` 的方法中包含,需要在BaseConfig.java接口文件中定义,在StandaloneEnvConfig.java和ClusterEnvConfig.java中分别实现,这部分不是很常用,具体方法可以参考已实现的部分,目前暂不列出。

0 comments on commit 4f8015c

Please sign in to comment.