Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-33637][table-planner][JUnit5 Migration] Introduce ArchTest to ban Junit 4 for module table-planner #23791

Merged
merged 4 commits into from
Nov 29, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -38,12 +38,6 @@ org.apache.flink.connectors.hive.HiveTableSinkITCase does not satisfy: only one
* reside in a package 'org.apache.flink.runtime.*' and contain any fields that are static, final, and of type InternalMiniClusterExtension and annotated with @RegisterExtension\
* reside outside of package 'org.apache.flink.runtime.*' and contain any fields that are static, final, and of type MiniClusterExtension and annotated with @RegisterExtension or are , and of type MiniClusterTestEnvironment and annotated with @TestEnv\
* reside in a package 'org.apache.flink.runtime.*' and is annotated with @ExtendWith with class InternalMiniClusterExtension\
* reside outside of package 'org.apache.flink.runtime.*' and is annotated with @ExtendWith with class MiniClusterExtension\
or contain any fields that are public, static, and of type MiniClusterWithClientResource and final and annotated with @ClassRule or contain any fields that is of type MiniClusterWithClientResource and public and final and not static and annotated with @Rule
org.apache.flink.connectors.hive.HiveTableSourceITCase does not satisfy: only one of the following predicates match:\
* reside in a package 'org.apache.flink.runtime.*' and contain any fields that are static, final, and of type InternalMiniClusterExtension and annotated with @RegisterExtension\
* reside outside of package 'org.apache.flink.runtime.*' and contain any fields that are static, final, and of type MiniClusterExtension and annotated with @RegisterExtension or are , and of type MiniClusterTestEnvironment and annotated with @TestEnv\
* reside in a package 'org.apache.flink.runtime.*' and is annotated with @ExtendWith with class InternalMiniClusterExtension\
* reside outside of package 'org.apache.flink.runtime.*' and is annotated with @ExtendWith with class MiniClusterExtension\
or contain any fields that are public, static, and of type MiniClusterWithClientResource and final and annotated with @ClassRule or contain any fields that is of type MiniClusterWithClientResource and public and final and not static and annotated with @Rule
org.apache.flink.connectors.hive.HiveTemporalJoinITCase does not satisfy: only one of the following predicates match:\
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,6 @@ org.apache.flink.formats.avro.AvroExternalJarProgramITCase does not satisfy: onl
* reside in a package 'org.apache.flink.runtime.*' and contain any fields that are static, final, and of type InternalMiniClusterExtension and annotated with @RegisterExtension\
* reside outside of package 'org.apache.flink.runtime.*' and contain any fields that are static, final, and of type MiniClusterExtension and annotated with @RegisterExtension or are , and of type MiniClusterTestEnvironment and annotated with @TestEnv\
* reside in a package 'org.apache.flink.runtime.*' and is annotated with @ExtendWith with class InternalMiniClusterExtension\
* reside outside of package 'org.apache.flink.runtime.*' and is annotated with @ExtendWith with class MiniClusterExtension\
or contain any fields that are public, static, and of type MiniClusterWithClientResource and final and annotated with @ClassRule or contain any fields that is of type MiniClusterWithClientResource and public and final and not static and annotated with @Rule
org.apache.flink.formats.avro.AvroFilesystemITCase does not satisfy: only one of the following predicates match:\
* reside in a package 'org.apache.flink.runtime.*' and contain any fields that are static, final, and of type InternalMiniClusterExtension and annotated with @RegisterExtension\
* reside outside of package 'org.apache.flink.runtime.*' and contain any fields that are static, final, and of type MiniClusterExtension and annotated with @RegisterExtension or are , and of type MiniClusterTestEnvironment and annotated with @TestEnv\
* reside in a package 'org.apache.flink.runtime.*' and is annotated with @ExtendWith with class InternalMiniClusterExtension\
* reside outside of package 'org.apache.flink.runtime.*' and is annotated with @ExtendWith with class MiniClusterExtension\
or contain any fields that are public, static, and of type MiniClusterWithClientResource and final and annotated with @ClassRule or contain any fields that is of type MiniClusterWithClientResource and public and final and not static and annotated with @Rule
org.apache.flink.formats.avro.typeutils.AvroSerializerConcurrencyCheckInactiveITCase does not satisfy: only one of the following predicates match:\
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +0,0 @@
org.apache.flink.formats.json.JsonBatchFileSystemITCase does not satisfy: only one of the following predicates match:\
* reside in a package 'org.apache.flink.runtime.*' and contain any fields that are static, final, and of type InternalMiniClusterExtension and annotated with @RegisterExtension\
* reside outside of package 'org.apache.flink.runtime.*' and contain any fields that are static, final, and of type MiniClusterExtension and annotated with @RegisterExtension or are , and of type MiniClusterTestEnvironment and annotated with @TestEnv\
* reside in a package 'org.apache.flink.runtime.*' and is annotated with @ExtendWith with class InternalMiniClusterExtension\
* reside outside of package 'org.apache.flink.runtime.*' and is annotated with @ExtendWith with class MiniClusterExtension\
or contain any fields that are public, static, and of type MiniClusterWithClientResource and final and annotated with @ClassRule or contain any fields that is of type MiniClusterWithClientResource and public and final and not static and annotated with @Rule
Original file line number Diff line number Diff line change
@@ -1,9 +1,3 @@
org.apache.flink.orc.OrcFileSystemITCase does not satisfy: only one of the following predicates match:\
* reside in a package 'org.apache.flink.runtime.*' and contain any fields that are static, final, and of type InternalMiniClusterExtension and annotated with @RegisterExtension\
* reside outside of package 'org.apache.flink.runtime.*' and contain any fields that are static, final, and of type MiniClusterExtension and annotated with @RegisterExtension or are , and of type MiniClusterTestEnvironment and annotated with @TestEnv\
* reside in a package 'org.apache.flink.runtime.*' and is annotated with @ExtendWith with class InternalMiniClusterExtension\
* reside outside of package 'org.apache.flink.runtime.*' and is annotated with @ExtendWith with class MiniClusterExtension\
or contain any fields that are public, static, and of type MiniClusterWithClientResource and final and annotated with @ClassRule or contain any fields that is of type MiniClusterWithClientResource and public and final and not static and annotated with @Rule
org.apache.flink.orc.writer.OrcBulkWriterITCase does not satisfy: only one of the following predicates match:\
* reside in a package 'org.apache.flink.runtime.*' and contain any fields that are static, final, and of type InternalMiniClusterExtension and annotated with @RegisterExtension\
* reside outside of package 'org.apache.flink.runtime.*' and contain any fields that are static, final, and of type MiniClusterExtension and annotated with @RegisterExtension or are , and of type MiniClusterTestEnvironment and annotated with @TestEnv\
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +0,0 @@
org.apache.flink.formats.parquet.ParquetFileSystemITCase does not satisfy: only one of the following predicates match:\
* reside in a package 'org.apache.flink.runtime.*' and contain any fields that are static, final, and of type InternalMiniClusterExtension and annotated with @RegisterExtension\
* reside outside of package 'org.apache.flink.runtime.*' and contain any fields that are static, final, and of type MiniClusterExtension and annotated with @RegisterExtension or are , and of type MiniClusterTestEnvironment and annotated with @TestEnv\
* reside in a package 'org.apache.flink.runtime.*' and is annotated with @ExtendWith with class InternalMiniClusterExtension\
* reside outside of package 'org.apache.flink.runtime.*' and is annotated with @ExtendWith with class MiniClusterExtension\
or contain any fields that are public, static, and of type MiniClusterWithClientResource and final and annotated with @ClassRule or contain any fields that is of type MiniClusterWithClientResource and public and final and not static and annotated with @Rule
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#
#Sat Nov 25 01:49:59 CST 2023
Junit4\ is\ forbidden,\ please\ use\ Junit5\ instead=a17c9d3e-9546-4787-a16e-91f74fc9a335
Tests\ inheriting\ from\ AbstractTestBase\ should\ have\ name\ ending\ with\ ITCase=95e87b01-d1fe-4240-9e71-b8105bccba6a
ITCASE\ tests\ should\ use\ a\ MiniCluster\ resource\ or\ extension=8905dbb5-c749-4c80-8848-c31e55f572f4
8 changes: 8 additions & 0 deletions flink-table/flink-table-planner/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,14 @@ under the License.
<scope>test</scope>
</dependency>

<!-- ArchUit test dependencies -->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-architecture-tests-test</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>

<!-- Table API Scala dependencies -->

<dependency>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.flink.architecture;

import org.apache.flink.architecture.common.ImportOptions;
import org.apache.flink.architecture.rules.BanJunit4Rules;

import com.tngtech.archunit.core.importer.ImportOption;
import com.tngtech.archunit.junit.AnalyzeClasses;
import com.tngtech.archunit.junit.ArchTest;
import com.tngtech.archunit.junit.ArchTests;

/** Architecture tests for test code. */
@AnalyzeClasses(
packages = {
"org.apache.flink.connector.file.table",
"org.apache.flink.formats.testcsv",
"org.apache.flink.table.api",
"org.apache.flink.table.planner",
"org.apache.flink.table.test",
},
importOptions = {
ImportOption.OnlyIncludeTests.class,
ImportOptions.ExcludeScalaImportOption.class,
ImportOptions.ExcludeShadedImportOption.class
})
public class TestCodeArchitectureTest {

@ArchTest
public static final ArchTests COMMON_TESTS = ArchTests.in(TestCodeArchitectureTestBase.class);

@ArchTest public static final ArchTests BAN_JUNIT4_TESTS = ArchTests.in(BanJunit4Rules.class);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to enable the ITCaseRules too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @JingGe for the review.
ITCaseRules was enabled and made some refactoring to meet the rules.
Please help review it again when you have time.

}
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,13 @@
import org.apache.flink.table.catalog.ObjectPath;
import org.apache.flink.table.catalog.exceptions.CatalogException;
import org.apache.flink.table.utils.CatalogManagerMocks;
import org.apache.flink.test.junit5.MiniClusterExtension;
import org.apache.flink.testutils.ClassLoaderUtils;
import org.apache.flink.testutils.junit.utils.TempDirUtils;
import org.apache.flink.util.TemporaryClassLoaderContext;

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
import org.junit.jupiter.api.io.TempDir;

import java.net.URLClassLoader;
Expand All @@ -51,6 +53,9 @@
/** IT Case for catalog ddl. */
class CatalogITCase {

@RegisterExtension
private static final MiniClusterExtension MINI_CLUSTER_RESOURCE = new MiniClusterExtension();

@TempDir Path temporaryFolder;

@Test
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
import org.apache.flink.util.Preconditions;

import org.junit.jupiter.api.TestInstance;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.api.extension.RegisterExtension;
import org.junit.jupiter.api.function.Executable;
import org.junit.jupiter.api.parallel.Execution;
import org.junit.jupiter.api.parallel.ExecutionMode;
Expand All @@ -66,9 +66,11 @@
/** Test base for testing aggregate {@link BuiltInFunctionDefinition built-in functions}. */
@Execution(ExecutionMode.CONCURRENT)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
@ExtendWith(MiniClusterExtension.class)
abstract class BuiltInAggregateFunctionTestBase {

@RegisterExtension
private static final MiniClusterExtension MINI_CLUSTER_EXTENSION = new MiniClusterExtension();

abstract Stream<TestSpec> getTestCaseSpecs();

final Stream<BuiltInFunctionTestBase.TestCase> getTestCases() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
import org.apache.flink.util.Preconditions;

import org.junit.jupiter.api.TestInstance;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.api.extension.RegisterExtension;
import org.junit.jupiter.api.function.Executable;
import org.junit.jupiter.api.parallel.Execution;
import org.junit.jupiter.api.parallel.ExecutionMode;
Expand Down Expand Up @@ -73,9 +73,11 @@
*/
@Execution(ExecutionMode.CONCURRENT)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
@ExtendWith(MiniClusterExtension.class)
abstract class BuiltInFunctionTestBase {

@RegisterExtension
private static final MiniClusterExtension MINI_CLUSTER_EXTENSION = new MiniClusterExtension();

Configuration getConfiguration() {
return new Configuration();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,14 @@
import org.apache.flink.table.functions.FunctionContext;
import org.apache.flink.table.functions.ScalarFunction;
import org.apache.flink.table.planner.plan.utils.JavaUserDefinedAggFunctions.WeightedAvg;
import org.apache.flink.test.junit5.MiniClusterExtension;
import org.apache.flink.types.Row;
import org.apache.flink.util.CollectionUtil;

import org.apache.calcite.sql.SqlMatchRecognize;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;

import java.util.Comparator;
import java.util.List;
Expand All @@ -49,6 +51,9 @@
/** IT Case for testing {@link SqlMatchRecognize}. */
class MatchRecognizeITCase {

@RegisterExtension
private static final MiniClusterExtension MINI_CLUSTER_EXTENSION = new MiniClusterExtension();

private StreamExecutionEnvironment env;
private StreamTableEnvironment tEnv;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,14 @@
import org.apache.flink.table.connector.sink.abilities.SupportsStaging;
import org.apache.flink.table.planner.factories.TestSupportsStagingTableFactory;
import org.apache.flink.table.planner.factories.utils.TestCollectionTableFactory;
import org.apache.flink.test.junit5.MiniClusterExtension;
import org.apache.flink.types.Row;
import org.apache.flink.util.FileUtils;
import org.apache.flink.util.TestLogger;

import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
import org.junit.jupiter.api.io.TempDir;

import java.io.File;
Expand All @@ -42,7 +43,10 @@
import static org.assertj.core.api.Assertions.assertThatCode;

/** The base case of atomic ctas ITCase. */
public abstract class AtomicCtasITCaseBase extends TestLogger {
public abstract class AtomicCtasITCaseBase {

@RegisterExtension
private static final MiniClusterExtension MINI_CLUSTER_EXTENSION = new MiniClusterExtension();

protected TableEnvironment tEnv;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,14 @@
import org.apache.flink.table.connector.sink.abilities.SupportsStaging;
import org.apache.flink.table.planner.factories.TestSupportsStagingTableFactory;
import org.apache.flink.table.planner.factories.utils.TestCollectionTableFactory;
import org.apache.flink.test.junit5.MiniClusterExtension;
import org.apache.flink.types.Row;
import org.apache.flink.util.FileUtils;
import org.apache.flink.util.TestLogger;

import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
import org.junit.jupiter.api.io.TempDir;

import java.io.File;
Expand All @@ -44,7 +45,10 @@
import static org.assertj.core.api.Assertions.assertThatThrownBy;

/** The base case of atomic rtas ITCase. */
public abstract class AtomicRtasITCaseBase extends TestLogger {
public abstract class AtomicRtasITCaseBase {

@RegisterExtension
private static final MiniClusterExtension MINI_CLUSTER_EXTENSION = new MiniClusterExtension();

protected TableEnvironment tEnv;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,35 +35,35 @@
/** Batch test base to use {@link RegisterExtension}. */
public class BatchAbstractTestBase {

public static final int DEFAULT_PARALLELISM = 3;
protected static final int DEFAULT_PARALLELISM = 3;

@RegisterExtension
public static MiniClusterExtension miniClusterResource =
private static final MiniClusterExtension MINI_CLUSTER_EXTENSION =
new MiniClusterExtension(
new MiniClusterResourceConfiguration.Builder()
.setConfiguration(getConfiguration())
.setNumberTaskManagers(1)
.setNumberSlotsPerTaskManager(DEFAULT_PARALLELISM)
.build());

@TempDir public static Path tmpDir;
@TempDir protected static Path tmpDir;

private static Configuration getConfiguration() {
Configuration config = new Configuration();
config.set(TaskManagerOptions.MANAGED_MEMORY_SIZE, MemorySize.parse("100m"));
return config;
}

public static File createTempFolder() throws IOException {
protected static File createTempFolder() throws IOException {
return TempDirUtils.newFolder(BatchAbstractTestBase.tmpDir);
}

public static File createTempFile() throws IOException {
protected static File createTempFile() throws IOException {
Path tmpDirPath = createTempFolder().toPath();
return TempDirUtils.newFile(tmpDirPath);
}

public static File createFileInTempFolder(String fileName) throws IOException {
protected static File createFileInTempFolder(String fileName) throws IOException {
Path tmpDirPath = createTempFolder().toPath();
return TempDirUtils.newFile(tmpDirPath, fileName);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

package org.apache.flink.table.planner.runtime.utils;

import org.apache.flink.runtime.testutils.MiniClusterResourceConfiguration;
import org.apache.flink.table.api.TableEnvironment;
import org.apache.flink.table.api.config.ExecutionConfigOptions;
import org.apache.flink.table.api.config.OptimizerConfigOptions;
Expand All @@ -30,10 +31,11 @@
import org.apache.flink.table.catalog.stats.CatalogColumnStatisticsDataLong;
import org.apache.flink.table.catalog.stats.CatalogTableStatistics;
import org.apache.flink.table.planner.factories.TestValuesTableFactory;
import org.apache.flink.util.TestLogger;
import org.apache.flink.test.junit5.MiniClusterExtension;

import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.extension.RegisterExtension;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;

Expand All @@ -49,10 +51,18 @@
* org.apache.calcite.rel.rules.LoptOptimizeJoinRule} together by changing the factor
* isBushyJoinReorder.
*/
public abstract class JoinReorderITCaseBase extends TestLogger {
public abstract class JoinReorderITCaseBase {

private static final int DEFAULT_PARALLELISM = 4;

@RegisterExtension
private static final MiniClusterExtension MINI_CLUSTER_EXTENSION =
new MiniClusterExtension(
new MiniClusterResourceConfiguration.Builder()
.setNumberTaskManagers(1)
.setNumberSlotsPerTaskManager(DEFAULT_PARALLELISM)
.build());

protected TableEnvironment tEnv;
private Catalog catalog;

Expand Down
Loading