Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create JUnit5 version of FlinkTestBase #9120

Merged
merged 6 commits into from
Nov 24, 2023

Conversation

cgpoh
Copy link
Contributor

@cgpoh cgpoh commented Nov 21, 2023

- Add FlinkTestBaseJU5 and MiniClusterResourceJU5 class
- Fixes apachegh-9078
@github-actions github-actions bot added the flink label Nov 21, 2023
@pvary
Copy link
Contributor

pvary commented Nov 21, 2023

@cgpoh: We usually only create a PR for a single version of Flink (latest). After that PR is merged, then we create a backport PR to get the changes to the older versions. This helps the reviewer (smaller number of changes), and the contributor (if changes was requested then it is a smaller work) too.

Please avoid having the J5 in the class names - we would like to remove the old classes, and after that J5 classes will be hard to understand. Create a new package? Find a better name for the classes? Whatever works - but keep in mind the goal 😄

Please migrate at least one test class to the new base package, so we can try things out.

I have found that TemporaryFolder, GenericAppenderHelper should be migrated too to have a full Junit5 test infra.

@cgpoh
Copy link
Contributor Author

cgpoh commented Nov 21, 2023

@pvary , thanks! Will update the class name and revert the changes for Flink 1.15 and 1.16

import static org.junit.jupiter.api.Assertions.assertTrue;

/** Test for {@link TableLoader}. */
public class TestCatalogTblLoader extends FlinkTestBaseUtils {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be an update of the original TestCatalogTableLoader class, just instead of using the old FlinkTestBase, extend the new FlinkTestBaseUtils

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So use the old class here, and extend the new base class. So it is easier to see, that all of the tests are the same, and also we do not need 2 test class for the same feature

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to that. We don't want to duplicate test classes themselves, but rather convert them from their Junit4 base class to their new Junit5 base class (once the new base class exists)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Will update the old class instead

import org.junit.jupiter.api.extension.RegisterExtension;
import org.junit.jupiter.api.io.TempDir;

public abstract class FlinkTestBaseUtils extends TestBaseUtils {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be only TestBase, or ClusterTestBase? Or something similar?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to using TestBase here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Renaming to TestBase

@pvary
Copy link
Contributor

pvary commented Nov 22, 2023

@cgpoh: Please run ./gradlew spotlessApply before pushing the changes to fix splotless errors and minimally ./gradlew build -x test -x javadoc -x integrationTest to catch checkstyle errors.

@cgpoh
Copy link
Contributor Author

cgpoh commented Nov 22, 2023

@pvary will do. Thanks!

import org.apache.flink.runtime.testutils.MiniClusterResourceConfiguration;
import org.apache.flink.test.junit5.MiniClusterExtension;

public class MiniFlinkClusterResource {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Do you mind renaming this to MiniClusterExtension?

Copy link
Contributor Author

@cgpoh cgpoh Nov 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pvary , I think I rename to MiniFlinkClusterExtension instead. If I rename to MiniClusterExtension, the following method will need to import MiniClusterExtension explicitly as below:

  public static org.apache.flink.test.junit5.MiniClusterExtension createWithClassloaderCheckDisabled() {
    return new org.apache.flink.test.junit5.MiniClusterExtension(
        new MiniClusterResourceConfiguration.Builder()
            .setNumberTaskManagers(DEFAULT_TM_NUM)
            .setNumberSlotsPerTaskManager(DEFAULT_PARALLELISM)
            .setConfiguration(DISABLE_CLASSLOADER_CHECK_CONFIG)
            .build());
  }

what do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed

@nastra nastra merged commit c427a56 into apache:main Nov 24, 2023
13 checks passed
@cgpoh cgpoh deleted the create-junit5-flinktestbase branch November 24, 2023 07:04
@pvary
Copy link
Contributor

pvary commented Nov 24, 2023

Thanks for your work @cgpoh and @nastra for the review!

@cgpoh: Please create the backport PR to v1.15, v1.16. It could be one PR, and please make sure that the resulting code is the same. This helps in the long run when we are backporting new features.

@cgpoh
Copy link
Contributor Author

cgpoh commented Nov 24, 2023

Thanks! Will create the PR later

rodmeneses pushed a commit to rodmeneses/iceberg that referenced this pull request Feb 19, 2024
rodmeneses added a commit to rodmeneses/iceberg that referenced this pull request Feb 19, 2024
devangjhabakh pushed a commit to cdouglas/iceberg that referenced this pull request Apr 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create JUnit5-version of FlinkTestBase
3 participants