Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-10934. Refactor TestOzoneRpcClient hierarchy #6747

Merged
merged 10 commits into from
May 31, 2024

Conversation

adoroszlai
Copy link
Contributor

@adoroszlai adoroszlai commented May 29, 2024

What changes were proposed in this pull request?

Refactor TestOzoneRpcClientAbstract and its subclasses:

  1. TestOzoneRpcClient and TestOzoneRpcClientWithRatis are subclasses of TestOzoneRpcClientAbstract. They used to execute the same test cases with OM Ratis disabled (relying on default setting) and enabled (explicitly), respectively. HDDS-4498 enabled OM Ratis by default, so now both classes test with OM Ratis enabled. This change disables OM Ratis for TestOzoneRpcClient, and fixes one test case that was failing with NPE in this setup.
  2. TestSecureOzoneRpcClient should directly extend TestOzoneRpcClientAbstract to avoid confusion caused by shadowed lifecycle methods.
  3. Allow passing MiniOzoneCluster.Builder to startCluster to avoid code duplication in TestSecureOzoneRpcClient.
  4. Rename TestOzoneRpcClientAbstract to OzoneRpcClientTests to follow naming conventions.
  5. Move test cases from TestOzoneRpcClientWithRatis to the base class to extend coverage for other configs.

(Items 2-5 were added after initial round of review.)

https://issues.apache.org/jira/browse/HDDS-10934

How was this patch tested?

CI:
https://github.com/adoroszlai/ozone/actions/runs/9282222672/job/25540423674#step:5:2276

@adoroszlai adoroszlai self-assigned this May 29, 2024
Copy link
Contributor

@errose28 errose28 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall fix makes sense. There's some other weird stuff in this area, not sure if we want to handle it in this PR or a follow up:

  • The tests run for both ratis and non-ratis cases (in TestOzoneRpcClientAbstract) and those run only for Ratis cases (in TestOzoneRpcClientWithRatis) seem kind of arbitrary.
    • Maybe we can either unify them so all tests run in both setups, or only a minimal amount of validation is done in the non-ratis setup.
  • TestSecureOzoneRpcClient extends TestOzoneRpcClient, so now the secure tests are only running in a non-ratis OM setup.
    • Also tests that are unique to TestOzoneRpcClientWithRatis are not run in a secure setup.

@adoroszlai adoroszlai marked this pull request as draft May 30, 2024 07:10
@adoroszlai
Copy link
Contributor Author

Thanks @errose28 for the review.

  • The tests run for both ratis and non-ratis cases (in TestOzoneRpcClientAbstract) and those run only for Ratis cases (in TestOzoneRpcClientWithRatis) seem kind of arbitrary.

    • Maybe we can either unify them so all tests run in both setups, or only a minimal amount of validation is done in the non-ratis setup.
    • Also tests that are unique to TestOzoneRpcClientWithRatis are not run in a secure setup.

Agree, we should move them to the parent class.

  • TestSecureOzoneRpcClient extends TestOzoneRpcClient, so now the secure tests are only running in a non-ratis OM setup.

This is not the case, since TestSecureOzoneRpcClient creates its own cluster, TestOzoneRpcClient#init is not invoked due to "shadowing" (which is a bit obscure). Otherwise it would be testing an unsecure setup or starting two clusters. This should be improved, too.

Will push a few more commits.

@adoroszlai adoroszlai marked this pull request as ready for review May 30, 2024 09:11
@adoroszlai adoroszlai changed the title HDDS-10934. Deduplicate TestOzoneRpcClient and TestOzoneRpcClientWithRatis HDDS-10934. Refactor TestOzoneRpcClient hierarchy May 30, 2024
@adoroszlai adoroszlai requested a review from errose28 May 30, 2024 10:15
@adoroszlai
Copy link
Contributor Author

@errose28 patch is updated, please take another look

Copy link
Contributor

@errose28 errose28 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing the refactoring @adoroszlai. LGTM

@adoroszlai adoroszlai merged commit 385c4ec into apache:master May 31, 2024
41 checks passed
@adoroszlai adoroszlai deleted the HDDS-10934 branch May 31, 2024 20:07
@adoroszlai
Copy link
Contributor Author

Thanks @errose28 for the review.

jojochuang pushed a commit to jojochuang/ozone that referenced this pull request Jun 15, 2024
(cherry picked from commit 385c4ec)
Change-Id: I99e2c6332816ea345c11928d9d1182642beb22c7
jojochuang pushed a commit to jojochuang/ozone that referenced this pull request Jun 17, 2024
(cherry picked from commit 385c4ec)

 Conflicts:
	hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestSecureOzoneRpcClient.java

Change-Id: I78df84af142772a188f48680b5e1aaa189afd7c3
jojochuang pushed a commit to jojochuang/ozone that referenced this pull request Jun 17, 2024
(cherry picked from commit 385c4ec)

 Conflicts:
	hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestSecureOzoneRpcClient.java

Change-Id: I78df84af142772a188f48680b5e1aaa189afd7c3

HDDS-10934 amendment

Change-Id: I7dfb930a6d113315dc6652a2d68e83337cdca752
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
2 participants