Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flaky test QuotaManagerTest#testDetectUserResource #421

Merged
merged 3 commits into from
Dec 15, 2022

Conversation

xianjingfeng
Copy link
Member

@xianjingfeng xianjingfeng commented Dec 14, 2022

What changes were proposed in this pull request?

Fix flaky test QuotaManagerTest#testDetectUserResource

Why are the changes needed?

The process of parsing quota file is asynchronous.
https://github.com/apache/incubator-uniffle/actions/runs/3695326069/jobs/6257565865

Does this PR introduce any user-facing change?

No

How was this patch tested?

No need

@codecov-commenter
Copy link

codecov-commenter commented Dec 14, 2022

Codecov Report

Merging #421 (22c24c3) into master (66c752f) will decrease coverage by 0.12%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master     #421      +/-   ##
============================================
- Coverage     58.54%   58.41%   -0.13%     
- Complexity     1611     1612       +1     
============================================
  Files           195      195              
  Lines         11035    11063      +28     
  Branches        966      976      +10     
============================================
+ Hits           6460     6463       +3     
- Misses         4195     4221      +26     
+ Partials        380      379       -1     
Impacted Files Coverage Δ
.../java/org/apache/uniffle/common/util/RssUtils.java 58.45% <0.00%> (-2.14%) ⬇️
...rg/apache/uniffle/client/util/DefaultIdHelper.java 100.00% <0.00%> (ø)
...pache/hadoop/mapreduce/task/reduce/RssShuffle.java 0.00% <0.00%> (ø)
...uniffle/storage/factory/ShuffleHandlerFactory.java 0.00% <0.00%> (ø)
...client/request/CreateShuffleReadClientRequest.java 0.00% <0.00%> (ø)
.../storage/handler/impl/MemoryClientReadHandler.java 0.00% <0.00%> (ø)
...torage/handler/impl/ComposedClientReadHandler.java 0.00% <0.00%> (ø)
...orage/request/CreateShuffleReadHandlerRequest.java 0.00% <0.00%> (ø)
...che/uniffle/client/impl/ShuffleReadClientImpl.java 88.57% <0.00%> (+0.10%) ⬆️
...rg/apache/uniffle/server/buffer/ShuffleBuffer.java 95.04% <0.00%> (+1.65%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@kaijchen
Copy link
Contributor

kaijchen commented Dec 14, 2022

Thanks @xianjingfeng for the work. A few suggestions here:

  1. You can use Awaitility for the wait, see https://github.com/apache/incubator-uniffle/search?q=awaitility
  2. Please show evidence of "this test is no longer flaky after this change".
    For example (see below), you can change this test to a ParameterizedTest locally and repeat it many times.
  static IntStream range() {
    return IntStream.range(0, 100);
  }
  @ParameterizedTest
  @MethodSource("range")
  public void testDetectUserResource(int dummy) throws Exception {

The test result (before the change):
NOTE: Seems there are some issues in iteration 58, maybe we should add a @Timeout.
image

@xianjingfeng
Copy link
Member Author

image

@kaijchen

Copy link
Contributor

@kaijchen kaijchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @xianjingfeng.

@kaijchen kaijchen merged commit c8f8292 into apache:master Dec 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants