Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Fix to use TezIdHelper to getTaskAttemptId instead of default IdHelper. #1062

Closed
3 tasks done
lifeSo opened this issue Aug 1, 2023 · 0 comments · Fixed by #1078
Closed
3 tasks done

[Bug] Fix to use TezIdHelper to getTaskAttemptId instead of default IdHelper. #1062

lifeSo opened this issue Aug 1, 2023 · 0 comments · Fixed by #1078
Assignees
Labels
0.8 For 0.8 version release

Comments

@lifeSo
Copy link
Collaborator

lifeSo commented Aug 1, 2023

Code of Conduct

Search before asking

  • I have searched in the issues and found no similar issues.

Describe the bug

when there is no task fail and attempt, the default IdHelper code before will work ok, but if where is task fail and attempt, it will filter block id.

The filter code:

  • ShuffleReadClientImp.java
    List<Long> removeBlockIds = Lists.newArrayList();
    blockIdBitmap.forEach(
        bid -> {
          if (!taskIdBitmap.contains(idHelper.getTaskAttemptId(bid))) {
            removeBlockIds.add(bid);
          }
        });

    for (long rid : removeBlockIds) {
      blockIdBitmap.removeLong(rid);
    }

idHelper.getTaskAttemptId(bid) will return wrong task attemt id if there is task attempt and use default idHelper.

Affects Version(s)

master

Uniffle Server Log Output

No response

Uniffle Engine Log Output

No response

Uniffle Server Configurations

No response

Uniffle Engine Configurations

No response

Additional context

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!
@jerqi jerqi added the 0.8 For 0.8 version release label Aug 2, 2023
jerqi pushed a commit that referenced this issue Aug 23, 2023
…of default IdHelper. (#1078)

### What changes were proposed in this pull request?

Fix to use TezIdHelper to getTaskAttemptId instead of default IdHelper.

### Why are the changes needed?

Fix to use TezIdHelper to getTaskAttemptId instead of default IdHelper.

Fix: [#1062](#1062)

### Does this PR introduce _any_ user-facing change?


No.

### How was this patch tested?
CI passed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.8 For 0.8 version release
Projects
None yet
2 participants