Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBASE-25932: Ensure replication reads the trailer bytes from WAL. (#3… #3346

Merged
merged 1 commit into from
Jun 1, 2021

Conversation

bharathv
Copy link
Contributor

@bharathv bharathv commented Jun 1, 2021

…332)

This bug was exposed by the test from HBASE-25924. Since this wal
implementations close the wal asynchronously, replication can potentially
miss the trailer bytes. (see jira comment for detailed analysis).

While this is not a correctness problem (since trailer does not have any entry data),
it erroneously bumps a metric that is used to track skipped bytes in WAL resulting
in false alarms which is something we should avoid.

Reviewed-by: Rushabh Shah rushabh.shah@salesforce.com
Signed-off-by: Viraj Jasani vjasani@apache.org
Signed-off-by Anoop Sam John anoopsamjohn@apache.org
(cherry picked from commit b04c3c7)

…ache#3332)

This bug was exposed by the test from HBASE-25924. Since this wal
implementations close the wal asynchronously, replication can potentially
miss the trailer bytes. (see jira comment for detailed analysis).

While this is not a correctness problem (since trailer does not have any entry data),
it erroneously bumps a metric that is used to track skipped bytes in WAL resulting
in false alarms which is something we should avoid.

Reviewed-by: Rushabh Shah <rushabh.shah@salesforce.com>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by Anoop Sam John <anoopsamjohn@apache.org>
(cherry picked from commit b04c3c7)
@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 26s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ branch-2.4 Compile Tests _
+1 💚 mvninstall 4m 9s branch-2.4 passed
+1 💚 compile 3m 31s branch-2.4 passed
+1 💚 checkstyle 1m 15s branch-2.4 passed
+1 💚 spotbugs 2m 18s branch-2.4 passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 44s the patch passed
+1 💚 compile 3m 22s the patch passed
+1 💚 javac 3m 22s the patch passed
-0 ⚠️ checkstyle 1m 12s hbase-server: The patch generated 2 new + 15 unchanged - 0 fixed = 17 total (was 15)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 19m 8s Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1.
+1 💚 spotbugs 2m 21s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 13s The patch does not generate ASF License warnings.
50m 39s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3346/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #3346
JIRA Issue HBASE-25932
Optional Tests dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname Linux 61de2104f512 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2.4 / 1972261
Default Java AdoptOpenJDK-1.8.0_282-b08
checkstyle https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3346/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count 86 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3346/1/console
versions git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 8s Docker mode activated.
-0 ⚠️ yetus 0m 6s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2.4 Compile Tests _
+1 💚 mvninstall 4m 23s branch-2.4 passed
+1 💚 compile 1m 6s branch-2.4 passed
+1 💚 shadedjars 6m 51s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 43s branch-2.4 passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 58s the patch passed
+1 💚 compile 1m 6s the patch passed
+1 💚 javac 1m 6s the patch passed
+1 💚 shadedjars 6m 48s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 39s the patch passed
_ Other Tests _
+1 💚 unit 137m 32s hbase-server in the patch passed.
166m 42s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3346/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #3346
JIRA Issue HBASE-25932
Optional Tests javac javadoc unit shadedjars compile
uname Linux 1be39224e3c8 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2.4 / 1972261
Default Java AdoptOpenJDK-11.0.10+9
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3346/1/testReport/
Max. process+thread count 4132 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3346/1/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 14s Docker mode activated.
-0 ⚠️ yetus 0m 7s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2.4 Compile Tests _
+1 💚 mvninstall 4m 2s branch-2.4 passed
+1 💚 compile 0m 59s branch-2.4 passed
+1 💚 shadedjars 6m 32s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 39s branch-2.4 passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 41s the patch passed
+1 💚 compile 0m 58s the patch passed
+1 💚 javac 0m 58s the patch passed
+1 💚 shadedjars 6m 33s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 35s the patch passed
_ Other Tests _
+1 💚 unit 206m 21s hbase-server in the patch passed.
233m 40s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3346/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR #3346
JIRA Issue HBASE-25932
Optional Tests javac javadoc unit shadedjars compile
uname Linux cc2c7b2bb199 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2.4 / 1972261
Default Java AdoptOpenJDK-1.8.0_282-b08
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3346/1/testReport/
Max. process+thread count 2530 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3346/1/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@bharathv bharathv merged commit 8a41cb7 into apache:branch-2.4 Jun 1, 2021
@bharathv bharathv deleted the HBASE-25932-branch-2.4 branch June 1, 2021 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants