Skip to content

Hadoop-19873: ABFS. Fail Prefetch Faster When Request Priority Enabled#8464

Open
manika137 wants to merge 13 commits intoapache:trunkfrom
ABFSDriver:HADOOP-19873_failFaster
Open

Hadoop-19873: ABFS. Fail Prefetch Faster When Request Priority Enabled#8464
manika137 wants to merge 13 commits intoapache:trunkfrom
ABFSDriver:HADOOP-19873_failFaster

Conversation

@manika137
Copy link
Copy Markdown
Contributor

Description of PR

JIRA: https://issues.apache.org/jira/browse/HADOOP-19873
Failing prefetch faster and sending the read request directly as missed cache read type (of higher request priority) when prefetch request priority config is enabled for better performance

How was this patch tested?

Test suite was run- added the results in comments

@manika137
Copy link
Copy Markdown
Contributor Author

============================================================
HNS-OAuth-DFS

[WARNING] Tests run: 237, Failures: 0, Errors: 0, Skipped: 3
[WARNING] Tests run: 860, Failures: 0, Errors: 0, Skipped: 172
[WARNING] Tests run: 159, Failures: 0, Errors: 0, Skipped: 8
[WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 23

============================================================
HNS-SharedKey-DFS

[WARNING] Tests run: 237, Failures: 0, Errors: 0, Skipped: 4
[WARNING] Tests run: 863, Failures: 0, Errors: 0, Skipped: 118
[WARNING] Tests run: 159, Failures: 0, Errors: 0, Skipped: 8
[WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 10

============================================================
AppendBlob-HNS-OAuth-DFS

[WARNING] Tests run: 237, Failures: 0, Errors: 0, Skipped: 3
[WARNING] Tests run: 860, Failures: 0, Errors: 0, Skipped: 183
[WARNING] Tests run: 136, Failures: 0, Errors: 0, Skipped: 9
[WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 23

============================================================
NonHNS-SharedKey-Blob

[WARNING] Tests run: 237, Failures: 0, Errors: 0, Skipped: 10
[WARNING] Tests run: 710, Failures: 0, Errors: 0, Skipped: 154
[WARNING] Tests run: 159, Failures: 0, Errors: 0, Skipped: 3
[WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 11

============================================================
NonHNS-OAuth-Blob

[ERROR] Tests run: 237, Failures: 0, Errors: 0, Skipped: 10
[WARNING] Tests run: 707, Failures: 0, Errors: 0, Skipped: 155
[WARNING] Tests run: 159, Failures: 0, Errors: 0, Skipped: 3
[WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 24

============================================================
AppendBlob-NonHNS-OAuth-Blob

[WARNING] Tests run: 237, Failures: 0, Errors: 0, Skipped: 10
[WARNING] Tests run: 702, Failures: 0, Errors: 0, Skipped: 176
[WARNING] Tests run: 136, Failures: 0, Errors: 0, Skipped: 4
[WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 24

============================================================
HNS-Oauth-DFS-IngressBlob

[WARNING] Tests run: 237, Failures: 0, Errors: 0, Skipped: 3
[WARNING] Tests run: 734, Failures: 0, Errors: 0, Skipped: 181
[WARNING] Tests run: 159, Failures: 0, Errors: 0, Skipped: 8
[WARNING] Tests run: 272, Failures: 0, Errors: 0, Skipped: 23

@hadoop-yetus
Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 34s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 5 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 41m 40s trunk passed
+1 💚 compile 1m 1s trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 compile 1m 4s trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 checkstyle 0m 57s trunk passed
+1 💚 mvnsite 1m 6s trunk passed
+1 💚 javadoc 1m 1s trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 javadoc 1m 0s trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 spotbugs 1m 32s trunk passed
+1 💚 shadedclient 30m 0s branch has no errors when building and testing our client artifacts.
-0 ⚠️ patch 30m 30s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 34s the patch passed
+1 💚 compile 0m 33s the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 javac 0m 33s the patch passed
+1 💚 compile 0m 35s the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 javac 0m 35s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 25s /results-checkstyle-hadoop-tools_hadoop-azure.txt hadoop-tools/hadoop-azure: The patch generated 7 new + 9 unchanged - 0 fixed = 16 total (was 9)
+1 💚 mvnsite 0m 37s the patch passed
+1 💚 javadoc 0m 29s the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 javadoc 0m 31s the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 spotbugs 1m 16s the patch passed
+1 💚 shadedclient 28m 26s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 2m 8s hadoop-azure in the patch passed.
+1 💚 asflicense 0m 35s The patch does not generate ASF License warnings.
117m 51s
Subsystem Report/Notes
Docker ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8464/1/artifact/out/Dockerfile
GITHUB PR #8464
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 1e49c93fe5d5 5.15.0-160-generic #170-Ubuntu SMP Wed Oct 1 10:06:56 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 86615ca
Default Java Ubuntu-17.0.18+8-Ubuntu-124.04.1
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8464/1/testReport/
Max. process+thread count 640 (vs. ulimit of 10000)
modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8464/1/console
versions git=2.43.0 maven=3.9.11 spotbugs=4.9.7
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 32s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 5 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 41m 20s trunk passed
+1 💚 compile 0m 57s trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 compile 1m 2s trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 checkstyle 0m 53s trunk passed
+1 💚 mvnsite 1m 5s trunk passed
+1 💚 javadoc 0m 57s trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 javadoc 0m 55s trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 spotbugs 1m 26s trunk passed
+1 💚 shadedclient 28m 52s branch has no errors when building and testing our client artifacts.
-0 ⚠️ patch 29m 22s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 34s the patch passed
+1 💚 compile 0m 32s the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 javac 0m 32s the patch passed
+1 💚 compile 0m 34s the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 javac 0m 34s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 25s /results-checkstyle-hadoop-tools_hadoop-azure.txt hadoop-tools/hadoop-azure: The patch generated 7 new + 9 unchanged - 0 fixed = 16 total (was 9)
+1 💚 mvnsite 0m 35s the patch passed
+1 💚 javadoc 0m 28s the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04
+1 💚 javadoc 0m 27s the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1
+1 💚 spotbugs 1m 10s the patch passed
+1 💚 shadedclient 27m 50s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 2m 7s hadoop-azure in the patch passed.
+1 💚 asflicense 0m 34s The patch does not generate ASF License warnings.
115m 29s
Subsystem Report/Notes
Docker ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8464/2/artifact/out/Dockerfile
GITHUB PR #8464
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 4e2575fc7a04 5.15.0-160-generic #170-Ubuntu SMP Wed Oct 1 10:06:56 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / f43dd01
Default Java Ubuntu-17.0.18+8-Ubuntu-124.04.1
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8464/2/testReport/
Max. process+thread count 638 (vs. ulimit of 10000)
modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8464/2/console
versions git=2.43.0 maven=3.9.11 spotbugs=4.9.7
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants