Skip to content

MAPREDUCE: constant-time compare when verifying the shuffle HMAC - #8687

Open
nishat-06 wants to merge 1 commit into
apache:trunkfrom
nishat-06:shuffle-hmac-consttime
Open

MAPREDUCE: constant-time compare when verifying the shuffle HMAC#8687
nishat-06 wants to merge 1 commit into
apache:trunkfrom
nishat-06:shuffle-hmac-consttime

Conversation

@nishat-06

@nishat-06 nishat-06 commented Aug 17, 2026

Copy link
Copy Markdown

Description of PR

SecureShuffleUtils.verifyHash checks the caller-supplied MAC against the recomputed HMAC with WritableComparator.compareBytes, which returns as soon as two bytes differ. This switches the check to MessageDigest.isEqual so the compare runs in constant time and no longer depends on how many leading bytes happen to match. It matches how the rest of the tree already compares secrets (MessageDigest.isEqual in the delegation-token and auth Signer paths). This is a hardening cleanup with no behavior change: a matching MAC still verifies and a wrong one is still rejected.

How was this patch tested?

Added TestSecureShuffleUtils in the mapreduce-client-core security package covering verifyReply accepting a matching hash and rejecting a tampered or truncated one, and ran it plus the existing TestFetcher against the module build on JDK 17 (both green). Checkstyle on the touched files is clean.

For code changes:

  • Does the title of this PR start with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: Have the integration tests been executed and the endpoint
    declared according to the connector-specific documentation? Note: Automated CI
    testing doesn't cover all cases so manual testing with cloud storage is still
    required.
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

AI Tooling

If an AI tool was used:

@hadoop-yetus

Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 1m 5s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 47m 4s trunk passed
+1 💚 compile 1m 4s trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 compile 1m 5s trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 checkstyle 1m 1s trunk passed
+1 💚 mvnsite 1m 10s trunk passed
+1 💚 javadoc 0m 51s trunk passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javadoc 0m 49s trunk passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 spotbugs 1m 56s trunk passed
+1 💚 shadedclient 35m 16s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 38s the patch passed
+1 💚 compile 0m 35s the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javac 0m 35s the patch passed
+1 💚 compile 0m 37s the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 javac 0m 37s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 31s the patch passed
+1 💚 mvnsite 0m 40s the patch passed
+1 💚 javadoc 0m 23s the patch passed with JDK Ubuntu-21.0.11+10-1-24.04.2-Ubuntu
+1 💚 javadoc 0m 22s the patch passed with JDK Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
+1 💚 spotbugs 1m 39s the patch passed
+1 💚 shadedclient 34m 16s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 8m 55s hadoop-mapreduce-client-core in the patch passed.
+1 💚 asflicense 0m 37s The patch does not generate ASF License warnings.
142m 7s
Subsystem Report/Notes
Docker ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8687/1/artifact/out/Dockerfile
GITHUB PR #8687
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 72840c0e11ed 5.15.0-185-generic #195-Ubuntu SMP Fri Jun 19 17:11:50 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 5a9eae8
Default Java Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.11+10-1-24.04.2-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.19+10-1-24.04.2-Ubuntu
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8687/1/testReport/
Max. process+thread count 1085 (vs. ulimit of 10000)
modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8687/1/console
versions git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens MapReduce shuffle authentication by eliminating data-dependent timing behavior during HMAC verification in SecureShuffleUtils, addressing a potential timing-oracle on the shuffle port. It also adds targeted unit tests to ensure reply-hash verification correctly accepts valid MACs and rejects tampered/truncated inputs.

Changes:

  • Replace WritableComparator.compareBytes with MessageDigest.isEqual for constant-time HMAC comparison in SecureShuffleUtils.verifyHash.
  • Add TestSecureShuffleUtils to validate verifyReply behavior for matching, tampered, and truncated hashes.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/SecureShuffleUtils.java Switches shuffle HMAC verification to constant-time MessageDigest.isEqual.
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/security/TestSecureShuffleUtils.java Adds unit tests covering verifyReply acceptance and rejection (tamper/truncation).

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@pan3793

pan3793 commented Aug 18, 2026

Copy link
Copy Markdown
Member

... so anyone able to reach the shuffle port gets a timing oracle on a keyed MAC and can recover a valid hash byte by byte to read another job's map outputs.

@nishat-06 I understand that from a security perspective, we should use the constant-time method MessageDigest.isEqual for key comparison. However, this method only constitutes a very small part of the entire call chain, and the minute difference in its call time will be drowned out by the latency noise of the entire call chain. Therefore, I believe your claimed cracking method is invalid, unless you can prove it in a real setup.

@nishat-06

Copy link
Copy Markdown
Author

Fair point, and I'll concede the framing. I don't have a real-setup PoC, and you're right that a remote byte-by-byte recovery is drowned out by the latency and jitter across the full request path, so the practical-exploit language in the description overstates it.

I'd rather keep this as plain hardening than an active-exploit claim: it makes the MAC check constant-time and consistent with how the rest of the tree already compares secrets (MessageDigest.isEqual in the delegation-token and Signer paths), with no behavior change. Happy to reword the description to drop the exploitability claim if that reads better to you.

@pan3793

pan3793 commented Aug 18, 2026

Copy link
Copy Markdown
Member

Happy to reword the description to drop the exploitability claim if that reads better to you.

That makes sense. Do you have a JIRA account? The Hadoop project requires each patch to have a dedicated JIRA ticket.

@nishat-06

Copy link
Copy Markdown
Author

Reworded the description to drop the exploitability claim and keep it as a plain constant-time hardening change.

On JIRA: I don't have an account set up. If you're able to file a ticket for this it'd be a big help, and I'll update the PR title to reference the id once it's created.

@pan3793

pan3793 commented Aug 18, 2026

Copy link
Copy Markdown
Member

I don't have permission to approve your JIRA account application. I just created MAPREDUCE-7543 for this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants