HDFS-17845. DataStreamer QuotaExceededException should be thrown to Client.#8321
Open
balodesecurity wants to merge 2 commits intoapache:trunkfrom
Open
HDFS-17845. DataStreamer QuotaExceededException should be thrown to Client.#8321balodesecurity wants to merge 2 commits intoapache:trunkfrom
balodesecurity wants to merge 2 commits intoapache:trunkfrom
Conversation
…lient. Previously, QuotaExceededException was silently swallowed in the DataStreamer exception handler: it was logged only at DEBUG level and streamerClosed was not set, so the client never received the error and the stream stalled indefinitely. Changes: - Remove the special-case DEBUG log for QuotaExceededException; all DataStreamer exceptions now emit a WARN log. - Set streamerClosed=true and notify dataQueue when a QuotaExceededException is caught, so the client thread is unblocked and the exception propagates immediately. - Add TestAbandonBlock#testQuotaExceptionPropagatedToClient to verify that DSQuotaExceededException reaches the caller and that a WARN log entry is emitted. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes two related problems when a
QuotaExceededException(e.g. disk-space quota) is thrown duringDataStreamer.addBlock():DEBUGlevel, making quota failures invisible in production logs.lastExceptionis never surfaced, leaving the client blocked indefinitely.Changes
DataStreamer.javaQuotaExceededException-specificDEBUGbranch; all DataStreamer exceptions now emit aWARNlog.lastException, setstreamerClosed = trueandnotifyAll()ondataQueueforQuotaExceededExceptionso the client thread is unblocked immediately.TestAbandonBlock.javatestQuotaExceptionPropagatedToClient: sets a 1-byte space quota on a directory, appends enough bytes to trigger a secondaddBlock, and asserts that:DSQuotaExceededExceptionis propagated to the caller.WARN-level"DataStreamer Exception"log entry is emitted.Test plan
TestAbandonBlock#testQuotaExceptionPropagatedToClientpasses locally ✅mvn package ... -DskipTests) ✅