fix: preserve original error in IcebergIterator._seek_to_usable_file#5092
Open
mengw15 wants to merge 1 commit into
Open
fix: preserve original error in IcebergIterator._seek_to_usable_file#5092mengw15 wants to merge 1 commit into
mengw15 wants to merge 1 commit into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5092 +/- ##
=========================================
Coverage 43.15% 43.15%
Complexity 2209 2209
=========================================
Files 1045 1045
Lines 40260 40261 +1
Branches 4250 4250
=========================================
+ Hits 17373 17376 +3
+ Misses 21815 21813 -2
Partials 1072 1072
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
62aae9c to
33ecec5
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates Iceberg table scan error handling so _seek_to_usable_file logs failures through Loguru and re-raises the original exception instead of replacing it with an empty generic Exception.
Changes:
- Adds
loguru.loggerimport toiceberg_document.py. - Replaces
print(...)andraise Exceptionwithlogger.exception(err)and bareraise.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The bare `raise Exception` (no args, no `from`) constructed a fresh empty exception, discarding the underlying error type, message, and traceback. Callers doing `except Exception as e: log.error(str(e))` saw only an empty string instead of the real diagnostic (catalog auth failure, S3 IO error, manifest corruption, etc.). The `print` also bypassed the project logger. Replace with a re-raise of the original exception and route the message through loguru. Closes apache#5091. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
33ecec5 to
094d060
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this PR?
IcebergIterator._seek_to_usable_filepreviously swallowed every error during file-scan setup:The bare
raise Exception(no args, nofrom) constructs a freshExceptionwith emptystr()and no__cause__. Callers that doexcept Exception as e: log.error(str(e))see only an empty class name — the original error type, message, and traceback are all lost. Theprintalso bypasses the project logger.This PR replaces the bare re-raise with a true re-raise of the original exception and routes the diagnostic message through
loguru, matching the existingexcept Exception as err: logger.exception(err)pattern used indata_processor.py:125,main_loop.py:422, andinput_port_materialization_reader_runnable.py:169:Callers now see the actual underlying exception (catalog auth failure, S3 IO error, manifest corruption, etc.) with its full class name, message, and traceback.
Any related issues, documentation, discussions?
Closes #5091.
How was this PR tested?
Added
amber/src/test/python/core/storage/iceberg/test_iceberg_iterator_error_paths.py, a slim mocked regression test: it patchesload_table_metadatato return aMockwhoserefresh()raisesRuntimeError("Catalog auth failure: token expired"), drivesnext(IcebergIterator(...)), and asserts the caller observes the originalRuntimeError(viapytest.raises(RuntimeError, match=...)). Locks in the contract that the except clause must not swallow the underlying exception's type/message.Run locally:
Result:
1 passed.Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (claude-opus-4-7)