Skip to content

v0.5.467

v0.5.467: [dataloader] Improve exception handling (#654)
@cbb330 cbb330 tagged this 31 Jul 21:27 
## Summary

**Problem:** OpenHouse DataLoader errors could not be reliably
correlated with the corresponding Tables Service request.
Timestamp-based correlation is noisy, and authentication failures were
represented as generic `OSError`s and retried as if they were transient
I/O failures.

**Solution:** assign a unique `X-Request-ID` to every outbound catalog
HTTP request and expose it through shared, typed catalog exceptions. The
exception hierarchy centralizes request-ID formatting and distinguishes
authentication, authorization, not-found, transport, HTTP, and
malformed-response failures.

## Changes

- [x] Client-facing API Changes
- [x] Internal API Changes
- [x] Bug Fixes
- [ ] New Features
- [ ] Performance Improvements
- [ ] Code Style
- [x] Refactoring
- [ ] Documentation
- [x] Tests

Details:

- Added a central request-aware HTTP session that generates a fresh UUID
for every prepared request.
- Added shared exception types:
  - `OpenHouseCatalogError`
  - `OpenHouseRequestError`
  - `OpenHouseTransportError`
  - `OpenHouseHTTPError`
  - `OpenHouseAuthenticationError`
  - `OpenHouseAuthorizationError`
  - `OpenHouseNoSuchTableError`
  - `OpenHouseInvalidResponseError`
- Centralized `X-Request-ID` rendering and exposed it as
`exception.request_id`.
- Preserved compatibility with callers catching PyIceberg
`NoSuchTableError`.
- Classified 401/403 and other non-transient 4xx failures as
non-retryable.
- Kept transport failures, HTTP 408/429, and 5xx responses retryable.

## Testing Done

- [ ] Manually Tested on local docker setup. Please include commands
ran, and their output.
- [x] Added new tests for the changes made.
- [x] Updated existing tests to reflect the changes made.
- [ ] No tests added or updated. Please explain why. If unsure, please
feel free to ask for help.
- [x] Some other form of testing like staging or soak time in
production. Please explain.

Validation performed:

- `make verify`
  - Ruff lint passed
  - Ruff formatting passed
  - Mypy passed
  - **272 unit tests passed**
- Added coverage for unique request IDs, typed 401/403/404/500 errors,
transport errors, malformed JSON, missing/empty `tableLocation`,
retryable failures, and non-retryable authentication failures.
- Used a known request ID against the deployed OpenHouse route and
confirmed it appeared in the Nginx access log; follow-up proxy
configuration work is tracked separately to preserve the same value
through Ambassador.

# Additional Information

- [ ] Breaking Changes
- [ ] Deprecations
- [ ] Large PR broken into smaller PRs, and PR plan linked in the
description.
Assets 2
Loading