PDFCLOUD-5382 Add retries and logging

[datalogics-kam]

PDFCLOUD-5382

Summary

Adds configurable, robust retries with exponential backoff and jitter to the pdfRest clients, along with structured, sanitized logging for requests, responses, and retry behavior. Improves error handling and observability, and introduces comprehensive tests covering success-after-retry, limits, and edge cases.

Changes

• Add retry logic and backoff mechanism for client requests

  • Exponential backoff with jitter; configurable max_retries.
  • Enhanced handling of transport, timeout, and server-side errors.
  • Tests for retry success, error propagation after limits, negative limits, and delay calculations.

• Add logging to pdfRest clients

  • Structured logs for requests/responses, attempts, delays, payloads, and errors.
  • Sanitized sensitive headers in log output.
  • Exception handling augmented with detailed log context.
  • Logging can be enabled by setting PDFREST_LOG to either info or debug

[datalogics-cgreen]

Please rebase to eliminate the duplicate commits.

[datalogics-kam]

Please rebase to eliminate the duplicate commits.

Done.

Technically there aren't duplicate commits because there isn't a Y-shaped branch pattern with the same commits on each side of the Y-fork. But I did it anyway, because GitHub doesn't clean up the commit list unless something happens.

[datalogics-cgreen]

So I had Codex review Codex, and:

- Retry delay ignores Retry-After: both sync and async paths always sleep for the exponential/
  jittered delay computed in src/pdfrest/client.py:476-784 and never look at Retry-After headers
  from 429/5xx responses. The main branch had no retry logic, so this is a regression risk unique
  to the new implementation: rate-limit responses will continue to be retried on the client’s
  schedule rather than the server’s requested window, which can prolong outages and keep the
  client throttled. Consider reading the header (seconds or HTTP-date) and using the larger of the
  exponential and server delays.

I don't believe pdfRest ever returns a 429 response, so maybe this is irrelevant.

- 408 Request Timeout responses are treated as permanent failures: _is_retryable_status only
  whitelists 429 and ≥500 (src/pdfrest/client.py:465-474). If pdfRest ever returns HTTP 408
  (server-side timeout) you now raise immediately even though the same scenario handled via network
  exceptions (httpx.TimeoutException → PdfRestTimeoutError) is retried. This asymmetry didn’t exist
  on main (no retries) but is easy to fix by including 408 (and possibly other transient 4xx such
  as 425/499 if pdfRest emits them) in the retryable set so behaviour matches server guidance.

Similar deal with 408s?

- Tests only cover server 500/503 flows: the retry/backoff helpers are exercised in tests/
  test_client.py:196-276, but there’s no coverage for 429/rate-limit handling, no assertion that
  timeouts/transport errors are retried, and no test that download_file/streaming calls also
  route through _execute_with_retry. Given how central this logic now is, that leaves gaps in
  confidence compared to main. Adding parameterized tests that simulate 429 with mocked Retry-
  After, httpx.TimeoutException, and download streaming errors would lock in the expected behaviour
  for both sync and async variants.

See item the 1st.

[datalogics-kam]

In the time this PR has been waiting, the pdfRest container actually started to return a 429.

I don't think anything's sending Retry-After unless it's from a gateway put in front of the server.

Thanks for having Codex review Codex. I had Codex review its work, and it didn't find as much, but I gave Codex these concerns to chew on.

[datalogics-kam]

I took another look at this and:

• Removed file rewinding. Uploads with streams are now only retried if they are retrying the inability to connect. That leaves the stream management up to the caller, which will know if the stream can be rewound, a file reopened, or if a retry isn't really possible with that kind of stream.
• create_from_paths has full retry, because it can reopen the files.