Skip to content

create: add --read-special-timeout option - #10090

Merged
ThomasWaldmann merged 1 commit into
borgbackup:masterfrom
ThomasWaldmann:read-special-timeout-5422
Aug 13, 2026
Merged

create: add --read-special-timeout option#10090
ThomasWaldmann merged 1 commit into
borgbackup:masterfrom
ThomasWaldmann:read-special-timeout-5422

Conversation

@ThomasWaldmann

@ThomasWaldmann ThomasWaldmann commented Aug 12, 2026

Copy link
Copy Markdown
Member

Fixes #5422.

With --read-special, backing up a fifo blocked forever when no writer ever connects: the read-only open() blocks until a writer shows up, and read() blocks until data arrives. To the user it just looks like borg hangs, and an unattended backup never completes.

What this does

Adds borg create --read-special-timeout SECONDS, default 1800 (30 minutes), so users who don't know the option exists are protected too. --read-special-timeout=0 keeps the previous wait-forever behavior.

  • fifos and char devices are opened with O_NONBLOCK (the open then succeeds even without a connected writer); block devices are unaffected and keep the current fast path.
  • content reads go through the new SpecialFileReader (helpers/fs.py), which waits for data via select() and raises OSError(ETIMEDOUT) when no data arrives for SECONDS. The timeout bounds the gaps in the data flow, including waiting for the writer to connect - a slow producer that keeps sending never times out, a stalled or never-connecting one does. That matches the request in the issue for something configurable, since a legitimate producer may take a while before it starts sending.
  • the timeout maps to the new BackupTimeoutError (rc 111): the file is skipped with an error, not retried (retrying would just wait SECONDS again, 10 more times), and the backup continues with the remaining files - consistent with how other per-file IO errors behave.

Behavior change worth noting (documented in usage/notes.rst): with a timeout active (now the default), a writer that connects but closes without sending anything cannot be distinguished from "no writer at all", so it is reported as a timeout rather than stored as a file with empty content.

No signals and no threads - plain POSIX non-blocking IO, so nothing here interferes with borg's SIGINT handling. select() rather than poll(), because macOS poll() has historically been unreliable for some character devices.

Tests

  • archiver: timeout expiring (fifo without writer → error, file skipped, rest of the backup succeeds, no retries), slow/bursty writer within the timeout (content round-trips), --read-special-timeout without --read-special errors out, and the big-fifo regression test on both read paths.
  • unit: SpecialFileReader (loop-fill, EOF, silent and stalled writer timeouts, fifo no-writer vs. late writer, ESPIPE on seek) and nonnegative_seconds.
  • manual smoke: dead fifo errors out after exactly the given timeout with rc 111; =0 keeps waiting indefinitely; 1 MiB through a bursty fifo round-trips intact.

🤖 Generated with Claude Code

@ThomasWaldmann ThomasWaldmann self-assigned this Aug 12, 2026
@codecov

codecov Bot commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.92%. Comparing base (0ccf95b) to head (5b9da3b).
⚠️ Report is 1 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #10090      +/-   ##
==========================================
+ Coverage   86.82%   86.92%   +0.09%     
==========================================
  Files          99       99              
  Lines       17342    17404      +62     
  Branches     2631     2642      +11     
==========================================
+ Hits        15057    15128      +71     
+ Misses       1589     1582       -7     
+ Partials      696      694       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Reading from a fifo (or char device) with --read-special could block
borg forever: opening a fifo read-only blocks until a writer connects,
and reads block until data arrives - a broken producer thus hangs an
unattended backup indefinitely.

Now, when no data arrives for --read-special-timeout SECONDS (default:
1800, i.e. 30 minutes; this includes waiting for a fifo's writer to
connect), borg gives up on that file: fifos and char devices are opened
with O_NONBLOCK and all content reads go through the new
SpecialFileReader, which waits for data via select() and raises
OSError(ETIMEDOUT) when the timeout expires. That maps to the new
BackupTimeoutError (rc 111), so the file is skipped with an error (and
not retried) while the backup continues with the remaining files.
--read-special-timeout=0 means: wait forever (previous behavior).

Note: with a timeout active (the default), a writer connecting but
closing without sending anything is reported as a timeout instead of
being stored as a file with empty content.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ThomasWaldmann
ThomasWaldmann force-pushed the read-special-timeout-5422 branch from 2074aa6 to 5b9da3b Compare August 13, 2026 09:56
@ThomasWaldmann
ThomasWaldmann merged commit 6fd5665 into borgbackup:master Aug 13, 2026
20 checks passed
@ThomasWaldmann
ThomasWaldmann deleted the read-special-timeout-5422 branch August 13, 2026 11:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a timeout on opening/reading from UNIX pipes when --read-special option is active

1 participant