Skip to content

Storage and layout

Xitee edited this page Oct 11, 2026 · 13 revisions

Storage and layout

Plan output storage and temporary storage separately. -o is the persistent output; -w selects working storage. By default, work lives under <output>/.bd-archive-work/ on the same filesystem.

Space needed

Operation Persistent output Temporary storage
Raw folder creation One disc folder: reflinks or hardlinks need no second payload allocation on the source filesystem; other files need space for verified copies; plus attachments, recovery and metadata Sparse filesystem/recovery stand-ins for planning
DAR folder creation All disc folders, burn manifest, isolated catalog and sidecars All DAR slices and recovery are created before folder assembly
Raw --iso One ISO containing payload, recovery and metadata The disc tree (reflinks, hardlinks or copies) plus recovery and metadata, removed after the ISO is written
DAR --iso All ISOs, catalog and sidecars All slices/recovery plus one ISO under construction during assembly
DAR restoration Restored files and chain marker; corruption report if needed All collected slices/catalogs; PAR2 fetched for damaged slices

Default DAR creation moves generated slices and recovery files into disc folders on the same filesystem. This preserves their existing data blocks: no second archive-sized write and no extra ISO under construction. Small catalogs and READMEs use reflinks when supported, or normal copies. With a workdir on another filesystem, transfer generally needs a normal copy; the work filesystem must hold all temporary slices/recovery, and the output filesystem must hold the complete result. A RAM workdir still results in only one persistent payload write, provided it does not swap.

For DAR --iso on one filesystem, budget the full archive plus recovery, one image under construction, catalogs and a margin. Each slice and its recovery are removed from scratch after its ISO is complete. For separate filesystems, allow the full scratch archive on one and the complete images on the other.

Raw creation builds the disc folder from the selected units with a Linux FICLONE reflink per file, otherwise a hardlink on the same filesystem, otherwise a verified copy; attachments use a reflink or a verified copy. DAR packing tries a reflink for each file that needs copying, otherwise a normal copy. On a compatible filesystem (including Btrfs), reflinked source and output files initially share data blocks. They are separate files: later writes to either allocate private blocks for the affected data. A hardlink is the same file under a second name. Source and destination must support cloning or linking between them; different filesystems and unsupported file configurations fall back to copying. Permission, I/O and storage errors are reported rather than hidden by a fallback. See Raw data discs.

Packing keeps the original folder/ISO. Folder-to-folder packing can use reflinks; a loop-mounted ISO normally requires copying its old payload to the output filesystem. Account for a full copy when reflinks are unavailable, and for additional copy-on-write allocation if either file changes later. Compression ratios are estimates. Reflinks do not reduce the bytes needed on the optical disc.

Each raw disc reports reflinked, hardlinked and copied byte totals; DAR discs report reflinked, copied and moved. These are logical file sizes by transfer method, not measurements of SSD writes or physical space savings. Permissions and mtimes are preserved. Restore staging is unchanged.

During extraction, all slices remain in staging while files are restored. Space must cover staged data plus the restored tree (including any larger intermediate generation state), repair files and overhead. A single-disc-sized workdir is insufficient for a multi-disc restore.

A tmpfs workdir reduces disk writes only if it has enough available capacity. There is no automatic spillover to another directory when -w fills. tmpfs may use swap; -o still determines where final disc folders, ISOs or restored files are written.

Workdir lifecycle

Use a dedicated workdir per operation, not a folder containing unrelated data. DAR creation owns <workdir>/tmp/ and removes that tree after successful creation. Raw creation uses a uniquely named temporary subdirectory for --iso and otherwise needs no workdir.

The default hidden workdir is cleaned after completed creation or extraction. Completed extraction also cleans it when it reports unrepaired corruption; earlier fatal failures can leave scratch behind. Custom extraction workdirs retain slices. Custom creation workdirs remain, but their generated scratch is cleaned on success.

Capacity and estimates

create detects writable bytes using dvd+rw-mediainfo, or uses -b BYTES. The detector reads positive Free Blocks, falling back to Track Size, and multiplies the reported count by 2048. Disc marketing capacity in GB and displayed GiB are different units; pass a byte count to -b.

DAR sizing subtracts a 1 MiB end margin, estimated catalog size and a 4 MiB PAR2/miscellaneous allowance, then accounts for recovery percentage and rounds slices down to whole MiB. That allowance is retained even with recovery disabled. Raw planning and --fill-recovery sizing use mkisofs -print-size with sparse payload and recovery placeholders and a 1 MiB margin. DAR --fill-recovery sizes each slice's recovery from its actual size and the slice sizing's reserve, without a separate measurement. Both modes check the final folder using mkisofs -print-size, which calculates the exact sector count with the same filesystem options used for burning. It reads directory/file metadata without writing the ISO payload. Optional ISO output is checked by its actual file size as well. All final capacity checks round image lengths up to full 32-KiB growisofs write blocks, including the zero padding at the end. Raw recovery planning includes this rounding before adding the 1 MiB safety margin. Burn checks and its media-class comparison use the padded length for both output formats. Image files and manifest image_bytes keep the actual filesystem length; existing manifests remain compatible and are remeasured before folder burning.

During folder burning, mkisofs generates the filesystem directly into growisofs. burn recomputes size after the insertion prompt and checks file metadata for changes. This removes the intermediate ISO write; it does not protect against I/O errors or concurrent changes during the burn. Keep folders unchanged and retain post-burn verification.

Burning uses spare=none to avoid a new spare-area reservation on blank, unformatted BD-R media. Detection does not walk format-capacity descriptors; do not assume identical capacity for previously formatted media. burn checks the inserted disc independently, subject to the exceptions on Burning.

Output on disk

Default raw output:

photos-2026-10/
├── manifest.json
└── disc/
    ├── Photos-2026-10/
    ├── Info.txt
    ├── README.txt
    └── .bd-archive/
        ├── checksums.sha512
        ├── recovery.par2
        ├── recovery.vol....par2
        └── raw-v3

Every raw run writes exactly one disc, so the disc folder is simply disc/, with its burn manifest beside it; choose a distinct output directory and -n per disc. With --iso, the raw image is images/disc_0001.iso.

Default DAR output (two discs):

photos-gen1/
├── discs/
│   ├── manifest.json
│   ├── disc_0001/Photos-gen1/...
│   └── disc_0002/Photos-gen1/...
├── Photos-gen1-catalog.0001.dar
└── Photos-gen1-catalog.0001.dar.sha512

discs/manifest.json is published after the complete set is ready. It records consecutive folder names, volume labels, publisher, Rock Ridge setting, computed image sizes and a relative-path metadata inventory. It stays outside the contents burned to disc. Preserve it and the complete folder set for burn. Move/copy the whole output together, retaining file permissions and mtimes.

With create --iso, images/disc_0001.iso, images/disc_0002.iso, etc. replace the discs/ tree; the catalogs stay beside images/. Existing ISO sets remain burnable without a manifest. Keep numbered names unchanged and keep every catalog slice and its checksum sidecar.

Folder output and ISO output have the same on-disc contents.

Files on a raw disc

For -n Photos-2026-10 --attach /data/disc-info/Info.txt:

/
├── Photos-2026-10/
│   └── ... selected units, with paths relative to the source ...
├── Info.txt
├── README.txt
└── .bd-archive/
    ├── checksums.sha512
    ├── recovery.par2
    ├── recovery.vol....par2
    └── raw-v3

Recovery files are omitted with -r 0. Checksums and PAR2 record paths relative to the disc root, including Photos-2026-10/, and cover the attachments too. The raw-v3 marker identifies this layout; it tells the verifier that the root README.txt, written after the checksums, is not payload. Everything else outside .bd-archive/ is payload.

Older raw discs remain supported by verification:

  • Source folder with README.txt, checksums.sha512 and recovery files at the root, recognized from those files without a marker.
  • Payload at the root with metadata under .bd-archive/, including existing discs with the former marker files.

Files on a DAR disc

Example for generation 1, disc 1, with recovery:

/
└── Photos-gen1/
    ├── Photos-gen1.0001.dar
    ├── Photos-gen1.0001.dar.sha512
    ├── Photos-gen1.0001.dar.par2
    ├── Photos-gen1.0001.dar.vol....par2
    ├── Photos-gen1-catalog.0001.dar
    ├── Photos-gen1-catalog.0001.dar.sha512
    └── README.txt

Later discs carry their corresponding slice, sidecar, recovery files and README. The isolated catalog goes on disc 1 only; the embedded master catalog is in the last data slice. In a one-disc archive, both copies are on the same disc.

A packed disc contains a folder for each archive/generation. Labels look like Photos_G01_0001, with a trailing + on a packed disc. Labels fit the 32-byte ISO9660 limit, so the name portion may be shortened. DAR filenames keep the full name and determine chain/generation identity during restoration.

Both modes use UDF + ISO9660 on disc; raw mode also enables Rock Ridge. Legacy flat DAR layouts and filenames without -gen<N> remain readable; the latter are interpreted as generation 1.

Generated README

Each generated README.txt is a compact archive fact sheet. It records ARCHIVE NAME, optional DESCRIPTION (set with --description) and FORMAT; DAR archives also record generation, disc number and compression. RAW format is UDF / ISO 9660 (Rock Ridge); DAR format is DAR. The checksum section contains only Algorithm (SHA-512) and File paths. Recovery information lists PAR2 coverage, redundancy, index and recovery-volume paths, or RECOVERY: None when disabled. All paths are relative to the README; wildcards identify sets of sidecar or recovery files. RAW and DAR use the same README writer, called once per disc after archive and recovery creation. RAW always records the actual recovery bytes and percentage; DAR records the -r value, or the actual recovery of a slice extended by --fill-recovery, and its disc-specific slice and catalog paths. RAW capacity planning reserves one 32-KiB write block for this README in addition to the disc-end margin. The UTF-8 size of an optional description, including continuation-line indentation, is reserved separately and rounded up to a write block.

The software section records bd-archive, Python, argcomplete and the creation tools selected for that run, with versions and project links. DAR also includes its reported libdar version. PAR2 is omitted when disabled; media detection and ISO mounting tools appear only when needed. Versions are collected once at startup, before archive creation. Failed queries, timeouts and unrecognized output abort creation immediately. Tools without a version-only query (dvd+rw-mediainfo and udisksctl) are listed with their name and project link only. No package lookup or manual version entry is required. These details refer to creation, not a later burn or verification run.

A raw README additionally has a FILES list of the disc root: the folder holding the archived files, each attachment, and the hidden .bd-archive/ folder with the checksums and recovery files, noting that file managers may not show names starting with a dot:

FILES:
  Photos-2026-10/  Archived files
  Info.txt         Attached file
  .bd-archive/     Checksums and recovery files; hidden folder, file managers may not show names starting with a dot

READMEs contain no installation, verification or repair commands. Operational instructions remain in Verification, Restoring and Raw-data-discs.

Clone this wiki locally