Incrementals and packing

Incrementals and packing

These features require create -m dar. Start with a full DAR archive.

Multisession is not supported and is not planned for the foreseeable future. Incrementals create new archive output to burn on new discs; --pack-with combines unburned contents before burning. Neither adds files to an already burned disc. See Multisession support and research.

Add an incremental generation

A chain consists of a full archive (generation 1) and later generations recording changes since their predecessor. Keep the same archive name (-n) and point --base at the previous generation's isolated catalog:

bd-archive create -m dar -s /data/photos -n Photos \
  -o /data/photos-gen2 \
  --base /data/photos-gen1/Photos-gen1-catalog.0001.dar

Use the current, complete source tree. This is a comparison against the previous state, not simply an upload of a separate folder of additions: files absent from the new source can be recorded as deleted.

DAR stores new/changed file data and records changes to the archive state. The new generation gets its own disc folders (or optional images) and catalog. To make generation 3, reference the generation 2 catalog and use a new output directory. Burn each generation with its usual burn -i command.

The name embedded in the base catalog filename must match -n, or creation refuses to proceed. The new generation number is the base generation plus one. A legacy catalog without -gen<N> is treated as generation 1.

Keep every generation needed for restoration. The latest incremental is not a standalone backup. An isolated catalog is an index, not a copy of unchanged files. A complete restore normally needs generation 1 and all subsequent generations through the desired state.

The incremental size preview uses catalog paths and file modification times as a heuristic. DAR makes the actual archival decisions; the preview is not an exact byte count of the delta.

Defer new files to avoid a nearly empty last disc

bd-archive create -m dar -s /data/photos -n Photos \
  -o /data/photos-gen2 \
  --base /data/photos-gen1/Photos-gen1-catalog.0001.dar \
  --min-last-disc-fill 50

This tries to reach a projected last-disc fill of at least 50% by excluding newest files from this generation. It does not change files in the source.

With --base, only paths absent from the base catalog are candidates. Files already known to the chain are not deferred merely because their timestamps changed. Without --base, all source files are candidates, and deferred files have no copy in this new archive until you include them in a later run.

The preview shows the deferred count, total size, oldest deferred timestamp and a few paths. Read it before confirming. Keep deferred files in the source and archive them in a later generation; no automatic later run is scheduled.

The threshold is a target, not a guarantee. If candidates run out, creation can continue below the target with a warning. If deferring them would leave zero estimated data bytes, it stops. A value of 0 disables deferral; values above 100 are invalid. Compression estimates also affect this selection.

Pack an unburned leftover disc with the next archive

Suppose the previous set's last disc folder, disc_0003, uses only part of a disc and has not been burned. A new archive can share that space:

bd-archive create -m dar -s /data/videos -n Videos \
  -o /data/videos-gen1 \
  --pack-with /data/photos-gen1/discs/disc_0003

An individual ISO such as /data/photos-gen1/images/disc_0003.iso also works. Pass the individual disc folder, not the entire create output directory. ISOs are loop-mounted read-only for inspection and assembly; folders need no mount. Keep a packed source folder unchanged throughout creation.

The old archive files share the new discs/disc_0001/ (or, with --iso, images/disc_0001.iso) with the new archive's first slice. That slice gets a smaller capacity budget; later slices use the normal budget. The combined disc's label ends with +.

The input must contain DAR slices, leave room for a new slice plus metadata, and not contain the exact name/generation being created. This prepares a new disc; it does not append to a burned disc. Original folders and ISOs remain unchanged. For folder output, old files are reflinked when source and destination support it (for example, on the same Btrfs filesystem). Otherwise they are copied normally, including files read from a loop-mounted ISO. Reflinks preserve independent file contents through copy-on-write; later changes to the original do not alter the prepared output. Allow space for normal copies when cloning is unavailable, and for later copy-on-write allocation. The transfer summary reports which method was used. --iso continues to write the payload into the new image.

The combined disc replaces the old disc in the burn plan. Do not burn the superseded original separately. burn does not track this relationship and an old set still discovers its original disc; keep a record of the replacement. Retain the original until the combined disc has been burned and verified.

Packing also works between generations of the same chain: combine --base with --pack-with, keeping the chain name. During restoration, both generations on the shared disc are collected. For separate chains, extract offers a chain picker and ignores the others. verify checks the discovered recovery/checksum sets across the disc.

Older flat DAR discs are repackaged into archive folders. You can repeat packing with a later leftover folder or image.

Next: Burning · Restoring a chain