Development

Development

The package is bd_archive; the console command is bd-archive. Runtime Python requirements and dependencies are defined in pyproject.toml.

Local environment

Create the project-local environment once (or use the existing .venv):

uv venv --python 3.11 .venv
uv pip install --python .venv/bin/python -e '.[dev]'
.venv/bin/python -m bd_archive --help

Use .venv binaries for Python, linting, tests and package commands when working in the repository. External tools are still installed separately. Read the repository's AGENTS.md before changing code.

Source installs also need a C compiler, libc headers and Linux userspace headers (see Installation). hatch_build.py builds src/bd_archive/_native/burn_timeout.so for editable and wheel installs. After changing its C source or build flags, rerun the editable install command; Python source edits alone do not require a native rebuild.

Checks

.venv/bin/python -m unittest discover -s tests
.venv/bin/ruff check .
.venv/bin/ruff format --check .

Some tests run real external tools with temporary data and skip when those tools are unavailable. Check skip counts before interpreting results. The current CI workflow runs Ruff lint and formatting checks plus test_burn_timeout.py; it does not run the complete unittest suite. No physical burn is required to run the tests.

Code structure

Layer	Responsibility
cli.py	Argument parser, completion, dispatch and top-level error handling
commands/	User workflows: create (DAR and raw), burn, verify, extract
archive/	Inventories, checksums, disc access, sizing, raw format, DAR chain discovery and verification
tools/	Wrappers around DAR, PAR2, mkisofs, growisofs and device/mount tools
shell/	Subprocess execution, dependency checks and byte formatting
ui/	Logging, prompts, progress and key input shared across layers

The dependency direction is commands → archive → tools → shell. _par2_helper.py is invoked by DAR's per-slice hook to generate recovery while new slice data may still be in the operating system's page cache.

Raw creation inventories the source and checks stat signatures after hashing, recovery generation and folder/ISO creation. DAR creates all slices first, then moves generated files into disc folders on the same filesystem; cross-filesystem workdirs require copies. Non-moved files (including raw payload and packed files) try tools/reflink.py first. Its FICLONE wrapper publishes a temporary file only after cloning and metadata preservation succeed. Unsupported-operation errors fall back to copy_with_progress; storage, permission and I/O failures propagate. Per-disc counters report logical bytes cloned, copied and moved. --iso retains the image assembly path; extraction's staging copy is unchanged.

archive/disc_folder.py handles folder preparation, exact mkisofs sizing and the off-disc version-1 manifest. A complete manifest makes the set burnable; missing/invalid manifests and changed file metadata are rejected. archive/sizing.py:disc_write_bytes converts filesystem/image lengths to full 32-KiB growisofs write blocks. Creation and burn capacity gates, raw recovery planning and the burn media-class comparison use this padded length. The manifest keeps the unpadded image length. Boundary tests exercise both burn inputs, final raw/DAR ISO gates, folder preparation and recovery planning, including exact fits and failures caused only by the final partial block. tools/mkisofs.py shares filesystem arguments between sizing and growisofs, which pins its MKISOFS backend to the executable used for sizing. Integration tests compare computed sizes to actual ISO bytes and restore multi-disc and packed archives; burn-device operations are mocked. Reflink tests cover fallback, metadata preservation, cancellation and fatal errors. Real clone tests create small temporary fixtures on the project's filesystem and verify independence after edits to the source and clone. Those tests skip when reflinks are unsupported.

archive/hardlinks.py tracks regular-file paths by (st_dev, st_ino) in the raw and DAR source scans. Creation rejects repeated identities among selected paths; DAR checks after auto-deferral when enabled. Tests cover external links, independent copies/reflinks, empty files, filesystem identity, exclusion handling and rejection before archive or recovery generation in both output formats.

archive.verify.verify_disc is shared by standalone verification and the post-burn check. extract stages and verifies slices itself, fetching PAR2 only for failed slices before extracting generations in order.

Native write timeout helper

tools/burn_timeout.py validates the timeout and selected block device, opens and pins the growisofs executable and the packaged shared library, rejects set-id/file-capability binaries and conflicting LD_PRELOAD/LD_AUDIT, and runs a constructor probe which exits before growisofs main. The actual process uses the same open files via /proc/self/fd and the same loader configuration. No optical drive is accessed by the probe. Loader errors prevent the burn.

_native/burn_timeout.c interposes ioctl(SG_IO) for outbound WRITE(10), WRITE(12) and WRITE AND VERIFY(10) commands to the selected block-device identity. It increases shorter timeouts, preserving longer values, payloads and kernel output fields. The caller's timeout field and the actual ioctl errno/return value are preserved on return. Forked children do not apply the override; preload configuration is removed before any child exec, including mkisofs and growisofs's unmount helper. This depends on growisofs's Linux SG_IO write path.

Native tests compile a fake device backend and client. They check device and command filtering, timeout overrides, unchanged data/status/errno, child isolation, and rejection of missing/invalid helpers or unsupported launch settings. An installed growisofs is probed when available. These tests do not burn media; real drive behavior still requires a supervised physical burn.

Versioning and builds

Hatchling and hatch-vcs derive the version from Git tags and generate src/bd_archive/_version.py. That file is gitignored; do not edit or commit it. __init__.py provides an installed-metadata fallback. The version is embedded in the ISO publisher field.

uv pip install --python .venv/bin/python build
.venv/bin/python -m build

The outputs are an sdist and a platform-specific py3-none-linux_* wheel under dist/. The C helper uses no Python ABI, but the wheel must match the architecture and libc runtime. The sdist includes the C source and build hook; rebuilding it requires the compiler. The native library is gitignored. Refresh an editable installation after changing version tags if its generated version is stale.

The Docker publishing workflow runs for pushed v*.*.* tags and builds amd64 and arm64 images. It does not publish Python distributions to PyPI. Changes to on-disc layout require an appropriate version tag under the repository's conventions.

Documentation maintenance

Keep the repository README focused on installation, a first command sequence and links to this wiki. Put workflows here, and update the CLI reference when build_parser() or command validation changes. Wiki pages use extensionless relative links and _Sidebar.md for navigation.

This documentation was updated on 2026-09-12 for the working-tree change that makes disc folders the default output and adds create --iso. The most relevant implementation locations are:

Documentation	Source to review
CLI/defaults	src/bd_archive/cli.py, validation in commands/create.py and commands/create_raw.py
Raw format	archive/raw.py, commands/create_raw.py, tools/par2.py
DAR creation, incrementals, packing	commands/create.py, archive/source_scan.py, tools/dar.py
Burning	commands/burn.py, tools/growisofs.py, archive/disc.py
Integrity scope	archive/verify.py, archive/checksums.py, commands/verify.py
Restore and staging	commands/extract.py, tools/dar.py
Capacity and layout	archive/disc_folder.py, archive/sizing.py, tools/mediainfo.py, tools/mkisofs.py, constants.py
Installation and containers	pyproject.toml, Dockerfile, .github/workflows/

Paths without a prefix in this table are relative to src/bd_archive/, except repository-level files and .github/. Check actual code, not only comments or old usage examples: staging, cleanup, checksum coverage and exit-code semantics are especially easy to overstate.