RAFAEL

Rust-Based Automated File-System Analyzer and Erasure Logger

RAFAEL is a multithreaded Linux file-system traversal and purge utility written in Rust. It scans directory trees beneath a target root, evaluates entries using file timestamps, removes eligible files and empty directories, and records every deletion—or proposed deletion during a dry run—in per-worker log files.

LANL software release number: O5179

The repository also includes maketree, a companion utility for generating deterministic test trees with configurable depth, branching, file counts, and file sizes.

RAFAEL can permanently delete large directory trees. Always begin with --dry-run, inspect the generated logs, verify the exception file, and confirm the target path before running without dry-run mode. The --erase option is intentionally destructive.

Features

• Parallel traversal with configurable worker threads and work stealing
• Age-based purging using access, change, and modification timestamps
• Optional ctime exclusion for environments where metadata changes should not reset file age
• Full-content erase mode for controlled cleanup operations
• Option to be IO bound or memory bound with the read_entire_dir option
• Required exception file for pruning protected directory subtrees
• Configurable directory-depth protection beneath the target root
• Dry-run mode with the same candidate evaluation and logging behavior as a real run
• Per-worker deletion logs, optional traversal logs, terminal statistics, and local syslog messages
• Optional progress display and per-thread work statistics
• Test-tree generation through the bundled maketree binary

How Eligibility Is Determined

In age mode, a file is eligible only when its newest evaluated timestamp is older than the requested age:

threshold = current time - age in days
eligible  = newest(atime, ctime, mtime) < threshold

With --ignore-ctime, RAFAEL evaluates only the newer of atime and mtime. This flag is primarily used to test against directories generated by the maketree utility as modifying ctime is outside the scope of this project.

Additional behavior:

• Files owned by UID 0 are protected in age mode.
• Symbolic links are evaluated without following them; RAFAEL removes the link itself, not its target.
• A directory is removed only when the directory is eligible, it is outside the protected depth, and its processed descendants do not prevent deletion.
• Exception matches prune the entire matching directory subtree from traversal.
• Block devices, character devices, and FIFOs are not deleted and prevent their parent directory from being removed.
• The target root itself is never removed.
• The current implementation begins work from directories immediately beneath the root. Non-directory entries located directly in the root are not processed.

In --erase mode, timestamp and UID checks are bypassed. Exception pruning, depth protection, root preservation, and special-device protections still apply.

Requirements

• Linux with statx, Unix permissions, and a local syslog interface
• A current Rust toolchain with Cargo
• Read and execute permission throughout the target tree
• Write permission for the selected log location
• Sufficient directory permissions to delete selected entries

Install Rust with the official instructions at https://www.rust-lang.org/tools/install.

Build

Build both binaries in release mode:

cargo build --release --bins

The resulting executables are placed at:

target/release/rafael
target/release/maketree

To build only RAFAEL:

cargo build --release --bin rafael

Quick Start

Create an exception file. A truly empty file is valid when no paths need to be excluded:

touch exceptions.txt

Preview entries older than 30 days:

./target/release/rafael \
  /absolute/path/to/target \
  --age 30 \
  --exception exceptions.txt \
  --dry-run

Review the generated worker logs, then perform the purge by removing --dry-run:

./target/release/rafael \
  /absolute/path/to/target \
  --age 30 \
  --exception exceptions.txt

Usage

rafael <ROOT> --exception <FILE> (--age <DAYS> | --erase) [OPTIONS]

Exactly one purge mode must be selected:

• --age <DAYS> deletes entries older than the specified age.
• --erase deletes all processable contents beneath the target root, subject to exceptions, depth protection, and special-file handling.

Common Examples

Dry-run an age-based purge with eight workers:

rafael /data/scratch \
  --age 45 \
  --exception /etc/rafael/exceptions.txt \
  --threads 8 \
  --dry-run

Ignore ctime and evaluate only atime and mtime:

rafael /data/scratch \
  --age 30 \
  --exception exceptions.txt \
  --ignore-ctime \
  --dry-run

Show a progress display and use a known entry count:

rafael /data/scratch \
  --age 30 \
  --exception exceptions.txt \
  --show-progress \
  --entry-count 5000000 \
  --dry-run

Erase processable contents while protecting the first two directory levels beneath the root:

rafael /data/staging \
  --erase \
  --exception exceptions.txt \
  -d 2

RAFAEL Options

Option	Description	Default
<ROOT>	Root directory whose child directory trees will be traversed	Required
-a, --age <DAYS>	Minimum age in days for age-based deletion	Not selected
--erase	Delete all processable contents regardless of age or UID	Disabled
-e, --exception <FILE>	File containing directory substrings to exclude	Required
-t, --thread-count <COUNT>	Number of traversal workers	4
-l, --rp-log-dir <PATH>	Base path for the timestamped run-log directory	rp_log_dir
--ignore-ctime	Evaluate age using only atime and mtime	Disabled
-d <LEVELS>	Number of directory levels beneath the root protected from directory removal	1
--dry-run	Log candidates without deleting them	Disabled
-v	Print worker traversal paths to standard output	Disabled
-vv	Write traversal paths to per-worker traversal logs	Disabled
--show-progress	Display approximate traversal progress	Disabled
-c, --entry-count <COUNT>	Explicit total used by the progress display	Used-inode estimate
-i, --thread-stats	Print per-worker scan and work-stealing statistics	Disabled
-h, --help	Display command help	—
-V, --version	Display version information	—
--read_entire_dir	Read all the contents of a directory into a vector before evaluating entries	Disabled

Run the binary for the authoritative CLI help:

rafael --help

Exception File

The exception file is required so that every run has an explicit pruning configuration. Each line is treated as a substring to search for in a directory path. When a directory path contains a listed value, RAFAEL skips that directory and its entire subtree.

Example:

/data/scratch/protected-project
lost+found
.keep

Important details:

• Use one distinctive path substring per line.
• Matching is substring-based, not glob-based or regular-expression-based.
• A broad value such as user may unintentionally match many paths.
• Use a zero-byte file when no exceptions are needed.
• Do not add blank lines: an empty substring matches every path and can cause all directory trees to be skipped.

Depth Protection

The -d option protects directory removal near the traversal root. It does not prevent RAFAEL from processing eligible files inside those directories.

For a root of /data/users:

-d 1  protects /data/users/<user> from directory removal
-d 2  also protects /data/users/<user>/<next-level>

The root is always preserved regardless of this setting.

Logs and Statistics

Each execution creates a timestamped log directory by appending the current date and time to the configured base path. For example:

rp_log_dir_07-23-2026_11:42:10/

Each worker writes a deletion log similar to:

worker-0-age-30.log
worker-1-age-30.log

Entries include the event time, action, path, source timestamps, and UID:

1721750000: WOULD DELETE /data/scratch/user/file.dat: atime=... ctime=... mtime=... UID=1001

With -vv, workers also create traversal logs:

worker-0-traversal.log

At the end of a run, RAFAEL prints totals for files and directories checked and purged, plus elapsed time. Verbose mode also prints throughput statistics. Start and completion records are sent to the local syslog service using the LOG_USER facility; syslog connection failures are reported but do not stop the run.

Generating Test Data with maketree

maketree creates a synthetic directory tree for testing RAFAEL without using production data.

maketree <ROOT> [OPTIONS]

Create a tree with depth 4, branching factor 4, and four files per generated directory:

maketree testing_data \
  --depth 4 \
  --branching-factor 4 \
  --file-count 4

Create a deterministic mixture of purgeable and non-purgeable entries, with 1 KiB written to each file:

maketree testing_data \
  --depth 4 \
  --branching-factor 4 \
  --file-count 4 \
  --data-size 1024 \
  --purgable \
  --verbose

When --purgable is enabled, selected files and directories have their access and modification times moved to approximately 60 days in the past.

maketree Options

Option	Description	Default
<ROOT>	Root directory to create	Required
-d, --depth <DEPTH>	Number of generated directory levels	2
-b, --branching-factor <COUNT>	Child directories created per directory	2
-c, --file-count <COUNT>	Files created in each generated non-root directory	2
-w, --data-size <BYTES>	Zero-filled bytes written to each file	0
-p, --purgable	Backdate a deterministic subset of entries	Disabled
-f, --force	Allow generation when the root already exists	Disabled
-v, --verbose	Print estimated directory, file, and data totals	Disabled
-i, --interactive	Show estimates and wait for input before creation	Disabled

maketree --help

Testing

Run the Rust test suite from the repository root:

cargo test

The integration tests create data beneath testing_artifacts/ and exercise dry-run, age-based purge, erase, and exception-pruning behavior. They also invoke the Unix find and wc commands, which must be available in PATH.

For additional diagnostics:

RUST_LOG=debug cargo test -- --nocapture

Project Layout

src/bin/rafael_main.rs   RAFAEL command-line entry point
src/bin/maketree.rs      Test-tree generator entry point
src/purger.rs            CLI, worker traversal, logging, and statistics
src/purger_main.rs       Purge orchestration and exception loading
src/metadata_utils.rs    statx metadata collection and age evaluation
src/purge_tree_utils.rs  Deferred directory-deletion candidate tree
src/safra.rs             Parallel termination detection
src/syslog_utility.rs    Local syslog event generation
tests/tests.rs           Integration tests

Operational Checklist

Before a real purge:

1. Confirm the root path is exact and absolute.
2. Validate the exception file and remove blank lines.
3. Run with --dry-run using the same options planned for production.
4. Review every worker log and the final candidate counts.
5. Confirm the depth-protection value preserves the required directories.
6. Ensure logs are written to persistent storage outside the purge target.
7. Remove --dry-run only after the candidate set has been approved.