rf - A minimalist framework for reproducible computation.

RF is a filesystem-based organizational framework for computational analyses. It defines a small contract for arranging directories, files, dependencies, and executable scripts so that a computational analysis is easier to understand, reproduce, and share.

The rf command helps execute and version-control analyses that follow this contract.

The contract

An RF analysis is a tree of computational nodes. A node is an ordinary directory containing two specially named directories:

node/
├── _c/
│   └── run
└── _o/

The contract has three principles.

1. Directories describe dependencies

Child nodes depend on their parent:

nodeA/
├── _c/
│   └── run
├── nodeB/
│   └── _c/
│       └── run
└── nodeC/
    └── _c/
        └── run

Here, nodeB and nodeC depend on nodeA and run only after it completes successfully.

A node can declare additional dependencies using symbolic links in _c/dep/. This expresses relationships that do not fit the parent-child hierarchy without introducing a separate configuration language.

2. _c and _o separate instructions from results

_c contains the material (whether human-authored or AI-generated) needed to understand and reproduce a step:

• executable code;
• parameters and configuration;
• documentation;
• dependency links;
• small inputs or references to external data.

_o contains material generated by running the step:

• results;
• logs;
• execution-state metadata.

This separation makes it clear which files describe the computation and which files can be regenerated.

3. Each computation has an argument-free driver

A computational node provides an executable script named _c/run.

RF invokes this script:

• without command-line arguments;
• with _o as its working directory;
• only after its dependencies have completed successfully.

The script may use any language or computational tool. Its job is to perform one step in the analysis workflow and write its results into _o.

Parameters that affect the result should be recorded in _c, rather than supplied manually when the script is launched.

Why this is useful

The analysis tree becomes a readable record of the computation.

From the location of an output, you can identify the sequence of analysis steps that produced it. From each node's _c directory, you can inspect the code, parameters, documentation, and dependencies required to reproduce that step.

RF deliberately does not introduce its own programming language or configuration format. Shell scripts, Python programs, containers, environment files, and domain-specific tools can all be used inside the same small structural contract.

The contract makes reproducibility easier to inspect and maintain, but it cannot preserve unavailable data or unspecified software environments. Environment definitions and data references should also be recorded in _c whenever they affect the result.

Installation

RF requires Python 3.10 or newer.

Install the current version directly from GitHub:

python -m pip install git+https://github.com/apuapaquola/rf.git

For development, clone the repository and install it in editable mode:

git clone https://github.com/apuapaquola/rf.git
cd rf
python -m pip install -e .

The execution backend currently requires Make and /bin/bash. Some presentation and version-control commands also require tree, Git, and Git Annex.

A minimal example

Create a node:

mkdir -p example/_c

cat > example/_c/run <<'EOF'
#!/usr/bin/env bash
set -euo pipefail

printf 'Hello from RF\n' > result.txt
EOF

chmod +x example/_c/run

Run it:

rf run example

The resulting node looks like:

example/
├── _c/
│   └── run
└── _o/
    ├── SUCCESS
    ├── nohup.out
    └── result.txt

You can also invoke RF as a Python module:

python -m rf run example

Running an analysis tree

Run one node:

rf run NODE

Run a node and its descendants in dependency order:

rf run --recursive NODE

Show the generated Makefile without executing it:

rf run --recursive --dry-run --verbose NODE

Inspect an analysis:

rf status NODE
rf tree NODE

Execution states

The existence and contents of _o record a node's execution state:

Representation	State
No _o directory	Ready to run
_o/RUNNING	Running
_o/SUCCESS	Completed successfully
_o/FAILURE	Failed
_o without a marker	Incomplete
Conflicting markers	Invalid

Markers created by RF contain JSON metadata such as timestamps, a run identifier, the hostname, and the driver's exit code. Existing empty SUCCESS markers remain supported for compatibility.

A failed, running, incomplete, or invalid dependency blocks its descendants. RF does not silently delete or overwrite partial output. Inspect or preserve the output, then explicitly remove _o before retrying.

Remove generated output explicitly with:

rf drop --force NODE

Add --recursive to operate on a subtree.

Version control

The _c/_o separation supports different version-control strategies:

• _c is tracked directly with Git;
• _o can be managed with Git Annex.

RF provides convenience commands for common operations:

rf clone         Clone an analysis and initialize Git Annex
rf commit        Commit code and output
rf commit-code   Commit only _c directories
rf get           Retrieve annexed output
rf drop          Remove output explicitly

These commands complement the filesystem contract. The analysis structure remains understandable without RF-specific metadata files.

Project status

RF is an early-stage implementation of the method.

The current software includes:

• recursive analysis-tree execution;
• parent-child and symbolic-link dependencies;
• explicit running, success, and failure states;
• protection against overwriting failed or partial output;
• Git and Git Annex convenience commands;
• modern Python packaging and command-line installation.

The execution backend currently uses Make and targets POSIX-like systems. More documentation, concrete examples, environment-management guidance, and additional version-control backends remain areas for development.

Background

The original RF preprint is available on bioRxiv.

The use of argument-free driver scripts is related to the approach described in Ioannidis et al., Repeatability of computational experiments.

Contributing

Contributions, examples, and discussions about reproducible analysis organization are welcome.

Run the test suite with:

python -m unittest discover -v

License

RF is distributed under the GNU General Public License version 3.