Project Bourne v0.2.0 — Artifacts and Lineage

Project Bourne v0.2.0 — Artifacts and Lineage

Project Bourne v0.2.0 expands the provenance core from recording executions to recording how scientific results are produced.

This release introduces first-class artifact provenance, experiment lineage, traceability, and safer execution-context capture while remaining application-, language-, framework-, and domain-agnostic.

Highlights

• Declare experiment inputs and outputs explicitly with --input and --output.
• Record a distinct version for every artifact capture, including:
  • stable artifact ULID
  • original and resolved paths
  • artifact role
  • capture time
  • file size
  • modification time
  • SHA-256 content identity
  • capture diagnostics
• Stream SHA-256 hashing in bounded chunks instead of loading entire files into memory.
• Distinguish artifact state truthfully using:
  • existence_state: present, missing, unknown
  • capture_status: complete, unreadable, unsupported, changed
• Record experiment lineage with --derived-from.
• Resolve lineage references using full ULIDs, unique prefixes, latest, and @N.
• Trace an output back to the experiment that produced it with bourne trace.
• Refuse to guess when an artifact reference is ambiguous.
• Capture safe execution-context information, including:
  • requested executable
  • resolved executable path
  • Bourne recorder executable
  • allow-listed Conda and virtualenv hints
  • conservative container-presence information
• Transactionally migrate existing v0.1.1 databases from schema version 1 to schema version 2 without resetting historical experiments.
• Preserve zero third-party runtime dependencies.

Example

bourne run \
  --input config.json \
  --output result.csv \
  -- python simulation.py config.json result.csv

bourne show @1

bourne run \
  --derived-from @1 \
  --input config_next.json \
  --input result.csv \
  --output result_next.csv \
  -- python simulation.py config_next.json result_next.csv

bourne trace result_next.csv

Conceptually:

config.json
    │
    ▼
Experiment A
    │
    ▼
result.csv
    │
    ▼
Experiment B
    │
    ▼
result_next.csv

Bourne can now preserve and inspect this provenance chain.

Artifact correctness

Bourne does not turn uncertainty into false provenance.

A missing artifact means Bourne established that the file was absent:

existence_state = missing
capture_status  = complete

If Bourne cannot determine whether an artifact exists, for example because it cannot inspect the path, the state remains explicitly unknown:

existence_state = unknown
capture_status  = unreadable

Other incomplete capture conditions are represented explicitly as well:

present + unsupported
present + unreadable
present + changed

Only artifacts captured as present + complete may use SHA-256 as valid content identity.

Compatibility

All existing v0.1.1 functionality remains available:

• bourne run
• bourne list
• bourne show
• bourne compare
• full and shortened experiment IDs
• latest
• @N
• Bash, Zsh, and Fish completion
• failed experiment persistence
• interrupted experiment persistence
• live stdout/stderr streaming

Existing v0.1.1 databases are migrated automatically and transactionally when opened by v0.2.0.

Current limitations

This release intentionally keeps the scope focused.

Not yet included:

• automatic artifact discovery
• artifact archival or uploading
• directory or object-store artifacts
• multiple lineage parents
• environment discovery or automatic environment selection
• dependency installation
• resource profiling and telemetry
• scientific verification
• remote execution
• Slurm or PBS orchestration
• MCP integration
• natural-language agents

stdout and stderr still stream live but accumulate in memory before final persistence.

Design principle

Bourne remains generic by default.

The core does not depend on Python, machine learning, fusion software, Docker, MPI, or any particular scientific application.

An unknown executable remains a first-class workload.

Future application-specific integrations can add additional knowledge without changing the underlying provenance model.

Every experiment has a history.