LayerFS v0.1.5 — ordinary storage, capacity, and honest limits

LayerFS v0.1.5 — ordinary storage, capacity, and honest limits

v0.1.5 ships ordinary schema-10 Init/Commit storage as the only storage path
and removes explicit compaction. New Stores use schema 10 with 4 KiB pages: exact
content-addressed admission, compression, bounded whole-file small content below
128 KiB with bounded delta chains, large-file CDC/extents, compact scoped
namespaces and authenticated pooled metadata. Canonical identity is unchanged.

What got better, measured:

• Pending-edit capacity (#116): an equal-length overwrite of committed
  content is retained as one base root plus one bounded splice descriptor — one
  64-byte descriptor instead of three 128-byte nodes. Pending workspace capacity
  moves from 5,461 to ≈29,959 spliced files at the same budget, and the
  public default-budget route now accepts 32,000 edits (32,000 × 64 B under
  the unchanged 2 MiB budget) with independent verification of all 32,000
  changed files before and after a fresh reopen.
• Distributed SDK-edit workloads: workspace-distributed-sdk-edit-500
  4.131 s → 0.666 s (0.16×) and -100 0.45× against the earlier v0.1.5
  baseline — the same #116 mechanism.
• Packed metadata (#107): pack rows 3,457 → 1,058, pack overflow slack
  −43 %, apparent Store −1.13 %, and allocated Store −1.9 % on
  store-footprint-unique-100000 (530,358,272 → 520,142,848 B). The
  self-declared ≥1.5 % allocated figure was missed numerically (+0.02 %
  allocated) and is accepted by explicit owner decision, not re-based.
• Stable wins: payload-create-* 0.79–1.02× and dedup-workspace exact/local-100 0.62–0.66× versus published v0.1.3.

What did not get better — published, not relabeled:

• One registered Tier-1 gate fails: dedup-history-unrelated-500-mixed-v2
  measured 16.107 s against unrelated-history500 < 15 s, dispositioned by
  an explicit owner waiver. Root cause: the owner-required #116 bounded pending
  representation (+1.98 s) plus the owner-accepted #107 pack coalescing
  (+0.33 s); the breach predates this campaign, and the cell is 0.89× v0.1.3.
  Residual risk: the worst-case unrelated-history tier runs ~7 % over its
  historical gate.
• 125 WARN cells; worst ratios dedup-cdc-scattered-100 3.45×,
  dedup-cross-file-unique-100 3.35×, namespace-100 3.22×.
• 139 of 198 comparable cells are ≥15 % slower than published v0.1.3,
  median 1.34×, +33.115 s total. This is context for future target
  selection, never an acceptance gate.
• The #116/#107 mechanisms add fixed per-iteration costs of +2.49 ms/exec and
  +1.46 ms/commit (#116) and ≈+0.4 ms/phase (#107).
• store-footprint-unique-100000 construction takes 5.397 s (+36 % versus
  fsync-qualified): the #107 pack-row UPDATE cost on one giant commit.
• Ordinary full157 allocated 83,951,616 B versus a Git control live
  56,197,120 B (accepted residual).
• dedup-cdc-scattered-100 leaves an ≈452 ms unattributed gap between timer
  and command window, and the slab_send_blocked_ns mechanism remains
  unattributed.
• Endurance is not qualified: the 600 s sustained proof was
  NOT_RUN_OPTIONAL under the frozen campaign declaration.

Validation: 227 registered selections terminal — 198/198 performance (182
fresh: 56 PASS / 125 WARN / 1 FAIL; 16 reused) and 29/29 proof-only (28 PASS +
1 not run) — with 182/182 fresh independent proofs PASS, every cleanup PASS, no
S0/S1, a VERIFIED_COLD namespace-100000 member, and the native gate PASS (530
tests, 118 s) on the measured tree. The campaign is inherited from issue #120;
no benchmark was re-run for this release.

Upgrade boundary: create a new schema-10 Store. Supported
schema-6/7/8/9 Stores connect without promotion; published v0.1.3/schema-5 Stores
are rejected. LayerStackStore::upgrade_format promotes a closed schema-7/8/9
Store to schema 9 only — there is no in-place promotion to schema 10 and no
downgrade. Compacted Stores from earlier builds remain readable (read-only
compatibility path). Use matching SDK/owner/daemon builds.

This is a source-only Developer Preview, not production storage. Crash and
power-loss durability are not promised. No crates.io package, prebuilt
executable or public runtime image is part of this release.

Start with the release record,
the versioned manual,
the acceptance and waivers and
every benchmark family and case.

Existing benchmark results by family

Times below are descriptive sums of the individual case timers, not campaign
wall time and not statistical speedup estimates. Positive changes mean slower
than the published v0.1.3 checkpoint, whose cache profile is undeclared. Rows
mixing fresh and reused values are marked in the release record; this table
reports all registered performance cases in the family.

Family	Performance cases	Reused	v0.1.3 seconds	v0.1.5 seconds	Change
payload_create_read	8	1	4.179260	3.454420	-17.34%
dedup_workspace_reuse	14	1	15.875100	15.301500	-3.61%
dedup_cross_file	10	2	1.402390	3.473250	+147.67%
dedup_cdc_locality	20	1	1.484610	3.510380	+136.45%
edit_length_preserving	12	1	0.082800	0.114680	+38.50%
edit_length_changing	32	2	0.230460	0.303170	+31.55%
edit_canonical_chunk_count	12	0	0.092550	0.129570	+40.00%
init_namespace	4	0	3.053450	5.559430	+82.07%
store_footprint	6	0	8.155620	13.464210	+65.09%
tiny_file_churn	20	1	8.529740	9.714270	+13.89%
namespace_mutation	4	0	0.314360	0.452180	+43.84%
directory_construction_traversal	12	1	7.629500	8.826820	+15.69%
workspace_change_locality	16	4	10.971850	15.227050	+38.78%
dedup_branch_history	20	0	47.049480	45.614220	-3.05%
git_tool_workflow	4	1	7.498040	12.683490	+69.16%
mixed_load_bearing	4	1	8.559560	9.345510	+9.18%
registered total	198	16	125.108770	147.174150	+17.64%

The five families that are faster than v0.1.3 in aggregate are exactly the ones
the #116/#107 mechanisms and the pre-existing ordinary-path wins touch; the
dedup families' large aggregate ratios compare the v0.1.5 ordinary path
(authenticated CAS + CDC + pack assembly) against pre-authentication v0.1.3
references and are published as #112 context, never as gate results. Read the
complete per-case tables.
Existing qualified results were reused; no benchmark was run for this release.

LayerFS v0.1.4 — storage efficiency Developer Preview

LayerFS v0.1.4 — storage efficiency Developer Preview

v0.1.4 adds packed SQLite storage, native FULL/PREFIX representations, bounded
shared Init/Commit publication, and reuse of authenticated Init comparison bytes.

The accepted full157 observation uses 184,582,144 allocated bytes, 15.374%
less than its 218,116,096-byte supplemental control. Adjacent duplicate-heavy
Init observations improve by 80–82% against the prior R26 candidate while
preserving exact collision checks. These are scoped observations, not universal
performance guarantees or statistically established distributions.

Validation covers 198 performance cases, 226 routine proofs, supplemental
small-file/SDK/FUSE workloads, 157 performance states and 157 retained-history
proofs. The optional 600-second endurance proof was not run.

Known tradeoffs: substantial regressions versus v0.1.3 remain. Three absolute
latency targets miss; the four historical Git comparisons are INELIGIBLE and
the current overall report remains INCOMPLETE. Owner acceptance does not turn
those outcomes into passes. Full157 historical verification also took longer
than the prior candidate. Read the complete tables and measurement boundaries.

Upgrade boundary: create a new schema-7 Store. Supported development
schema-6 Stores retain the legacy format. Released v0.1.3/schema-5 Stores are
not migrated or opened; preserve them with their original binaries. Directory
import starts new history. Use matching SDK/owner/daemon builds.

This is a source-only Developer Preview, not production storage. Crash and
power-loss durability are not promised. The source assets and checksums are attached to the GitHub release.

See the versioned manual, accepted-tradeoff record and full terminal evidence
in the accompanying source tree.

The included issue #98 Workspace repair is fully qualified: adjacent Commit time improved 37.8%, final allocated storage is unchanged, and all benchmark/proof/full157 checks passed. Final .venv observations and higher unpaired full157 wall times are reported explicitly. Remaining historical performance misses and four ineligible Git comparisons stay visible.

Every benchmark family and case. Existing qualified results were reused; no benchmark was rerun for publication.

Existing benchmark results by family

Times below are descriptive sums of individual case timers, not campaign wall time or statistical speedup estimates. Positive changes mean slower than the eligible published v0.1.3 checkpoint.

Family	Performance cases	Passing proofs	v0.1.3 seconds	v0.1.4 seconds	Change
payload_create_read	8	8	4.179502	3.872964	-7.33%
dedup_workspace_reuse	14	14	15.874997	14.074111	-11.34%
dedup_cross_file	10	10	1.402381	3.154912	+124.97%
dedup_cdc_locality	20	21	1.484589	3.187850	+114.73%
edit_length_preserving	12	12	0.082806	0.119696	+44.55%
edit_length_changing	32	32	0.230456	0.306447	+32.97%
edit_canonical_chunk_count	12	12	0.092536	0.135438	+46.36%
init_namespace	4	4	3.053609	4.903439	+60.58%
store_footprint	6	6	8.156506	12.273350	+50.47%
tiny_file_churn	20	20	8.529795	8.996516	+5.47%
namespace_mutation	4	4	0.314367	0.429782	+36.71%
directory_construction_traversal	12	12	7.628812	9.710653	+27.29%
workspace_change_locality	16	16	10.971227	13.240867	+20.69%
dedup_branch_history	20	20	47.049815	55.856741	+18.72%
git_tool_workflow	4	4	—	8.877153	INELIGIBLE
mixed_load_bearing	4	4	8.559964	8.592193	+0.38%
workspace_reliability	0	27	—	—	—

Elapsed comparison labels: 49 SEVERE, 95 REVIEW, 31 OBSERVED_INCREASE, 19 NO_INCREASE, 4 INELIGIBLE. Execution success is separate from performance acceptance. The overall comparison report remains INCOMPLETE.

Full157 retained-state storage: 184,582,144 B versus 218,116,096 B, a 15.374359% reduction. The supplemental storage control is not the published v0.1.3 performance baseline. Full157 performance/verification wall observations were 481.975846875 s / 613.815488042 s, higher than the previous 447.246954625 s / 545.826543958 s; these are unpaired historical observations.

Source and evidence identities

Qualified product source: 9cfb4be477116646258ea0621280ed13b1824c6d. Documentation and packaging updates preserve that production code.

Host benchmark SHA-256: 47b4d44e3e961f3b6a57d2c18dc6d195973133dbdfb87681a55f1b8e07abe3a2.

Sealed benchmark image: sha256:42eb806fcfe9db1dc28098eb423340de3e4eaf81fd487389128305e65606d347. These are benchmark identities, not newly published binary assets.

Immutable qualified evidence · Versioned manual · All 198 case results.

Released commit: 101fa273d815f3aaedb0e06ba0de7b0777d83def. Annotated tag object: c1c3830c29a3dbece4a490c06e922f1f81295091. Final PR CI: 34338286986. Final merged tree equals qualified PR #99's tree; production code matches the qualified source.

Six source-only assets were verified against the tagged Git tree (4,161 source members; 2,888 selected evidence members). SHA256SUMS covers the other five assets. No prebuilt executables, public runtime image or crates.io publication is included.

LayerFS 0.1.3 Developer Preview

LayerFS 0.1.3 Developer Preview

LayerFS 0.1.3 is a benchmark-driven redesign of Commit processing, temporary backing, live Workspace execution, and authenticated reads.

Highlights

• Shared live Workspace core for FUSE operations and SDK edits, with commands and handles continuing across live-FUSE Commit boundaries.
• In-place checkpoint installation, ordered namespace finalization, shared backing segments, and owned object admission remove repeated reconstruction, file cleanup, copying, and spill readback.
• Payload, CAS/CDC, and Workspace-reuse scaling improvements with separately recorded qualification campaigns.
• Approximately 3× faster measured high-tier Git workflows: 5.83→1.85 seconds and 14.12→4.64 seconds in the corrected comparison (one baseline observation, three-run final medians); backing requests fell about 92%.
• Repairs for stale mapped-page writeback over SDK edits, failed-owner Discard, hard-link preservation, and post-publication presentation recovery. Release qualification additionally fixed premature SDK-protection retirement while admitted writebacks were still queued.
• 198/198 performance cases and 226/226 routine proofs pass across 17 families. Final mixed-file lifecycle observations include create 1,000 files/100 MiB in 0.990 seconds and delete in 0.259 seconds. These are single observations, not latency guarantees.
• Simplified benchmark infrastructure and bounded history verification with explicit coverage.

Read the numbered changelog, technical explanation, and every-family/every-case benchmark tables.

Upgrade and limits

Schema 4 is migrated to schema 5 on connect; old binaries cannot reopen the upgraded Store. Keep a separate pre-upgrade backup and use matching 0.1.3 SDK/CLI, owner, daemon, and FUSE builds. Canonical object identities remain unchanged. Materialized Workspaces retain the active-execution Busy restriction.

This is a source-only Developer Preview, following the existing release model. There are no published crates.io packages, executables, or runtime images. Crash/power-loss durability remains unsupported. Git latency targets remain missed; history verification includes sampled snapshots, and the separate 600-second endurance test is excluded from the final checkpoint. Historical exploratory results and changed workload recipes remain explicitly identified.

Verification and assets

See the verification record, manual, and release contract.

Checksummed source archives, the published benchmark-evidence bundle, Cargo.lock, and LICENSE accompany this release. SHA256SUMS identifies each asset.

Published source and CI

• Annotated tag: v0.1.3 at 49f0f5be911e7f5b92afc2becab45144ad7a1307.
• Merged release PR #85.
• Successful final PR CI.
• Successful merged-source CI.
• Repaired-source local checks: 348 native test results passed, one pre-existing ignored; all 26 FUSE library tests passed; three predeclared live Docker/FUSE runs passed all three tests each (9/9 executions). Original failures and scope are retained in the verification record.

The benchmark measurements remain pinned to their original source. The release includes the separately verified admitted-writeback correction; no timing is relabeled as a new run of this release source.

LayerFS 0.1.2 Developer Preview

LayerFS 0.1.2 Developer Preview

LayerFS 0.1.2 adds universal owner-side regular-file range editing and reports
complete SDK-only edit benchmarks across 1, 10, 100 and 500 MiB files.

Highlights

• One SDK range-edit operation covers overwrite, append, prepend, insertion,
  deletion, growth, shrinkage, truncation and zero extension.
• Localized edits reuse unchanged content through the shared piece-tree engine;
  FUSE presentation refresh invalidates the edited inode without remounting.
• A daemon close/remount race is fixed: watcher and mount reservations retire
  before close acknowledgement, without sleeps or retries.
• Three complete edit families: 56 cases, 560 baseline/candidate performance
  samples and 112 independent source-arm verification proofs.
• Fresh namespace initialization and Store-footprint measurements complete the
  five-family release inventory. Performance and full verification are separate.

SDK prepend results

One 4 KiB prepend, five candidate samples per size. Times are medians in ms.
These source-bound measurements are from the completed SDK campaign, not the
obsolete full-file temp-copy workload.

File	Edit	Commit	Combined
1 MiB	1.674	2.919	4.680
10 MiB	2.164	2.779	4.883
100 MiB	2.967	4.289	7.257
500 MiB	3.344	11.122	14.300

See the benchmark report
for all operations, N, median/min–max tables, source identities, accepted
20/20/30 ms ceilings, three narrowly approved Edit-parity exceptions, and
memory-observation limitations. Combined medians are calculated independently.
The claim is localized, size-stable edits with bounded Commit latency—not
size-independent Commit or universal hardware/OS performance.

Compatibility and limitations

This is a source-only Developer Preview. It retains the current SQLite Store
and canonical identity format; use matching SDK and daemon versions. No
executables, crates.io packages, or runtime images are published. Keep independent
backups: live-process acknowledgement is not a crash/power-loss durability promise.

The compatible Store remains above the original primary footprint goal.
Alternative storage designs, including authenticated physical packs, are
far-future, unscheduled exploration in #18—not a near-term optimization promise.

Memory tables distinguish sampled observations from native lifetime peaks;
they do not prove continuous category ceilings or exact edit-phase attribution.
Historical failed attempts remain preserved. The original SDK evidence and
the later daemon-fix/release-refresh source are identified separately.

Assets

Source archives, Cargo.lock, LICENSE, and SHA256SUMS accompany this release.
Read the manual
and verification record.

LayerFS 0.1.1 Developer Preview

LayerFS 0.1.1 Developer Preview

LayerFS 0.1.1 preserves the 0.1.0 public and storage contracts while improving
large existing-directory initialization, localized small-edit Commit,
Workspace Create, and bounded reads.

Highlights:

• bounded initialization with eight existing producers, four fixed slabs, and
  the calling thread as the sole SQLite admission owner;
• no canonical object-segment spool or parent payload copy on the admitted
  direct path;
• exact operation-local metadata reuse without a persistent cache;
• localized Commit planning for ordinary content-only edits; and
• demand-loaded authenticated Workspace bootstrap objects and bounded
  read-ahead.

Correctness, resource, cleanup, FUSE/materialization equality, managed Docker,
native quality, namespace performance, and registered payload gates pass in
the terminal retained evidence. The workspace version is 0.1.1.

Start here

LayerFS 0.1.1 is distributed as source. Build the CLI with Rust 1.85 or newer:

git clone --branch v0.1.1 --depth 1 https://github.com/Ephemeral-AI-Lab/layerfs.git
cd layerfs
cargo build --release -p layerfs-cli
./target/release/layerfs --version

Continue with the
0.1.1 quickstart
and review the
release limitations.

Prebuilt executables, crates.io packages, and runtime images are not published
for 0.1.1. The release contains deterministic source archives, Cargo.lock,
the MIT license, and SHA256SUMS.

LayerFS 0.1.0 Developer Preview

LayerFS 0.1.0 Developer Preview

LayerFS is a SQLite-backed, content-addressed filesystem engine for ephemeral
agent Workspaces and durable shared history. CAS, content-defined chunking, and
copy-on-write reuse unchanged bytes and filesystem structure, so parallel
Branches and Workspaces do not multiply the shared base.

Highlights

• One SQLite Store for LayerStacks, immutable Layers, named Branches,
  immutable Commits, and canonical objects.
• Zero-copy Branch Fork from an immutable Layer or eligible Commit.
• Ephemeral materialized or real-FUSE Workspaces with explicit Commit and End.
• Fresh-process command execution with bounded, paged output.
• Public Rust SDK and CLI for storage, Workspace, history, monitoring, and
  managed-container lifecycle operations.
• Store-wide deduplication with authenticated canonical reads, CDC for
  localized file edits, and structural COW for changed paths.

Measured results

In the final seven-pair public-SDK, real-FUSE campaign against Cloudflare
Computer, LayerFS measured:

Lifecycle	LayerFS	Computer	LayerFS speedup
Cold create 32 MiB	161.231 ms	1,660.321 ms	10.07×
EDIT16	169.133 ms	2,631.062 ms	15.80×
Prepend 10 bytes	232.394 ms	2,484.210 ms	10.48×
Read 32 MiB	119.154 ms	780.946 ms	6.53×
Registered total	690.196 ms	7,579.414 ms	10.76×

Incremental semantic storage was 97.19% lower for EDIT16 and 99.92% lower for
a ten-byte prepend to a 32 MiB file. See the
full benchmark report
for methodology, environment, source seals, acknowledgement policy, and raw
evidence references.

Start here

LayerFS 0.1.0 is distributed as source. Build the CLI with Rust 1.85 or newer:

git clone --branch v0.1.0 --depth 1 https://github.com/Ephemeral-AI-Lab/layerfs.git
cd layerfs
cargo build --release -p layerfs-cli
./target/release/layerfs --version

Continue with the
0.1.0 quickstart
and review the
current limitations.

Developer Preview boundary

This is a pre-release for evaluation, integration, benchmark reproduction, and
design research. It is not production storage, does not claim crash- or
power-loss durability at every acknowledgement point, and is not a hardened
hostile-code sandbox. Keep an independent copy of important data.

Prebuilt executables, crates.io packages, and runtime images are not published
for 0.1.0. The release contains deterministic source archives, Cargo.lock,
the MIT license, and SHA256SUMS.