fastdup v0.8.1

fastdup v0.8.1

RPM fastdup-0.8.1-1.el10.x86_64.rpm for Rocky Linux 10 x86-64.

A single fix to the adaptive cache budget. No format change: v0.8.1 reads and
writes the same epoch-3 repository as v0.8.0, and no migration or re-ingest is
required.

A warm cache no longer shrinks between jobs

The shared memory budget scored each cache pool from its recent hits, misses and
evictions, decayed by an EWMA. That is the right rule while pools compete for
the same RAM. It is the wrong rule when nothing is competing: between two backup
jobs the counters go quiet, the score decays, and the pool's target falls back
toward whatever is currently resident. The next job then starts against a cold
cache that the appliance had already earned and had room to keep.

A hit-only phase had the same effect. Status sampling continues while a job
reads without missing, so hits accumulated while misses and evictions did not.
The target still shrank, class ceilings dropped below residency, and the cache
evicted useful entries — manufacturing the misses that the score had not seen.

A disk-backed pool now retains its previously granted target out of headroom
that no other pool wants, after measured demand has been served. The retention
is deliberately narrow:

• It applies only to disk-backed pools. Scratch buffers still take leftovers.
• It never grows an idle target; it only declines to shrink one.
• If the retained amounts do not all fit in the free headroom, the targets are
  genuinely competing, and the ordinary score-based allocation and damping run
  unchanged — with no bias toward whichever workload held the lease first.
• Under hard memory pressure the retention is skipped entirely and the payload
  is reclaimed.

Covered by warm_disk_cache_keeps_its_target_while_headroom_is_uncontested,
which asserts both directions: the target holds across 200 rebalance rounds in
an active and in a hit-only phase, and a real pressure snapshot still collapses
it to the fixed minimum. The existing hand-over test was extended to fill the
shared budget before switching workloads, so it keeps proving that an idle pool
loses to an active one once the headroom is actually contested.

Also in this release

docs/research/veeam-hardened-repository-concept.md records the concept for
running the original Veeam repository services in a local Linux system container
against a fastdup-backed XFS-compatible path, instead of the current SMB route.
It is a request for comments, not a committed roadmap: it names the existing
building blocks, and it is explicit that the critical path — translating
FICLONE/FICLONERANGE, which the FUSE mount answers with ENOTTY today — is
unsolved, that an authorization model for immutability and a synchronous commit
mode would each need a new ADR against accepted ones, and that an emulated
filesystem is not a Veeam-supported configuration.

Install

curl -fLO https://github.com/ThatIsCraZy/fastdup/releases/download/v0.8.1/fastdup-0.8.1-1.el10.x86_64.rpm
curl -fLO https://github.com/ThatIsCraZy/fastdup/releases/download/v0.8.1/SHA256SUMS
sha256sum --check --ignore-missing SHA256SUMS
sudo dnf install ./fastdup-0.8.1-1.el10.x86_64.rpm

Install during a maintenance interval; replacing files alone does not restart
running Runtime or Samba processes. Upgrading from v0.8.0 needs no repository
work. Upgrading from 0.7.x still requires a re-provision and a re-ingest,
because v0.8.0 moved the repository format to epoch 3.

fastdup remains a research prototype; retain independent backups.

fastdup v0.8.0

fastdup v0.8.0

RPM fastdup-0.8.0-1.el10.x86_64.rpm for Rocky Linux 10 x86-64.

Important

Repository format epoch 3. This release does not migrate existing pools.
A repository written by 0.7.x is refused at mount, not converted. Plan a
re-provision and a re-ingest before you install 0.8.0 on an appliance that
holds data you still need.

v0.8.0 is the release in which Veeam backup over SMB stopped being a pending
qualification and became a measured result, and in which the write path stopped
losing throughput the longer it ran.

Veeam

Veeam Backup & Replication writing a VMware backup job to a fastdup SMB share
now completes. Five consecutive sessions on the test appliance reported
Success; in the session recorded in detail, all eight tasks succeeded with
Veeam reporting the target as the bottleneck, 283.4 GiB transferred in a 465.8 s
task window, and 74,710 SMB writes, 768 flushes and 803 closes with zero
failures.

Fast Clone over SMB is confirmed with the 4 KiB clone geometry introduced in
0.7.1: aligned clones, partial clone lengths and consecutive clones against the
same source.

This is a working configuration measured on one appliance. It is not a
Veeam-certified or Veeam-qualified integration, and it carries no support
statement or performance SLA.

Work is under way to integrate the Veeam Repository Agent into the appliance,
so that fastdup can be added as a Linux hardened-style repository rather than
only as an SMB share. That work is not part of this release.

Veeam backup qualification ·
clone alignment ·
partial clone length ·
consecutive clones.

Ingest no longer degrades with uptime

Three defects compounded into a write path that got monotonically slower under
sustained load. Five identical 2 GiB copies used to run 78.4 → 72.3 → 68.4 →
64.9 → 60.9 MB/s.

• The commit cut waited on the mutation-admission fence. A writer parked on
  ingest capacity held the admission read guard while begin_commit wanted the
  write guard, and only the commit cut could return that capacity. Every
  generation was released by the five-second supervisor timeout. begin_commit
  now announces the cut before requesting the fence, and a writer that would
  block seals its batch and returns instead. Both the single-stream ring wait
  and the multi-stream ingest budget honour the announcement (ADR 0096).
  Measured: freeze 5,001 ms → 0.93 ms across 110 generations.
• The cut then waited on the pending-region gate. The stall moved to
  ingest_wait, 27 five-second stalls per three minutes. A counted commit-cut
  hatch, held for exactly the cut-wait window and tracked separately from the
  one-generation watchdog hatch, admits that backlog (ADR 0097). Measured: 0
  five-second stalls.
• The pending-region ledger leaked. Lane eviction and the commit-cut drain
  both detached Lane payload without settling its reservation, so the ledger
  reached 3,256 MiB against a 304 MiB gate holding 1.9 MiB of actual residue.
  Past a full gate every staging reservation blocks unconditionally. Both paths
  now settle (ADR 0098), and WriteThroughStatus reports pending_region_bytes
  and pending_residue_bytes next to the gate size.

Measured on an aged pool: steady-state dedup ingest 61 MB/s and falling →
224–256 MB/s and stable. On a fresh epoch-3 pool: 380–422 MB/s, checkpoint
206 ms for 807 MB logical.

Namespace as a content-addressed tree (ADR 0095)

The Namespace is no longer one canonical payload cut by FastCDC. It is
published as independently decodable record-range shards — inode shards, entry
shards, and a graph root carrying counters and one reference per shard.

Shard boundaries derive from the record keys alone, so the partition is a pure
function of the record set and two generations sharing a key range publish the
identical object. Two positional rules bound what the key predicate cannot:
8,192 records and 4 MiB of payload per shard, because one inode may carry a
mebibyte of xattrs.

Measured at 120,000 entries: encode_graph 29.3 ms → 13.7 ms, 33 → 200 shards.
Changing one inode retires exactly one shard and publishes exactly one new
shard, with no repartition. Each Namespace is proven once during construction
rather than walked a second time to encode (ADR 0094).

This is what moves the repository format to epoch 3.

Mount and maintenance

• Bounded mount audits. OnlineSimilarityRepository::open audited each run
  one 4 KiB Direct-I/O page at a time, single-threaded, in front of every mount:
  22 s at ~18 MB/s on a device that streams at 110 MB/s. All three audit passes,
  the Bucket semantic walk and the Exact Index page warmer now use bounded
  256 KiB range reads. Measured: 22 s → 3 s.
• Stranded jobs no longer disable the appliance. A job left in running
  with no task executing it made the WebUI and the server reject every
  repository command, indefinitely. ControlStore::open now fails every
  still-open job — jobs run in-process and cannot survive a restart — and the
  sampler closes orphans within one tick by comparing open jobs against the
  jobs actually executing. Both checks are exact rather than timeout-based, so a
  long offline scrub is never reaped.
• Ingest-capacity waits are accounted on every release path, including the
  announced commit cut that almost every stall actually takes.
  ingest_ring_wait_ns previously reported 30 ms while writers were parked for
  46 s.
• The Share capacity manifest names itself when its Share inodes do not
  exist, instead of failing the mount with a bare NoEntry. A re-provisioned
  pool keeps /etc/fastdup/share-capacities.json but not the inodes it names.
  It still fails closed: silently dropping a configured quota would be worse.

Verified

	Before	After
freeze per generation	5,001 ms	0.93 ms
ingest_wait five-second stalls per 3 min	27	0
Steady-state dedup ingest, aged pool	61 MB/s, falling	224–256 MB/s, stable
Pending-region ledger	2,260 MiB	1.9 MiB of 304 MiB
Similarity audit at mount	22 s	3 s
encode_graph at 120k entries	29.3 ms	13.7 ms

Workspace suite: 1,111 tests passed, 0 failed. Clippy clean on every crate
changed in this release.

Install

curl -fLO https://github.com/ThatIsCraZy/fastdup/releases/download/v0.8.0/fastdup-0.8.0-1.el10.x86_64.rpm
curl -fLO https://github.com/ThatIsCraZy/fastdup/releases/download/v0.8.0/SHA256SUMS
sha256sum --check --ignore-missing SHA256SUMS
sudo dnf install ./fastdup-0.8.0-1.el10.x86_64.rpm

Install during a maintenance interval; replacing files alone does not restart
running Runtime or Samba processes. Reconnect SMB clients after activation and
arrange service restarts around backup jobs.

Because of the epoch change, an existing repository must be re-provisioned and
re-ingested. fastdup remains a research prototype; retain independent backups.

Open work

• The commit-side mirror for ADR 0095. Encoding still visits every shard.
  Skipping untouched shards needs the dirty record set, which
  namespace_root_for_commit currently discards by rebuilding every
  DurableInode and NamespaceEntry from scratch. Reachability, directory link
  counts and (parent, name) uniqueness have to be proven from the changed set
  plus retained summaries before that is safe.
• Advanced Reduction currently reads more than it saves on the observed
  workload: 40,168 queries, 50 candidates, 13.1 MB read from the DATA tier to
  save 9.3 MB. It remains opt-in and defaults to disabled on new shares.
• Write-latency asymmetry under Veeam load. Intervals with write-through
  queue backpressure ran at 811 MB/s and 28 ms average write latency; intervals
  without it ran at 299 MB/s and 329 ms, carrying the long tail. Not explained
  yet.

fastdup v0.7.4

fastdup v0.7.4

RPM fastdup-0.7.4-1.el10.x86_64.rpm for Rocky Linux 10 x86-64.

fastdup v0.7.4 bounds missing-index reads, reports repository health independently
of telemetry availability, returns retained allocator pages to the OS, and shows
disk IOPS in the WebUI.

• Resolve missing Exact-index hints through each Container's compact Recovery
  Index, then read and verify only required Records and independent Bases.
  Indexed verification continues past individual misses, and shared compressed
  Records are decoded once per proof pass. Complete scrub still validates full
  Containers.
• Derive repository availability from exact FUSE mount-table presence and the
  repository service state. Missing metrics and normal closed write admission
  no longer create false outage alerts; confirmed process and integrity failures
  remain latched until successful Runtime evidence arrives.
• Run a single bounded background allocator worker that asks glibc to return
  free arena pages only when allocator slack still occupies anonymous RSS. It
  does not evict live objects or cache entries. Allocator usage, trim count and
  last trim time are exposed separately from cache occupancy.
• Display measured read/write IOPS next to MB/s in disk activity and the Storage
  overview. Missing historical samples remain unavailable rather than becoming
  zero.

Validation covers 31 Store tests for bounded fallback, verification, recovery
and concurrent misses; 42 Control tests; 15 Runtime tests; two allocator-policy
tests; and 46 UI tests. Clippy passed with warnings denied for the changed Rust
libraries and Runtime binary. The TypeScript/Vite production build and desktop
and mobile Chromium checks passed without browser errors or horizontal overflow.
The locked Rust release build, UI production build and RPM build also passed.

The bounded-read regression requested 50,560 bytes and made no whole-Container
payload read for one 8-KiB target. A synthetic allocator A/B reduced retained RSS
by about 96%, with a measurable trim cost. These are focused qualification
results, not end-to-end Veeam performance results. This version had not yet been
installed or measured on the test appliance when the release was prepared.

No durable format migration is required. Install the RPM for Rocky Linux 10
x86-64 during a maintenance interval; replacing files alone does not restart
running Runtime or Samba processes. Verify the download with SHA256SUMS.
FastDup remains a research prototype; retain independent backups.

Bounded fallback-read qualification ·
Runtime-health and allocator qualification ·
Allocator A/B.

sha256sum --check --ignore-missing SHA256SUMS
sudo dnf install ./fastdup-0.7.4-1.el10.x86_64.rpm

fastdup v0.7.3

fastdup v0.7.3

RPM fastdup-0.7.3-1.el10.x86_64.rpm for Rocky Linux 10 x86-64.

fastdup v0.7.3 ships adaptive compressed Verified Read caching and fixes the Metadata-GC republication crash, with accurate Runtime health in the WebUI.

• Independently compress cold verified payloads with LZ4 when it saves RAM. Hot decoded entries share the same adaptive budget; no backend Base or Dictionary read is required to reuse a compressed entry.
• Show compressed/decoded occupancy, represented payload bytes, saved RAM, codec costs and workspace usage in Telemetry.
• Retire stale Metadata-GC journal additions before exact collection. Reusing previously collected content after a concurrent pin release or interrupted collection no longer triggers the duplicate-addition assertion.
• Report Runtime loss and write pauses in the persistent topbar, reset stale throughput and audit health transitions. A live management agent no longer implies a healthy repository.
• Reclaim disconnected FUSE mountpoints after acquiring the appliance lease, then recover normally.
• Refresh English/German README, product page, screenshots and download links.

Validation: 58 GC/maintenance tests, 39 Control tests, 15 Runtime tests and 44 UI tests passed for the included crash/status fix. Cache qualification additionally covers compression integrity, memory accounting and a bounded workload A/B benchmark. The same runtime changes were verified on the test appliance in 0.7.2-2: automatic stale-mount cleanup, successful recovery, updated HTTPS UI and authenticated 8 MiB SMB round trip with matching SHA-256. A complete Veeam retry remains pending.

No durable format migration is required. Install the RPM for Rocky Linux 10 x86-64 during a maintenance interval; replacing files alone does not restart running Runtime/Samba processes. Verify the download with SHA256SUMS. FastDup remains a research prototype; retain independent backups.

Release validation: the locked Rust release build, TypeScript/Vite UI build and RPM build passed. The product page was checked in German and English at 1440 px and 390 px, including screenshots and release/download links, without browser errors or horizontal overflow.

Cache qualification ·
Crash and Runtime-health qualification.

sha256sum --check --ignore-missing SHA256SUMS
sudo dnf install ./fastdup-0.7.3-1.el10.x86_64.rpm

fastdup v0.7.2

fastdup v0.7.2

RPM fastdup-0.7.2-1.el10.x86_64.rpm for Rocky Linux 10 x86-64.

Veeam Fast Clone now reuses verified decoded Manifest nodes through the shared
adaptive RAM budget. Independent Samba clones run asynchronously with bounded
parallelism; conflicting file identities retain arrival order. Source and
target CLOSE wait for pending clone application, including interrupted clients.
This release also includes the post-v0.7.1 fixes for exact byte-granular clone
continuations: partial-cluster lengths and unaligned offsets are supported
without rounding or buffered DATA copies. Bounds and Integrity checks remain.

Total reduction now means current logical allocation divided by occupied DATA
and Metadata bytes. The WebUI distinguishes this from Exact Dedup since mount,
corrects older history from its recorded occupancy and leaves missing values
unavailable. Telemetry opens on cache effectiveness, includes Manifest Nodes,
offers five-minute/lifetime counters and optional raw columns, and separates
read avoidance, GC/scrub, latency, io_uring and checkpoints. Historical summaries
and disk rows consistently use the selected sample.

Cold Base-read admission retains the v0.7.1 learned policy: Similarity Sketch
quality, chunk size, independent compression and marginal savings determine
which feedback group is consulted. Measured savings/read-and-codec cost drive
admission, with warm-cache reuse and continuing exploration. It is an adaptive
utility estimate, not a calibrated probability of content equality.

Qualification and measurement limits.
No durable format migration is required. Learning and caches refill after the
runtime restart; incomplete background scrub retains its existing resume rules.

Download the RPM and SHA256SUMS, verify the hash and install:

sha256sum --check --ignore-missing SHA256SUMS
sudo dnf install ./fastdup-0.7.2-1.el10.x86_64.rpm

Use a backup maintenance interval to stop SMB clients, unmount the repository
cleanly, install, and restart the Agent, Control Plane, repository and Samba.
Replacing package files alone does not activate an already running runtime or
Samba worker. FastDup remains a research prototype; retain independent backups.

fastdup v0.7.1

fastdup v0.7.1

RPM fastdup-0.7.1-1.el10.x86_64.rpm for Rocky Linux 10 x86-64.

Veeam's 8 KiB Duplicate Extents request was rejected with invalid parameter
because the generated share and VFS default required 64 KiB alignment. Both now
use the volume's 4 KiB geometry. Existing enabled Integrity xattrs are presented
consistently with the current geometry, allowing old and new enabled files to
clone together without rewriting backup data. Bounds, pre-sizing, policy,
one-syscall and byte-integrity checks remain enforced.

Cold compression Base reads now use Similarity Sketch bands and measured
marginal savings versus read/codec cost. Bounded volatile learning, periodic
exploration and verified RAM reuse reduce speculative reads; false rejections
can sacrifice optional compression savings. Normal reads and scrub still verify
bytes fully. New GC & Reduction counters expose skips, backend attempts,
exploration, RAM reuse and useful Base trials.

Includes v0.7's storage overview, five-minute cache telemetry and durable
continuation of incomplete scrub rounds.

Download the RPM and SHA256SUMS, verify the hash, and install with:

sudo dnf install ./fastdup-0.7.1-1.el10.x86_64.rpm

Activation requires the updated repository runtime, agent/control services and
Samba module. Regenerate managed share configuration using the updated agent;
an explicit old clone alignment = 65536 overrides the new module default.
Reconnect SMB clients after activation. Arrange service restarts around backup
jobs. No repository reinitialization or data migration is required.

Binary RPM, source RPM and SHA-256 manifest are attached. Tests cover the exact
reported clone offsets, zero DATA I/O during clone/checkpoint, crash recovery,
legacy Integrity flags and cold-read admission. A complete Veeam synthetic-full
retry remains necessary: fastdup is still an experimental single-node prototype.

fastdup v0.7

fastdup v0.7

Release date: 7 September 2026. RPM: fastdup-0.7.0-1.el10.x86_64.rpm
for Rocky Linux 10 x86-64. Git tag: v0.7.

Storage overview

After repository initialization, Drives shows logical allocation versus
combined physical Metadata/DATA usage. Each volume shows its purpose, actual
filesystem usage and capacity, assigned device/UUID, backing drives and current
read/write load. The provisioning selector remains available before initialization.

Logical allocation excludes sparse holes and counts hard-linked files once.
The runtime samples cached inode counters in bounded, nonblocking batches without
reading DATA; a timestamp identifies the latest completed observation. Physical
usage includes filesystem overhead, indexes and data awaiting reclamation. It is
not a compression-only ratio. Missing runtime/mount statistics appear unavailable,
not as an empty repository or the usage of the underlying OS filesystem.

Cache telemetry

Switch between Last 5 minutes and Total since mount. The recent view uses
server-side differences of hits, misses and evictions, including the shared-budget
pools. RAM usage and budgets remain gauges at the selected sample time.

The agent retains at most 301 one-second observations. Runtime changes, counter
resets and clock reversal reset the baseline. Partial windows show their observed
duration; no activity has no hit-rate denominator. Browser reloads do not erase
the window. Historical samples retain the window calculated at collection time;
older samples without it remain explicitly unavailable.

Recovery and scrub

Includes the committed-Metadata fast mount and paced background verification,
adaptive shared cache budget, and durable resume of incomplete scrub rounds.
Saved full checks require reconciliation with current Container envelopes,
current Chunk requirements and independent Bases. Normal reads and offline scrub
retain full payload verification. See ADR 0092.

An upgrade cannot recover progress that an older runtime never persisted.
Completed rounds and incomplete rounds older than seven days start over at the
next mount. Saved progress is historical verification, not a permanent healthy flag.

Installation

Download the binary RPM and SHA256SUMS from this release, verify the RPM hash,
then install with sudo dnf install ./fastdup-0.7.0-1.el10.x86_64.rpm.
The release also includes a source RPM. Updating the running repository runtime
requires a service restart; arrange this around active backup jobs.

fastdup remains an experimental single-node prototype. This release does not
claim completed Veeam qualification or device-loss protection. Keep independent
copies of important data.

fastdup v0.6.4

Current RPM revision: 0.6.4-14

Normal startup now selects and validates the committed Metadata graph without
visiting the Container inventory. The previous revision still checked every
Container's structure before making the share available.

• Container inventory, structural and payload verification run in the paced
  background worker after mount. The worker also checks every Chunk required
  by the selected Commit, so a completely missing Container cannot escape the
  audit. RETIRING Locations cannot satisfy these requirements.
• Actual reads retain complete content and dependency verification. Online GC
  waits for successful initial scrub; integrity failures keep new writes closed.
• The selected Metadata graph must validate. A damaged newer committed graph
  does not silently roll back. Invalid WAL suffixes are durably truncated only
  after validating the exact selected prefix; accepted records are not rewritten.
• No format migration. Metadata graph validation and active-index recovery
  remain part of startup; DATA-tier disaster recovery retains full verification.
• Validation: the inventory regression drops from 232 DATA operations to zero;
  missing/corrupt DATA, dependent Bases, RETIRING Locations, corrupt newest
  Metadata, before/after-I/O recovery faults and checkpoint crash oracles pass.
  150 library/binary tests, 60 appliance integration tests, 56 maintenance tests,
  9 Recovery Checkpoint fault tests, production Clippy and the UI build passed.

VM validation: installed revision 14 mounted in 19.051 seconds, versus
about 13 minutes for the preceding structural startup. Committed Metadata
validation itself took 1.783 seconds. The pool had grown to 42,346 Containers.
SMB 3.1.1 write/flush/overwrite/append and byte-exact readback passed while the
initial scrub was running. These are operational observations, not a controlled
cold-cache or sustained-throughput benchmark.
VM evidence.

Install fastdup-0.6.4-14.el10.x86_64.rpm; checksums are in
SHA256SUMS-v0.6.4-14.txt. A repository process restart activates the new path.
ADR 0091
records the changed verification timing and remaining startup costs.

Previous RPM revision: 0.6.4-13

Revision 13 mounts after structural recovery and moves full DATA verification
into a paced background scrub (ADR 0090).

• Startup validates the committed Namespace and Manifest graph, Container seals,
  every Record header and Chunk Table, Recovery Index, structural commitment,
  and required Chunk/Base availability without reading DATA payloads.

• After mounting, a cancellable worker fully verifies a snapshot of published
  Containers at reduced CPU and idle I/O priority. Reads are limited to 256 KiB
  per operation, with pauses that increase during frontend activity.

• The WebUI shows scrub state, verified Containers and bytes, actual read bytes,
  and failures. Online GC waits for the initial pass to succeed. A verification
  failure persistently blocks new writes for that mount; demand reads continue
  to verify the data they return.

• Periodic Recovery Checkpoint copies verify the pinned committed Metadata graph
  and its copied image without repeatedly rereading all DATA. Metadata-loss
  restoration and offline scrub retain full content verification.

• Writer durability barriers and on-disk formats are unchanged. Includes the
  shared adaptive cache budget, telemetry and Veeam fixes from revision 12.

• Validation: structural corruption and payload-read exclusion, dependent Bases,
  actual paced scrub success/failure/cancellation, mutation admission, both
  checkpoint publication crash matrices, Namespace and maintenance regressions,
  telemetry/UI tests, TypeScript and production Clippy.

• Installed on the test VM: this startup mounted in about 13 minutes versus
  about 30 minutes for the preceding full-proof startup. SMB 3.1.1 write, flush,
  overwrite, append and byte-exact readback passed while scrub was running.
  These observations are not a controlled throughput benchmark.
  Deployment evidence.

Install fastdup-0.6.4-13.el10.x86_64.rpm; checksums are in
SHA256SUMS-v0.6.4-13.txt. Restart the repository process to activate the update.
Structural startup defers detection of payload corruption to demand reads and
scrub. A restart begins a fresh initial scrub pass; progress is not a durable
verification certificate.

Previous RPM revision: 0.6.4-12

Revision 12 adds a shared adaptive RAM budget and fixes two failures reproduced
from the Veeam test VM.

• Rebuildable caches compete by recent measured benefit under a sampled 92% RAM
  ceiling. DATA-fallback caches receive priority over Metadata-fallback caches;
  a donor's reservation is released only after eviction.
• The WebUI shows the common budget and each pool's tier, hit rate, resident,
  target and reserved bytes. Older telemetry remains readable.
• Large logical Manifest layouts are partitioned into bounded physical leaves.
  The exact InvalidObjectLength(20620288) append reproducer now passes;
  mixed shrink/header updates keep unchanged tree paths and DATA proofs.
• GC bootstrap uses one directory-name snapshot for row count and generation,
  avoiding RowCountMismatch during concurrent Container publication.
• Validation includes cache competition/concurrency/pressure, Metadata fault
  injection, byte-exact recovery and scrub, all 56 end-to-end maintenance tests,
  telemetry/UI tests, TypeScript and production Clippy. Local warm Similarity
  replay improves from 8.324 ms to 0.750 ms; this is not a Veeam throughput claim.
• No durable-format migration. Includes all earlier 0.6.4 revisions.

Install fastdup-0.6.4-12.el10.x86_64.rpm; checksums are in
SHA256SUMS-v0.6.4-12.txt. Existing repository processes must be restarted to use
the fixes. Restart recovers the last complete commit, not an uncommitted cut.

fastdup v0.6.4

Package version 0.6.4 · 6 September 2026 · Rocky Linux 10 · x86-64

English

• Share free space now equals logical quota minus logically allocated bytes.
  Empty thin-provisioned shares no longer appear partly used by the physical
  storage limit. Physical write admission remains independent.
• Fix nested SMB directory creation for guest sessions (including VeeamZIP).
• Add measured runtime AVX-512 acceleration with AVX2/scalar fallback.
• Fix empty drive inventory after interactive login or the initial password change.
• Fix guest and authenticated SMB access: a dedicated non-login fastdup-smb
  identity owns share roots; Samba still enforces the configured user/group ACLs.
  Guest-enabled shares include the guest principal. Generated includes restore
  the global parser context, and RPM installation enables samba_share_fusefs
  while SELinux stays enforcing.
• Add Settings → SMB users to list and create SMB accounts. Credentials go
  through the authenticated, CSRF-protected API to the privileged local agent.
  Passwords go to smbpasswd through stdin, never into argv, files, Jobs or audit
  records. Existing Unix/SMB identities cannot be overwritten. New accounts have
  no shell or home directory. Select them in the share's allowed users.
• Adapt an oversized Small-File quota to 20% of Metadata filesystem capacity,
  rounded down to KiB and bounded by the protected commit reserve. Existing
  fitting limits remain unchanged. The journal and WebUI report the requested
  and effective values. Physical exhaustion or an impossible commit reserve
  still fails closed; the quota does not weaken storage admission.

WebUI accounts, Linux accounts and SMB passwords are separate. Windows clients
may prohibit guest authentication; authenticated SMB accounts support those
clients without weakening client signing policy. No durable format changes. AVX-512 evidence.

Deutsch

• Freier Share-Speicher ist Quota minus logisch belegte Bytes. Physische
  Poolgrenzen lassen leere Shares nicht mehr belegt erscheinen.

• Verschachtelte Verzeichnisse unter VeeamZIP sind auch als Gast beschreibbar.

• AVX-512 beschleunigt Similarity-Votes, Sparse-XOR und gleichförmige SeqCDC-
  Eingaben. Wechselnde SeqCDC-Eingaben nutzen weiterhin das schnellere AVX2.

• Laufwerke erscheinen direkt nach Login und initialem Passwortwechsel.

• Gast- und authentifizierter SMB-Zugriff funktionieren mit getrennten
  Management- und SMB-Dienstidentitäten. Die RPM-Installation richtet die
  SELinux-FUSE-Berechtigung ein, ohne SELinux abzuschalten.

• Unter Einstellungen → SMB-Benutzer lassen sich SMB-Konten mit Passwort
  anlegen. Danach den Benutzer in der Freigabe erlauben. Bestehende Konten werden
  nicht überschrieben; WebUI- und SSH-Zugang bleiben getrennt.

• Auf kleinen Metadata-Volumes wird ein zu großes Small-File-Limit automatisch
  auf 20% der Dateisystemkapazität begrenzt. Journal und WebUI zeigen eine Warnung
  mit angefordertem und wirksamem Limit. Die Commit-Reserve bleibt geschützt.

Validation

See v0.6.4 validation for tests and VM checks.

fastdup v0.6 — Online Similarity und optimierter Datenpfad

fastdup v0.6

English · Deutsch

English

Package version 0.6.0 · 5 September 2026 · Rocky Linux 10 · x86-64

Download the binary RPM · SHA-256 checksums · Source RPM

Changes since v0.5

• Persistent online similarity: New independent chunks become compression candidates during the current mount. No remount or offline rebuild is needed.
• Advanced Reduction per SMB share: Disable it, enable it, or inherit the repository default. New WebUI shares start with Advanced Reduction disabled. Existing dependent data remains readable after disabling it.
• Durable depth-one codecs: ZSTD_PREFIX and Sparse-XOR use an independently decodable base and a shared bounded trial policy. Dictionary compression remains a research path.
• Optimized data path: Parallel preparation of fragmented chunks, SIMD fingerprints, fewer buffer copies during reads and container assembly, finer-grained file locking, and adjusted CPU admission.
• More responsive management: History aggregation with bounded memory use, bounded historical queries, and separate execution of sampling, maintenance, and network requests.
• Updated documentation: German and English READMEs, a restructured GitHub website, current A/B measurements, and a sourced explanation of Rust's memory-safety benefits.

Measurements

The reports document development builds leading to v0.6; they are not benchmarks of the newly packaged RPM.

Workload	Normal reduction	Advanced Reduction
Three identical ISO uploads over loopback SMB, median	1,061.0 MiB/s	941.5 MiB/s
Same SMB series, storage savings including metadata	67.823%	67.904%
50 Linux 6.12 TAR versions, total repository	10.93 GiB	3.02 GiB
Same Linux corpus, copy + fsync	202.48 MiB/s	116.92 MiB/s

Identical copies mostly benefit from Exact Dedup. For the Linux corpus, Advanced Reduction saves an additional 72.35% of total space, at 42.25% lower write throughput. The Linux and SMB series use different development builds. The reports describe hardware, binary hashes, measurement limits, and the absence of full target readbacks: SMB series, Linux A/B.

Installation

curl -fLO https://github.com/ThatIsCraZy/fastdup/releases/download/v0.6/fastdup-0.6.0-1.el10.x86_64.rpm &&
curl -fLO https://github.com/ThatIsCraZy/fastdup/releases/download/v0.6/SHA256SUMS &&
sha256sum --check --ignore-missing SHA256SUMS &&
sudo dnf install ./fastdup-0.6.0-1.el10.x86_64.rpm &&
sudo systemctl enable --now fastdup-agent.service fastdup-control.service

WebUI: https://<appliance-host>:8080/. The initial certificate is self-signed. First login: admin / fastdup01.; set a new password of at least twelve characters before making management changes.

Installation does not format drives or start a repository automatically. Subsequent provisioning in the WebUI erases both selected devices. Setup requires separate metadata and DATA devices. The Samba module is built against Samba 4.23.5, which the RPM requires.

The source RPM contains prepared package files, including binaries. The complete source code and build script are available at tag v0.6.

Existing installations and limitations

fastdup remains a research prototype. Before changing packages, cleanly unmount the repository and retain an independent copy of the data. The pre-production formats do not yet provide a general upgrade or downgrade guarantee; a successful RPM upgrade does not establish repository compatibility. Migration of an existing v0.5 repository was not qualified for this release. See the format policy and writer downgrade fence.

No built-in device-loss protection, replication, WORM, or performance SLA. Samba Fast Clone remains experimental and is not Veeam-qualified. Rust reduces memory-error risks but does not replace a full security review or production qualification.

Release validation

• Rust workspace, including all targets: 744 tests passed, 0 failures, and 16 intentionally ignored manual benchmarks/qualifications across 127 suites.
• Clippy passed across the entire workspace and all targets with -D warnings.
• WebUI: 28 tests passed across six files; TypeScript and Vite builds succeeded.
• Samba VFS contract test passed; module built against Samba 4.23.5.
• RPM digests and the RPM transaction in test mode passed. The package was not installed on the running system during these checks.
• GitHub website: German and English checked at widths of 320, 390, 768, and 1440 pixels, including navigation, images, clipboard, and disabled browser storage.

---

Deutsch

Paketversion 0.6.0 · 5. September 2026 · Rocky Linux 10 · x86-64

Binary-RPM herunterladen · SHA-256-Prüfsummen · Source-RPM

Änderungen seit v0.5

• Permanente Online-Similarity: Neue unabhängige Chunks werden im laufenden Mount als Kompressionskandidaten aufgenommen. Ein Remount oder Offline-Rebuild ist dafür nicht mehr nötig.
• Advanced Reduction je SMB-Freigabe: Ausschalten, einschalten oder den Repository-Standard erben. Neue WebUI-Freigaben starten mit ausgeschaltetem Advanced-Pfad. Bereits gespeicherte abhängige Daten bleiben nach dem Abschalten lesbar.
• Dauerhafte Depth-1-Codecs: ZSTD_PREFIX und Sparse-XOR nutzen eine unabhängig dekodierbare Basis und eine gemeinsame begrenzte Trial-Policy. Dictionary-Kompression bleibt Forschung.
• Optimierter Datenpfad: Parallele Vorbereitung fragmentierter Chunks, SIMD-Fingerprints, weniger Pufferkopien bei Reads und Containeraufbau, feinere Dateisperren und angepasste CPU-Admission.
• Reaktionsfähigere Verwaltung: Historien-Aggregation mit begrenztem Speicherbedarf, begrenzte historische Abfragen und getrennte Ausführung von Sampling, Wartung und Netzwerkanfragen.
• Überarbeitete Dokumentation: Deutsche und englische README, neu gegliederte GitHub-Webseite, aktuelle A/B-Messungen und ein belegter Abschnitt zum Speichersicherheitsvorteil von Rust.

Messungen

Die Berichte dokumentieren Entwicklungsbuilds auf dem Weg zu v0.6; sie sind kein Benchmark des neu paketierten RPMs.

Workload	Normale Reduktion	Advanced Reduction
Drei identische ISO-Uploads über Loopback-SMB, Median	1.061,0 MiB/s	941,5 MiB/s
Gleiche SMB-Serie, Speicherersparnis inkl. Metadaten	67,823 %	67,904 %
50 Linux-6.12-TAR-Versionen, Repository gesamt	10,93 GiB	3,02 GiB
Gleicher Linux-Korpus, Copy+fsync	202,48 MiB/s	116,92 MiB/s

Identische Kopien profitieren vor allem von Exact Dedup. Beim Linux-Korpus spart Advanced Reduction 72,35 % zusätzlichen Gesamtplatz, bei 42,25 % geringerem Schreibdurchsatz. Die Linux- und SMB-Serien verwenden unterschiedliche Entwicklungsbuilds. Die Berichte nennen Hardware, Binary-Hashes, Messgrenzen und die fehlenden vollständigen Ziel-Readbacks: SMB-Serie, Linux-A/B.

Installation

curl -fLO https://github.com/ThatIsCraZy/fastdup/releases/download/v0.6/fastdup-0.6.0-1.el10.x86_64.rpm &&
curl -fLO https://github.com/ThatIsCraZy/fastdup/releases/download/v0.6/SHA256SUMS &&
sha256sum --check --ignore-missing SHA256SUMS &&
sudo dnf install ./fastdup-0.6.0-1.el10.x86_64.rpm &&
sudo systemctl enable --now fastdup-agent.service fastdup-control.service

WebUI: https://<appliance-host>:8080/. Das erste Zertifikat ist selbstsigniert. Erster Login: admin / fastdup01.; vor Management-Änderungen ein neues Passwort mit mindestens zwölf Zeichen setzen.

Die Installation formatiert keine Datenträger und startet kein Repository automatisch. Die spätere Provisionierung in der WebUI löscht die beiden ausgewählten Geräte. Für die Einrichtung werden getrennte Metadata- und DATA-Geräte benötigt. Das Samba-Modul ist gegen Samba 4.23.5 gebaut; das RPM fordert diese Version an.

Der Source-RPM enthält die vorbereiteten Paketdateien einschließlich Binärdateien. Der vollständige Quellcode und das Build-Skript stehen unter Tag v0.6 bereit.

Bestehende Installationen und Grenzen

fastdup bleibt ein Forschungsprototyp. Vor einem Paketwechsel das Repository sauber aushängen und eine unabhängige Kopie der Daten behalten. Die Vorproduktionsformate haben noch keine allgemeine Upgrade- oder Downgrade-Garantie; ein erfolgreicher RPM-Paketwechsel beweist keine Repository-Kompatibilität. Es wurde keine Migration eines bestehenden v0.5-Repositories für dieses Release qualifiziert. Siehe Formatpolitik und [Writer-Downgrade-Sperre](https://github.com/ThatIsCraZy/fastdup/blob/v0.6/docs/adr/0071-fence-writer-d...

fastdup 0.5.0 · Rocky Linux 10 RPM

fastdup 0.5.0

English · Deutsch

English

Direct download

Download fastdup-0.5.0-1.el10.x86_64.rpm
Rocky Linux 10 · x86-64 · 8.2 MB

SHA-256 checksums · Source RPM

First installable RPM release, including:

• FUSE repository runtime and offline maintenance
• HTTPS control plane with an embedded WebUI
• Privileged provisioning agent
• Samba VFS adapter fastdup.so, built and tested against Samba 4.23.5
• systemd units and resource slices
• io_uring, XFS project quota, and runtime configuration
• Dynamic small-file placement policy

Installation

curl -LO https://github.com/ThatIsCraZy/fastdup/releases/download/v0.5/fastdup-0.5.0-1.el10.x86_64.rpm
sudo dnf install ./fastdup-0.5.0-1.el10.x86_64.rpm
sudo systemctl enable --now fastdup-agent.service fastdup-control.service

Then open the WebUI at https://<appliance-host>:8080/.

Caution: Storage provisioning through the WebUI erases the selected devices. The repository is therefore not started automatically.

---

Deutsch

Direkter Download

fastdup-0.5.0-1.el10.x86_64.rpm herunterladen
Rocky Linux 10 · x86-64 · 8.2 MB

SHA-256-Prüfsummen · Source-RPM

Erstes installierbares RPM-Release mit:

• FUSE-Repository-Runtime und Offline-Wartung
• HTTPS-Control-Plane mit eingebetteter WebUI
• privilegiertem Provisionierungs-Agent
• Samba-VFS-Adapter fastdup.so, gebaut und getestet gegen Samba 4.23.5
• systemd-Units und Ressourcen-Slices
• io_uring-, XFS-Projektquota- und Runtime-Konfiguration
• dynamischer Small-File-Platzierungsrichtlinie

Installation

curl -LO https://github.com/ThatIsCraZy/fastdup/releases/download/v0.5/fastdup-0.5.0-1.el10.x86_64.rpm
sudo dnf install ./fastdup-0.5.0-1.el10.x86_64.rpm
sudo systemctl enable --now fastdup-agent.service fastdup-control.service

Danach die WebUI unter https://<appliance-host>:8080/ öffnen.

Achtung: Die Storage-Provisionierung über die WebUI löscht die ausgewählten Geräte. Das Repository wird deshalb nicht automatisch gestartet.